Question

We have a fairly large disk array with roughly 2-3 million XML files on it. The disk is formatted with NTFS and we would like to search the filesystem using wildcards. So something like * SomePartOfTheFilename * would be a typical search query.

We are using .Net and are finding that using DirectoryInfo appears to be slow.

DirectoryInfo directoryInfo = new DirectoryInfo(directory);

List<FileInfo> fileInfos = directoryInfo.GetFiles(searchString, SearchOption.AllDirectories).ToList();

Using Loops and recursion is also very slow.

Is there a lower level API call we can use to directly search the NTFS index?

Using dir * SomePartOfTheFilename * /s from the command line is almost instant. Is there something there that can be leveraged?

Answer 1

I m not sure if you can use the Indexing service, but it may be handy for what you are trying to do:

http://msdn.microsoft.com/en-us/library/ee805985%28VS.85%29.aspx

http://www.codeproject.com/KB/database/Indexing_Service_HOW-TO.aspx

It allows you to create complex queries against the NTFS index of the files on a computer.

Answer 2

You may use MFT directly (See: NTFS Wiki). That is data table where all information about files is located. You can see the structure of MFT for example here or here. The Windows API ends up in the same table so you can alternatively try to speed the searches up to guarantee that it will be paged in memory before search (simple read of e.g. c:$Mft is enough).

友情链接