Indexing never completes on ReFS drive

[justtoby]

I have an ReFS drive with several hundred thousand files. Everything never completes the index on the drive. There are NTFS drives on this system and it has no problems indexing those.

I've let it run for several hours , but the UI just reads "scanning ...."

This drive has been scanned/indexed fine in the past

I disabled scanning all the other drives except the refs volume to try to get some meaningful info from the debug.

from the debug log I saw -

Code: Select all

copy config
fire create events
start rebuild thread
create thread
EVENT: 00000001400531a0 0000000000598490
EVENT: 0000000140053150 0000000000598490
rebuild thread started
build fs list
new results 0
build indexes
open volume \\?\Volume{x}
opened 788 0.000083
EVENT: 000000014005cbd0 0000000000598490
refs sector size 512
EVENT: 0000000140053150 0000000000598490
new results 0
serial number x, sector size 512, sectors per cluster 128, version 3.1
failed to open dir 00000000000000000000000000000600 87
EVENT: 00000001400531a0 0000000000598490
failed to open dir 00000000000000000000000000000600 87
EVENT: 00000001400531a0 0000000000598490
MSG: 00000000000405e0 031f 0000000000000001 0000000000000000
failed to open dir 00000000000007060000000000000000 5
MSG: 00000000000405e0 0402 0000000000000000 0000000000000000
MSG: 00000000000406a2 c093 fffffffffffffffc 0000000000000000
MSG: 00000000000405e0 c093 fffffffffffffffc 0000000000000000
MSG: 00000000000406a2 c093 fffffffffffffffc 0000000000000000
MSG: 0000000000000000 c04a 0000000000000010 0000000000000000
MSG: 00000000000406a2 c093 0000000000000000 0000000000000000
MSG: 00000000000505f8 c093 0000000000000000 0000000000000000
MSG: 00000000000505f8 1401 0000000000000000 0000000000000000
MSG: 00000000000505f8 1401 0000000000000000 0000000000000000

Looks like it's having issues opening the directories that contain the files?

Any ideas on how to further troubleshoot this?

[void]

Thanks for the debug logs.

The failed to open dir messages are normal.
ReFS volumes usually start at FileID 00000000000000000000000000000600 or 00000000000006000000000000000000.
00000000000006000000000000000000 in your case.
Access denied (error 5) when reading 00000000000007060000000000000000 is interesting. -Does running Everything as an admin help?

When ReFS indexing finishes, you should see the following in the debug log:
x seconds
where x is the number of seconds taken to index the ReFS volume.

Everything is most likely stuck scanning a folder junction with a circular reference.
Is Everything using any CPU or is the memory usage growing slightly if you leave Everything running with the status bar showing "scanning..."

Could you please check your refs volume for a folder junction pointing to itself or a parent:

• from a command prompt, select your refs drive:
  d:
  where d: is your refs volume.
• Change directory to the root:
  cd /
• Run the following command:
  dir /al /s | findstr "<JUNCTION>"
• Please also try running the following command:
  dir /al /s | findstr "<SYMLINKD>"

You should see something like:
08/12/2017 23:04 <JUNCTION> Junction Name [D:\path\to\real\folder]

https://superuser.com/questions/823959/how-to-view-all-the-symbolic-links-junction-points-hard-links-in-a-folder-using

[justtoby]

HI, Thanks for the prompt reply.

Does running Everything as an admin help?

I am running it as admin.

Is Everything using any CPU or is the memory usage growing slightly if you leave Everything running with the status bar showing "scanning..."

Very low CPU. Mostly fluctuating between 0% and 0.1% occasionally it will spike to 0.5%. The memory seems to climb very slowly. Maybe 100K every 2-3 mins.

Run the following command:
dir /al /s | findstr "<JUNCTION>"
Please also try running the following command:
dir /al /s | findstr "<SYMLINKD>"

Both those result in file not found after some time

It may be worth noting that this volume is mounted on a folder, not sure that would be an issue as the NTFS volumes are mounted the same. Changing that mount back to a letter doesn't change the result.

Watching the debug log I notice that the only thing being reported is

Code: Select all

MSG: 00000000000b03ba 0113 000000000000002d 0000000000000000

Just repeating that over and over again

[NotNull]

Try it with:

Code: Select all

dir /al /s D:\

(where D:\ is the ReFS volume)
You will probably some circular reference like D:\path\FolderX\another folder\FolderX\....

You can pause the output with the pause button or by clicking with the mouse in the CMD window.
Resume by pressing the space bar

Alternative:

Code: Select all

dir /al /s /p D:\

That will pause after each page of output.

[void]

Please try Everything 1.4.1.952:
Everything-1.4.1.952.x64.zip
Everything-1.4.1.952.x86.zip

I've added more ReFS indexing debug information to this version.
What is shown in the Debug Console while the status bar is showing "scanning..."?

[justtoby]

Thanks for that version, I've installed it and run it with the debug option.

I can see it indexing the files but very slowly, like it's crawling the directories. I would post the debug here, but some of the information may be considered confidential. I'll be happy to send the non-sensitive parts to you if you think it will help diagnosing the issue.

I should mention that all these these drives are Amazon Storage Gateways and not 'local'. Thus the access time and read speed is limited. I suspect that it will eventually finish, however it looks like it will take many hours.

What seems strange to me is that all was well with it until early last week when Everything became unresponsive and I had to kill it and rebuild the data base.

There are a total of 7 storage gateway drive on this server with about 40 million files. A normal rebuild time would be around 45 mins to an hour. The 6 NTFS drives complete in an expected fashion. It just the ReFS drive that seems newly slow. Based on what I see in the version provided, I can now see why.

I'm almost to the point where I'm going to try to replace it with an NTFS drive and see if I get better results. Thanks you for your prompt attention, I'm happy to keep troubleshooting with you if you have other ideas.

The dir option that NotNull suggested is still running, which I would expect considering the time it takes to enumerate all the files on that drive.


edit:

Code: Select all

dir /al /s /p D:\

Completed with no results

[void]

ReFS does not have a Master File Table (MFT).
Indexing ReFS volumes is slow.

Did you recently add a folder with many subfolders?
If possible, try compressing folders with many subfolders into zip files.

[justtoby]

That volume is an archive location that receives files daily. I suppose it's possible that the number of files since the last index has grown to the point where the difference in scan times is noticeable from the NTFS drives.

I'll just relocate the files to a new NTFS volume as it will only grow in the future.

Thanks much for the assistance.