Obsolete "c:\" index entries not disappearing even after force rebuild

If you are experiencing problems with "Everything", post here for assistance.
Post Reply
xxz
Posts: 2
Joined: Mon May 13, 2019 8:25 am

Obsolete "c:\" index entries not disappearing even after force rebuild

Post by xxz »

I encounter this problem again and again, with various "c:\" entries and on various occasions, and as said, even when, after deleting those entries (from within EV or from a file manager), I then do a force rebuild, close EV, restart EV, these entries continue to appear in the EV list if they meet the current search.

So many a times, I even wonder if they have successfully been deleted, but they have, since they don't appear anymore in my file manager(s which are all set to "show hidden files, show system files"), and if I try to "open" those files from within their respective EV list entry, I get an error message "Windows can't find...".

Thus, it seems that even a force rebuild will not correctly rebuild the index, for "c:\" at least (I have not seen this with other other drives, but with "c:\", again and again, and for entries in various "c:\" folders.

If I understand the "force rebuild" correctly, I cannot direct it to just a specific drive, but it will rebuild the indexes of all drives which are connected at the time I trigger the command.

My EV settings for "c:\" are:
Indexes - NTFS - C:
(Automatically include new fixed volumes yes)
Include in database yes
Enable USN Journal yes
Max size 32768 KB
Allocation delta 8192 KB
Load recent changes from USN journal yes
Monitor changes yes

Should I set the numbers 32768 and 8192 to numbers much bigger, perhaps to about 100,000 and about 30,000, respectively? But then, a "force rebuild" would probably delete the whole index for the drive in question, so that logically, after the "force rebuild", entries would be missing from the list then if the allocated space to the list isn't big enough, but previously-deleted list entries should not continue to appear then though?

Speaking of version 1.4.1.1005 x64, with W10 x64 English.

I'd like to add that I use EV, for my user data (on "d:\", etc.), for looking-up files, and that works very well, but that for "c:\", I try to use EV to confirm that there is no leftover-data of obsolete programs, and/or to delete such leftover-data, in case, so this task is hampered by the above-mentioned, continuing appearing of deleted data on "c:\"; I think - I'm not sure, haven't tried - that days later, those obsolete list entries will finally vanish, but I'd prefer them to vanish immediately, or then at least after that "force rebuild"; this being said, for looking up data, EV is simply invaluable!
void
Developer
Posts: 16748
Joined: Fri Oct 16, 2009 11:31 pm

Re: Obsolete "c:\" index entries not disappearing even after force rebuild

Post by void »

I encounter this problem again and again, with various "c:\" entries and on various occasions, and as said, even when, after deleting those entries (from within EV or from a file manager), I then do a force rebuild, close EV, restart EV, these entries continue to appear in the EV list if they meet the current search.
To remove offline volumes:
  • In Everything, from the Tools menu, click Options.
  • Click the NTFS tab on the left.
  • Select an offline volume and click Remove.
    (Consider enabling Automatically remove offline volumes to automate this process)
  • Click OK.

If I understand the "force rebuild" correctly, I cannot direct it to just a specific drive, but it will rebuild the indexes of all drives which are connected at the time I trigger the command.
You can force a rebuild for a single drive with the /reindex <path> search command.
For example:
  • In Everything, type in the following search and press ENTER:
    /reindex c:
    where c: is the drive to reindex.

My EV settings for "c:\" are:
Indexes - NTFS - C:
(Automatically include new fixed volumes yes)
Include in database yes
Enable USN Journal yes
Max size 32768 KB
Allocation delta 8192 KB
Load recent changes from USN journal yes
Monitor changes yes


Should I set the numbers 32768 and 8192 to numbers much bigger, perhaps to about 100,000 and about 30,000, respectively?
If Everything is constantly reindexing an out-of-date drive, please consider increasing the drives maximum journal size to 131072 KB.
Leave allocation delta as 8192 KB.
If you change the maximum size of the journal, the change will not occur until the next trim (allocation delta is met).


But then, a "force rebuild" would probably delete the whole index for the drive in question, so that logically, after the "force rebuild", entries would be missing from the list then if the allocated space to the list isn't big enough, but previously-deleted list entries should not continue to appear then though?
Everything will keep your existing indexes if they are offline.



If Everything is missing changes to your C: drive, please make sure you are using NTFS indexing under Tools -> Options -> NTFS -> C: -> Include in database.

Please ensure you are not excluding any items under Tools -> Options -> Exclude.
xxz
Posts: 2
Joined: Mon May 13, 2019 8:25 am

Re: Obsolete "c:\" index entries not disappearing even after force rebuild

Post by xxz »

Thank you very much for your kind and detail replay, really valuable info in there, especially the hint for the selective index rebuild, since often, I'd like to rebuild the indexes for c: and d: but in a moment I have connected several external drives which do not need re-indexing in-between, too.

(And please excuse my late reply, I had not found my login data.)

But there is a misunderstanding: I do not have problems with EV not updating indexes, especially of external drives, and in order to include NEW content into a non-up-to-date index, but my problem is the internal, system drive c:\, where, even AFTER a forced index rebuild, entries of deleted files appear again and again.

That's why I mentioned my settings for the internal c: drive, here they are again:

My EV settings for "c:\" are:
Indexes - NTFS - C:
(Automatically include new fixed volumes yes)
Include in database yes
Enable USN Journal yes
Max size 32768 KB
Allocation delta 8192 KB
Load recent changes from USN journal yes
Monitor changes yes

so that you could check if those settings are correct, or if perhaps one of them is NOT correct, hence the described problem; I know those settings are specific to every drive.

Thinking about it, perhaps

Enable USN journal yes, and especially
Load recent changes from USN journal yes

might be the culprit for the problem? Since I don't grasp what that USN journal is, after all, and I suppose, I might be wrong of course:

Without those "yes", or at least without the second "yes", a forced EV index rebuild for the drive in question would trigger EV to rebuild the index for that drive from scratch, whilst the same forced rebuild for that drivem WITH those "yes", would be done by EV, using, more or less or even entirely, some system index / file list, which is that "USN journal".

So the problem would probably lie in the W10 "USN journal" which, obviously, is NOT correctly updated, for deletions at least (but perhaps it's always correct for new additions), and thus, EV builds a, now partly wrong, index, from what the obviously not correctly in time updated USN journal tells it, and thus, for c: at least, I should probably switch those "yes" to "no".

And you said, "If Everything is constantly reindexing an out-of-date drive", which is not my problem, but I said, even after doing a force rebuild, by manually triggering the force rebuild command, the obsolete entries reappear, instead of vanishing by the force rebuild... as said, I now suspect my USN journal settings to be responsible for this, so in the end, it seems that W10 (which should be "responsible" for the USN journal) does not correctly update its USN journal, i.e. doesn't delete entries there in time (but later on only, perhaps every 2 or 3 days or such).

As for your hint to check if I might have excluded something, my setting is the default one, i.e. "Enable exclude list", but which is entirely empty - I had never used that -, so I now even un-checked the "Enable" setting.



As for disabling offline volumes - as said, I don't have problems insofar - I wish to add that the appearance of offline volumes, with their EV-indexed data, even when they are offline, is, at the end of the day, the absolutely BEST thing in EV, since all the time, I use this EV search, specifying the drives in question in case, in order to check IF I've got some file, from the web for example, anywhere on my external drives, and that without having to connect them for this check.

Thus, I'm extremely grateful for this incredibly valuable feature and would never ever think of excluding external drives from my search results, a simple "c:" or "d:" within my search then allowing to specify my local drives if I want to search locally!

As said, EV enabling its users to even check, any time, what's on their non-connected drives - or on which external drive it is, in case -, is pure gold: thank you so much for this fantastic tool!
void
Developer
Posts: 16748
Joined: Fri Oct 16, 2009 11:31 pm

Re: Obsolete "c:\" index entries not disappearing even after force rebuild

Post by void »

My EV settings for "c:\" are:
Indexes - NTFS - C:
(Automatically include new fixed volumes yes)
Include in database yes
Enable USN Journal yes
Max size 32768 KB
Allocation delta 8192 KB
Load recent changes from USN journal yes
Monitor changes yes
You settings are correct.

I recommend unchecking Load recent changes from USN journal.
If you are not using recent changes, enabling Load recent changes from USN journal will only slow down Everything on system startup.



The USN Journal has nothing to do with Everything.
It is a transaction journal maintained by the NTFS driver.
NTFS can use the transaction journal to recover errors.
It's no longer possible to disable the USN Journal, as the OS will recreate it on boot.

Everything doesn't need the USN Journal to index your volume.
Everything uses the USN Journal to keep your indexes up-to-date.



Can you give an example of a file that stays even after a force rebuild.
The file might really exist, but is hidden from Windows Explorer.

If these inaccessible files persist, we can add them to your exclude list under Tools -> Options -> Exclude.
A common folder to add is
C:\System Volume Information
which will contain restore points and other system information that is normally inaccessible.
Post Reply