Being able to search while Everything is indexing

[Phantom1472]

Sometimes when I launch Everything, there is a long wait time while it's indexing the entire file system, why can't I still search the previous index while this is happening? most of the FS is the same...

Obviously the results would be out of date until the indexing is finished, but it's still useful in my opinion, instead of just a blank screen.

I'm using Version 1.5.0.1331a (x64), Win 10.

Thanks!

[void]

Thank you for the issue report Phantom1472,

You should be able to search while Everything 1.5 updates in the background.

However, there's is likely something wrong your with Everything database or your file systems are being spammed with changes.

What is the last rebuild reason shown under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason?
Please check changes to your file systems under Index -> Index Journal.
If there are millions of changes since Everything was started, Everything may need to reindex your drives.



Please make sure Everything is installed correctly and running as a standard user:

• In Everything, from the Tools menu, click Options.
• Click the General tab on the left.
• Check Store settings and data in %APPDATA%\Everything. (-if unchecked, Everything might be unable to save your database to disk)
• Uncheck Run as administrator.
• Check Everything Service. (Please make sure this is tick-checked and not square-checked)
• Click OK.
• Exit Everything (right click the Everything tray icon and click Exit).
• Restart Everything.

[Phantom1472]

Sorry, my last post had a mistake, Everything was not indexing, It said "Scanning local drive C:" or something like that, while that was happening, the file list was completely blank.

What is the last rebuild reason shown under Tools -> Debug -> Statistics -> Build -> Last Rebuild Reason?:
Out of date index is available

If there are millions of changes since Everything was started, Everything may need to reindex your drives.:
only ~2000 from what I see there.

Please make sure Everything is installed correctly and running as a standard user:
everything was as you said.

[raccoon]

I would like to put in a request that Everything ought to work (be able to search) while the index is being Sorted. Fast Sorting should happen in the background, in parallel, and be non intrusive. Currently it adds a lengthy delay before the index is available to be searched, especially with millions of objects to be sorted. Only Name/Path sort should be necessary before the database becomes accessible.

[void]

Thank you for your reply Phantom1472,

Out of date index is available

Could you please take note of your USN Journal ID values:

• In Everything, from the Tools menu, under the Debug submenu, click Statistics.
• Could you please copy these statistics and save them in a text file somewhere on your PC.

The next time you see Everything rebuild, could you please:

• Wait for the rebuild to complete.
• In Everything, from the Tools menu, under the Debug submenu, click Statistics.
• Could you please copy these statistics and save them in a new text file somewhere on your PC.
• Please compare the USN Journal ID values for each volume.
• Has one changed?
  -If so, there might be a program that is deleting/rebuilding your USN Journal.

I will add more information to the "Out of date index is available" message to make this easier in the next alpha update.

I would like to put in a request that Everything ought to work (be able to search) while the index is being Sorted.

I will consider sorting in the background.

Thank you for the suggestion.

Ideally, Everything should only be indexing once.

[raccoon]

I will consider sorting in the background.
Thank you for the suggestion.
Ideally, Everything should only be indexing once.

Tis true, but sorting often takes much longer than just indexing NTFS volumes. Especially noticed when reproducing crash bugs (like startwith:), and agonizing wait for sort to complete.

[void]

Everything 1.5.0.1339a adds more information to Tools -> Debug -> Statistics -> Build -> Rebuild Reason.

This might help find the reason for rebuild.

The next time Everything reindexes, could you please:
Wait for Everything to finish reindexing.
From the Tools menu, under the Debug submenu, click Statistics.
Under Build, what is shown as the rebuild reason?

[ChrisGreaves]

From the Tools menu, under the Debug submenu, click Statistics.
Under Build, what is shown as the rebuild reason?

Hitching a free ride here. I thought I'd take a look and found:-

Code: Select all

Build
Count:	1
Total duration:	00:00
Minimum duration:	0.681403 seconds
Maximum duration:	0.681403 seconds
Average duration:	0.681403 seconds
Last duration:	0.681403 seconds
Last build date:	2023-03-08 7:07 PM
Last rebuild reason:	CRC Error

... which puzzles me a lot. I took delivery of this HP15 a week ago, Win11 and an SSD (at long last!) and didn't expect to see any mention of a Cyclic Redundancy Check for at least another month

Are CRC reasons common? I can understand that a cosmic-ray might have struck a wire during one of trillions of CRC checks and the system, naturally, re-checked. As much as a dropped packet on the internet just means a request to resend that packet. We would never know about it.

Or am I about to get a break from work while the laptop goes back to BestBuy and a replacement sent out?

As well: If the "cosmic ray"/"accident" is the cause for rebuilds, is the rebuild recommenced from scratch, or just that one packet of data from the disk drive?

Thanks, Chris

[void]

I have never seen a CRC Error on my testing PCs.

From what I have seen on the forums, CRC Error usually occurs when your PC shuts down and SSD buffers are not flushed to disk.
Your Everything database is most likely filled with zeroes.

Everything will tell the system to flush the database to disk on system shutdown.
It seems some hardware is ignoring this flush to disk command.

[ChrisGreaves]

I have never seen a CRC Error on my testing PCs.

So now in one teeny-tiny area I am more advanced than are you?!!???

From what I have seen on the forums, CRC Error usually occurs when your PC shuts down and SSD buffers are not flushed to disk.
Your Everything database is most likely filled with zeroes.

... and presumably Everything detects the corrupt db and immediately takes steps to correct it by rebuilding; right?

It seems some hardware is ignoring this flush to disk command.

Maybe it is time for me to buy a new laptop? :flee:

Code: Select all

Build
Count:	2
Total duration:	00:05
Minimum duration:	0.681403 seconds
Maximum duration:	4.438880 seconds
Average duration:	2.560141 seconds
Last duration:	4.438880 seconds
Last build date:	2023-03-09 6:10 PM
Last rebuild reason:	Out of date NTFS volume Y: is available

I took another look just now. [dis]mounting an external drive seems like a good reason for rebuilding or at least changing status of an index. (6:10pm is the end of my nightly backup of data to a *crypted USB drive)

What value is there in a user's periodic automatic analysis of the debug-build data? Might it serve to make use of Everything's ever-watchful eye to anticipate disk failures or, perhaps inherent hardware problems?
After all, our disks are our lifeblood.

Thanks, Chris

[void]

Everything 1.5.0.1341a will now keep a backup of your Everything.db in the following location:

%LOCALAPPDATA%\Everything\Everything.backup.db

If you are using the portable version, the Everything.backup.db will be stored in the same location as your Everything.exe.



The backup is created when you exit Everything or daily at 4am (or after a search window is closed if this scheduled save is missed).

Everything will try to load the backup database if there is something wrong with the normal Everything.db.
The backup is only created when the normal Everything.db is loaded successfully.



To disable the creation of the database backup file:

• Copy and paste the following into your Everything search box:
  /db_backup=0
• Press ENTER in your Everything search box.
• If successful, db_backup=0 is shown in the status bar for a few seconds.
• db_backup=1 is the default.

db_backup



This database backup file will allow you to access filenames from a previous index after exiting Everything, which might be useful for recovering filenames from a dead drive.

[David.P]

Just a quick question on this.

Does "update in background" only apply to drives or folders that have already been indexed, or can it be applied also to the initial scanning of added drives or folders, for example, by some setting?

[void]

The initial index is done in the foreground.
Rescanning is done in the background.

To perform the initial index in the background (do this before adding your folder indexes):

• In Everything 1.5, from the Tools menu, click Options.
• Click the Advanced tab on the left.
• To the right of Show settings containing, search for:
  index
• Select: folder_background_index
• Set the value to: true
• Click OK.

Enabling folder_background_index will decrease indexing performance and efficiency. Everything may freeze/hang for a few minutes when the scan completes in the background and builds your index. If you index over 1million files I highly recommend you restart Everything (File -> Exit) after the initial index. Without the restart Everything will use more memory than normal.

[Herkules97]

I have never seen a CRC Error on my testing PCs.

From what I have seen on the forums, CRC Error usually occurs when your PC shuts down and SSD buffers are not flushed to disk.
Your Everything database is most likely filled with zeroes.

Everything will tell the system to flush the database to disk on system shutdown.
It seems some hardware is ignoring this flush to disk command.

When my laptop were to close, it did not allow the database to be saved in time or maybe it was my desktop..
It took too long and so the files that didn't finish in time were stuck. It can take a minute to save it all while the PC may restart within 20 seconds.
Either that or I dreamt that whole scenario up.

If true, this leads to having a .tmp file that the system tries to load from.
One can then remove the .tmp file so it uses the old database and if not enough data has been added will just load the recent changes element.
Other alternative is that it just re-scans like it would if you tried loading from the .tmp file.

It's possible if it wasn't fake, that they aren't exiting Everything first.
Regardless of whether my situation happened or not, I always exit Everything before exiting the system and have done that since 2018 or whenever I started using this program. This allows any time needed for the process to redline in Process Explorer, indicating I can safely restart and keep EBV state right before restarting. Some data may be missed even with recent changes element if Windows modifies too many files, I've not touched the sizes for recent changes stuff. I think I did for a folder monitoring database and it may have been why it missed stuff because it was too big, not sure.