Constantly have to "Force Rebuild"

fury · Post by **fury** » Mon Apr 08, 2019 2:57 am

I'm running Everything 1.4.1.935 (x64) under Win10 Pro x64 1809. I started having issues recently. Frequently, new files won't show up until I manually do a "Force Rebuild".

It's not the search string at fault, because I leave that as-is, and the files appear after the rebuild is complete.

This happens on all my drives, and I've run CHKDSK, to no avail.

Post by **void** » Mon Apr 08, 2019 3:29 am

I suspect this is due to short filenames being used in the USN Journal.
I would need to see debug logs to be sure.

Please disable short filenames and let me know if the issue persists.

To disable short filenames:

From a admin command prompt, run the following command:
fsutil.exe behavior set disable8dot3 1

This operation takes effect immediately (no restart required).
https://support.microsoft.com/en-au/help/121007/how-to-disable-8-3-file-name-creation-on-ntfs-partitions

Resolving shortnames in the USN journal is on my TODO list.

fury · Post by **fury** » Wed Apr 10, 2019 2:22 pm

Thank you.

I disabled 8.3 creation (and verified the registry setting), but it seems the issue persists.

I'll look into enabling debug logging.

Post by **void** » Thu Apr 11, 2019 4:41 am

Thanks for the reply.

Could you please check "monitor changes" is enabled for all your NTFS volumes:

In Everything, from the Tools menu, click Options.
Click the NTFS tab on the left.
For each NTFS volume:
- Check Monitor Changes.
Click OK.

fury · Post by **fury** » Thu Apr 11, 2019 4:45 am

Thanks. It's enabled for all, and was previously.

Post by **void** » Fri Apr 12, 2019 1:44 am

Thanks for the reply.

Please try enabling debug logging:

In Everything, type in the following search and press ENTER:
/debug_log=1
Restart Everything:
Type in the following search and press ENTER:
/restart

Notes:
Debug logs can grow very large.
Debug logs are overwritten when you start Everything.

After you have detected Everything has missed a file system change, could you please send your %TEMP%\Everything Debug Log.txt to support@voidtools.com

Please compress this file before sending. If the compressed file is larger than 10MB, please let me know.

To disable debug logging:

In Everything, type in the following search and press ENTER:
/debug_log=0
Restart Everything:
Type in the following search and press ENTER:
/restart

fury · Post by **fury** » Fri Apr 12, 2019 2:15 am

I apologize (well, potentially). Before I posted previously, Everything did fail to index a new file I'd just created. But since then, I haven't had an issue. So the previous fix may have at least partially solved the problem.

Sorry--I only use it a few times a day, so it takes time for me to establish patterns. I test for a living, and don't have the wherewithal to do it off-hours as well.

I'll follow your advice if needed, and will follow up either way.

Post by **void** » Fri Apr 12, 2019 6:11 am

Thanks for the reply.

I should note that disabling 8.3 filenames will only effect new files.
Old files will still have 8.3 filenames which might be used in the USN change journal.

fury · Post by **fury** » Sat Sep 28, 2019 1:53 am

I still had the issue after my last post. But privacy concerns prevented me from creating and sending a log.

However, I reverted to 1.4.1.877 awhile back, and have not had the issue with it.

Post by **therube** » Thu Oct 03, 2019 4:18 pm

Anything "special" with the files that did not show; name, path, how they were created, application that created them ... ?

fury · Post by **fury** » Thu Oct 03, 2019 5:51 pm

Not that I'm aware of. It was happening with a variety of files, created by different applications.

tuska · Post by **tuska** » Fri Oct 11, 2019 5:29 pm

I would also like to join on this subject and give an example below.
As a rule, all entries in the status bar only appear on my PC, when I manually run "Force Rebuild".

Example:
- y.txt created in D:\Daten
- 941,688 entries are displayed
- wfn:y.txt -> file exists in Everything (immediately!)
- Execute export ... 941688.efu (150 MB)
- Force Rebuild
- 943,748 entries are displayed
- Execute export ... 943748.efu (151 MB)

Difference in .efu files: 2060 entries (!)
Compare file contents: 2261 difference(s) found
The difference in entries is usually between 200 and a few thousand entries.
The number of entries often differs by about 1250.

Not immediately in the index are usually files from the following folders:
C:\Program Files (x86)\Norton 360\... -> example below...
C:\ProgramData\Norton\...
C:\Program Files\WindowsApps\... -> example below...

Examples - the following two files were only visible in Everything after "Force Rebuild":

1. file BrandCfg.dat.data - changed/last access on 5.8.2019, created on 20.2.2018
"C:\Program Files (x86)\Norton 360\NortonData\22.7.0.76\hlinks\BrandCfg.dat.data",9860,132094933254656574,131636388178757516,32

2. file bth.inf_loc - changed/created/last access on 3.10.2018
"C:\Program Files\WindowsApps\Microsoft.LanguageExperiencePackde-DE_18362.14.49.0_neutral__8wekyb3d8bbwe\Windows\System32\driverstore\de-DE\bth.inf_loc",5202,131830417574845224,131830417512185068,262176

The following questions arise for me:
- How can it be that I find a just created text file (D:\Daten\y.txt) IMMEDIATELY in Everything and
other files from drive C: with an older date are not in the index?
- Could it be that Everything has no access to these files for some reason?

: 2019-10-11_Settings.png (47.55 KiB) Viewed 78947 times

C:\Tools\Everything\Everything.ini

Code: Select all

; Please make sure Everything is not running before modifying this file.
[Everything]
run_as_admin=0
allow_http_server=1
allow_etp_server=1
window_x=126
window_y=0
window_wide=1794
window_high=1222
maximized=1
minimized=0
fullscreen=0
ontop=0
alpha=255
match_whole_word=0
match_path=0
match_case=0
match_diacritics=0
match_regex=0
view=0
thumbnail_size=128
thumbnail_fill=0
min_thumbnail_size=32
max_thumbnail_size=256
medium_thumbnail_size=64
large_thumbnail_size=128
extra_large_thumbnail_size=256
thumbnail_load_size=0
shell_max_path=0
allow_multiple_windows=1
allow_multiple_instances=0
run_in_background=1
show_in_taskbar=1
show_tray_icon=1
minimize_to_tray=0
toggle_window_from_tray_icon=0
alternate_row_color=0
show_mouseover=1
check_for_updates_on_startup=1
beta_updates=1
show_highlighted_search_terms=1
text_size=0
hide_empty_search_results=0
clear_selection_on_search=1
show_focus_on_search=1
new_window_key=0
show_window_key=0
toggle_window_key=0
language=1031
show_selected_item_in_statusbar=1
statusbar_selected_item_format=
show_size_in_statusbar=1
statusbar_size_format=0
open_folder_command2=$exec("D:\Daten\Programme\TotalCommander\TCD_USB-Stick\TC32-64USB\Explorer.exe" /O /A /S "%1")
open_file_command2=
open_path_command2=$exec("D:\Daten\Programme\TotalCommander\TCD_USB-Stick\TC32-64USB\Explorer.exe" /O /A /S "%1")
explore_command2=
explore_path_command2=$exec("D:\Daten\Programme\TotalCommander\TCD_USB-Stick\TC32-64USB\Explorer.exe" /O /S $parent("%1"))
window_title_format=
taskbar_notification_title_format=
instance_name=
translucent_selection_rectangle_alpha=70
min_zoom=-6
max_zoom=27
context_menu_type=0
auto_include_fixed_volumes=0
auto_include_removable_volumes=0
auto_remove_offline_ntfs_volumes=1
auto_remove_moved_ntfs_volumes=1
auto_include_fixed_refs_volumes=0
auto_include_removable_refs_volumes=0
auto_remove_offline_refs_volumes=1
auto_remove_moved_refs_volumes=1
find_mount_points_on_removable_volumes=0
last_export_type=0
max_threads=0
reuse_threads=1
find_subfolders_and_files_max_threads=0
single_parent_context_menu=0
auto_size_1=512
auto_size_2=640
auto_size_3=768
auto_size_aspect_ratio_x=9
auto_size_aspect_ratio_y=7
auto_size_width_only=0
auto_size_path_x=1
auto_size_path_y=2
sticky_vscroll_bottom=1
last_options_page=0
draw_focus_rect=1
date_format=
time_format=
listview_item_high=0
single_click_open=2
underline_icon_titles=0
icons_only=0
auto_scroll_repeat_delay=250
auto_scroll_repeat_rate=50
open_many_files_warning_threshold=16
debug=0
debug_log=0
verbose=0
lvm=1
ipc=1
home_match_case=0
home_match_whole_word=0
home_match_path=0
home_match_diacritics=0
home_regex=0
home_search=1
home_filter=1
home_sort=0
home_view=0
home_index=1
allow_multiple_windows_from_tray=0
single_click_tray=1
close_on_execute=0
double_click_path=1
update_display_after_scroll=1
update_display_after_mask=1
auto_scroll_view=0
double_quote_copy_as_path=0
snap=0
snaplen=10
rename_select_filepart_only=1
rename_move_caret_to_selection_end=0
rename_nav=0
search_edit_move_caret_to_selection_end=0
search_edit_drag_accept_files=1
select_search_on_mouse_click=1
focus_search_on_activate=1
reset_vscroll_on_search=1
wrap_focus=0
load_icon_priority=0
load_thumbnail_priority=0
load_fileinfo_priority=0
always_request_all_fileinfo=0
header_high=0
hide_on_close=0
max_hidden_windows=0x00000000
winmm=0
menu_escape_amp=1
menu_folders=1
menu_folder_separator=\
menu_items_per_column=0
new_inherit=1
full_row_select=1
tray_show_command_line=
dpi=96
ctrl_mouse_wheel_action=1
allow_open=1
allow_context_menu=1
allow_delete=1
allow_rename=1
allow_cut=1
allow_copy=1
allow_paste=1
allow_drag_drop=1
allow_window_message_filter_dragdrop=0
auto_column_widths=0
hotkey_explorer_path_search=1
get_key_name_text=1
paste_new_line_op=0
fast_ascii_search=1
match_path_when_search_contains_path_separator=1
allow_literal_operators=1
allow_round_bracket_parenthesis=0
expand_environment_variables=1
search_as_you_type=1
always_update_query_on_search_parameter_change=0
convert_forward_slash_to_backslash=1
match_whole_filename_when_using_wildcards=0
operator_precedence=1
replace_exact_trailing_star_dot_star_with_star=1
allow_exclamation_point_not=1
search_command_prefix=
auto_complete_search_command=1
double_buffer=1
search=
show_number_of_results_with_selection=1
date_descending_first=1
size_descending_first=1
size_format=1
alpha_select=0
tooltips=1
listview_tooltips=1
show_detailed_listview_tooltips=0
rtl_listview_edit=0
force_path_ltr_order=1
force_path_left_align=1
date_time_order=0
date_time_align=1
size_align=3
invert_layout=0
update_layout_on_input_language_change=0
control_shift_action=3
change_search_rtl_reading_action=3
invert_layout_action=3
bookmark_remember_case=1
bookmark_remember_wholeword=1
bookmark_remember_path=1
bookmark_remember_diacritic=1
bookmark_remember_regex=1
bookmark_remember_sort=1
bookmark_remember_view=0
bookmark_remember_filter=1
bookmark_remember_index=1
bookmark_remember_search=1
bookmark_organize_x=559
bookmark_organize_y=339
bookmark_organize_wide=529
bookmark_organize_high=513
exclude_list_enabled=0
exclude_hidden_files_and_folders=0
exclude_system_files_and_folders=0
include_only_files=
exclude_files=
db_location=
db_multi_user_filename=0
db_compress=0
index_size=1
fast_size_sort=1
index_date_created=1
fast_date_created_sort=1
index_date_modified=1
fast_date_modified_sort=1
index_date_accessed=1
fast_date_accessed_sort=1
index_attributes=1
fast_attributes_sort=1
index_folder_size=1
fast_path_sort=1
fast_extension_sort=1
extended_information_cache_monitor=1
db_update_thread_priority=-15
index_recent_changes=1
refs_file_id_extd_directory_info_buffer_size=0
folder_update_thread_mode_background=1
folder_update_rescan_asap=1
monitor_thread_mode_background=1
monitor_retry_delay=30000
monitor_update_delay=1000
monitor_pause=0
usn_record_filter=0xffffffff
cancel_delay=0x000003e8
allow_ntfs_open_file_by_id=0
always_update_folder_recent_change=0
editor_x=328
editor_y=109
editor_wide=1327
editor_high=1053
editor_maximized=1
file_list_relative_paths=0
rename_x=631
rename_y=271
rename_wide=658
rename_high=620
rename_match_case=0
rename_regex=1
advanced_copy_to_x=514
advanced_copy_to_y=204
advanced_copy_to_wide=658
advanced_copy_to_high=620
advanced_copy_to_match_case=0
advanced_copy_to_regex=0
advanced_move_to_x=570
advanced_move_to_y=446
advanced_move_to_wide=658
advanced_move_to_high=620
advanced_move_to_match_case=0
advanced_move_to_regex=0
advanced_search_x=557
advanced_search_y=132
advanced_search_wide=777
advanced_search_high=934
advanced_search_page_y_offset=1484
advanced_search_focus_id=1098
advanced_search_warnings=1
max_recv_size=8388608
display_full_path_name=0
size_tiny=10240
size_small=102400
size_medium=1048576
size_large=16777216
size_huge=134217728
themed_toolbar=1
show_copy_name=3
show_copy_path=1
show_copy_full_name=2
show_open_path=2
show_explore=1
show_explore_path=1
copy_path_folder_append_backslash=0
custom_verb01=
custom_verb02=
custom_verb03=
custom_verb04=
custom_verb05=
custom_verb06=
custom_verb07=
custom_verb08=
custom_verb09=
custom_verb10=
custom_verb11=
custom_verb12=
filters_visible=1
filters_wide=514
filters_right_align=1
filters_tab_stop=0
filter=EVERYTHING
filter_everything_name=Alle Dateien | Everything   (Filter + Lesezeichen + Optionen ausschalten: STRG + Y)
filter_organize_x=557
filter_organize_y=339
filter_organize_wide=443
filter_organize_high=512
preview_visible=0
preview_x=575
preview_tab_stop=0
preview_mag_filter=0
preview_min_filter=0
preview_fill=0
show_preview_handlers_in_preview_pane=0
preview_load_size=0
preview_context=0x00000000
preview_release_handler_on_clear=0
sort=Name
sort_ascending=1
always_keep_sort=0
index=0
index_file_list=
index_etp_server=
index_link_type=1
status_bar_visible=1
select_search_on_focus_mode=1
select_search_on_set_mode=2
search_history_enabled=1
run_history_enabled=1
search_history_days_to_keep=60
run_history_days_to_keep=90
search_history_keep_forever=0
run_history_keep_forever=1
search_history_always_suggest=0
search_history_always_suggest_extend_toolbar=0
search_history_visible_count_max=24
search_history_always_suggest_visible_count_max=1
search_history_show_all_max=256
search_history_suggestion_max=256
search_history_show_all_sort=2
search_history_suggestion_sort=1
search_history_show_above=0
search_history_sort=0
search_history_sort_ascending=1
search_history_x=323
search_history_y=306
search_history_wide=1322
search_history_high=620
search_history_column_search_wide=938
search_history_column_search_order=0
search_history_column_count_wide=69
search_history_column_count_order=1
search_history_column_date_wide=126
search_history_column_date_order=2
etp_server_enabled=0
etp_server_bindings=
etp_server_port=21
etp_server_username=
etp_server_password=
etp_server_welcome_message=
etp_server_log_file_name=
etp_server_logging_enabled=1
etp_server_log_max_size=4194304
etp_server_log_delta_size=524288
etp_server_allow_file_download=1
ftp_allow_port=1
ftp_check_data_connection_ip=1
http_server_enabled=0
http_server_bindings=
http_title_format=
http_server_port=80
http_server_username=
http_server_password=
http_server_home=
http_server_default_page=
http_server_log_file_name=
http_server_logging_enabled=1
http_server_log_max_size=4194304
http_server_log_delta_size=524288
http_server_allow_file_download=1
http_server_items_per_page=32
http_server_show_drive_labels=0
http_server_strings=
service_pipe_name=
name_column_pos=0
name_column_width=420
path_column_visible=1
path_column_pos=1
path_column_width=875
size_column_visible=1
size_column_pos=3
size_column_width=123
extension_column_visible=1
extension_column_pos=2
extension_column_width=97
type_column_visible=0
type_column_pos=4
type_column_width=120
last_write_time_column_visible=1
last_write_time_column_pos=4
last_write_time_column_width=125
creation_time_column_visible=1
creation_time_column_pos=5
creation_time_column_width=122
date_accessed_column_visible=1
date_accessed_column_pos=6
date_accessed_column_width=122
attribute_column_visible=0
attribute_column_pos=8
attribute_column_width=87
date_recently_changed_column_visible=0
date_recently_changed_column_pos=7
date_recently_changed_column_width=191
run_count_column_visible=0
run_count_column_pos=10
run_count_column_width=120
date_run_column_visible=0
date_run_column_pos=8
date_run_column_width=191
file_list_filename_column_visible=0
file_list_filename_column_pos=12
file_list_filename_column_width=120
translucent_selection_rectangle_background_color=
translucent_selection_rectangle_border_color=
thumbnail_mouseover_border_color=
preview_background_color=
ntfs_volume_guids="\\\\?\\Volume{126add44-5607-11e3-ba64-806e6f6e6963}","\\\\?\\Volume{a5c60579-19c1-498c-bac1-dbe9b35d7cec}","\\\\?\\Volume{a5c60579-19c1-498c-bac1-dbe9b35d7cec}","\\\\?\\Volume{bdb6548c-d22e-44e6-9bd1-b54279c282ee}","\\\\?\\Volume{a22bd413-8774-4cc7-a86b-be8e4b6f51af}","\\\\?\\Volume{a5c60579-19c1-498c-bac1-dbe9b35d7cec}","\\\\?\\Volume{bdb6548c-d22e-44e6-9bd1-b54279c282ee}","\\\\?\\Volume{a22bd413-8774-4cc7-a86b-be8e4b6f51af}"
ntfs_volume_paths="C:","C:\\_Quelle\\D-I","C:\\_Quelle\\D-J","C:\\_Ziel\\D-I","C:\\_Ziel\\D-J","D:","I:","J:"
ntfs_volume_roots="","","","","","","",""
ntfs_volume_includes=1,0,0,0,0,1,1,1
ntfs_volume_load_recent_changes=1,0,0,0,0,1,1,1
ntfs_volume_include_onlys="","","","","","","",""
ntfs_volume_monitors=1,1,1,1,1,1,1,1
refs_volume_guids="\\\\?\\Volume{126add6b-5607-11e3-ba64-806e6f6e6963}","\\\\?\\Volume{126add6c-5607-11e3-ba64-806e6f6e6963}","\\\\?\\Volume{126add6d-5607-11e3-ba64-806e6f6e6963}","\\\\?\\Volume{126add6e-5607-11e3-ba64-806e6f6e6963}"
refs_volume_paths="G:","K:","L:","M:"
refs_volume_roots="","","",""
refs_volume_includes=0,0,0,0
refs_volume_load_recent_changes=0,0,0,0
refs_volume_include_onlys="","","",""
refs_volume_monitors=1,1,1,1
filelists=
filelist_monitor_changes=
folders=
folder_monitor_changes=
folder_buffer_size_list=
folder_rescan_if_full_list=
folder_update_types=
folder_update_days=
folder_update_ats=
folder_update_intervals=
folder_update_interval_types=
exclude_folders=
connect_history_hosts=
connect_history_ports=
connect_history_usernames=
connect_history_link_types=
etp_client_rewrite_patterns=
etp_client_rewrite_substitutions=
file_new_search_window_keys=334
file_open_file_list_keys=335
file_close_file_list_keys=
file_close_keys=343,27
file_export_keys=339
file_copy_full_name_to_clipboard_keys=9539
file_copy_path_to_clipboard_keys=9283
file_set_run_count_keys=
file_create_shortcut_keys=
file_delete_keys=8238
file_delete_permanently_keys=9262
file_edit_keys=
file_open_keys=8205
file_open_selection_and_close_everything_keys=
file_explore_path_keys=
file_open_new_keys=
file_open_path_keys=8461
file_open_with_keys=
file_open_with_default_verb_keys=
file_play_keys=
file_preview_keys=
file_print_keys=
file_print_to_keys=
file_properties_keys=8717
file_read_extended_information_keys=8517
file_rename_keys=8305
file_run_as_keys=
file_exit_keys=627,121
file_copy_name_to_clipboard_keys=
file_open_selection_and_do_not_close_everything_keys=
file_open_most_run_keys=
file_open_last_run_keys=
file_custom_verb_1_keys=
file_custom_verb_2_keys=
file_custom_verb_3_keys=
file_custom_verb_4_keys=
file_custom_verb_5_keys=
file_custom_verb_6_keys=
file_custom_verb_7_keys=
file_custom_verb_8_keys=
file_custom_verb_9_keys=
file_custom_verb_10_keys=
file_custom_verb_11_keys=
file_custom_verb_12_keys=
indexes_folders_rescan_all_now_keys=120
indexes_force_rebuild_keys=119
edit_cut_keys=8536
edit_copy_keys=8515,8493
edit_paste_keys=8534,9261
edit_select_all_keys=8513
edit_invert_selection_keys=
edit_copy_to_folder_keys=
edit_move_to_folder_keys=
edit_advanced_advanced_copy_to_folder_keys=
edit_advanced_advanced_move_to_folder_keys=
view_filters_keys=
view_preview_keys=337
view_status_bar_keys=
view_details_keys=
view_medium_thumbnails_keys=
view_large_thumbnails_keys=
view_extra_large_thumbnails_keys=
view_increase_thumbnail_size_keys=
view_decrease_thumbnail_size_keys=
view_window_size_small_keys=561
view_window_size_medium_keys=562
view_window_size_large_keys=563
view_window_size_auto_fit_keys=564
view_zoom_zoom_in_keys=443
view_zoom_zoom_out_keys=445
view_zoom_reset_keys=304,352
view_go_to_back_keys=549,166
view_go_to_forward_keys=551,167
view_go_to_home_keys=548
view_go_to_show_all_history_keys=1352,328
view_sort_by_name_keys=305
view_sort_by_path_keys=306
view_sort_by_size_keys=307
view_sort_by_extension_keys=308
view_sort_by_type_keys=309
view_sort_by_date_modified_keys=310
view_sort_by_date_created_keys=311
view_sort_by_attributes_keys=312
view_sort_by_file_list_filename_keys=
view_sort_by_run_count_keys=
view_sort_by_date_run_keys=
view_sort_by_date_recently_changed_keys=313
view_sort_by_date_accessed_keys=
view_sort_by_ascending_keys=
view_sort_by_descending_keys=
view_refresh_keys=116
view_fullscreen_keys=122
view_toggle_ltrrtl_direction_keys=
view_on_top_never_keys=
view_on_top_always_keys=340
view_on_top_while_searching_keys=
search_match_case_keys=329
search_match_whole_word_keys=322
search_match_path_keys=341
search_match_diacritics_keys=333
search_enable_regex_keys=338
search_advanced_search_keys=448
search_add_to_filters_keys=
search_organize_filters_keys=1350
bookmarks_add_to_bookmarks_keys=1348
bookmarks_organize_bookmarks_keys=324
tools_options_keys=336
tools_console_keys=
tools_file_list_editor_keys=
tools_connect_to_etp_server_keys=
tools_disconnect_from_etp_server_keys=
help_everything_help_keys=112
help_search_syntax_keys=1363
help_regex_syntax_keys=1362
help_command_line_options_keys=1355
help_everything_website_keys=1136
help_check_for_updates_keys=1360
help_about_everything_keys=368
help_donate_keys=
search_edit_focus_search_edit_keys=326,114
search_edit_delete_previous_word_keys=4360
search_edit_auto_complete_search_keys=4384
search_edit_show_search_history_keys=
search_edit_show_all_search_history_keys=4646,4648
result_list_item_up_keys=8230,4134
result_list_item_down_keys=8232,4136
result_list_page_up_keys=8225,4129
result_list_page_down_keys=8226,4130
result_list_start_of_list_keys=8228
result_list_end_of_list_keys=8227
result_list_item_up_extend_keys=9254,5158
result_list_item_down_extend_keys=9256,5160
result_list_page_up_extend_keys=9249,5153
result_list_page_down_extend_keys=9250,5154
result_list_start_of_list_extend_keys=9252
result_list_end_of_list_extend_keys=9251
result_list_focus_up_keys=8486,4390
result_list_focus_down_keys=8488,4392
result_list_focus_page_up_keys=8481,4385
result_list_focus_page_down_keys=8482,4386
result_list_focus_start_of_list_keys=8484
result_list_focus_end_of_list_keys=8483
result_list_focus_up_extend_keys=9510,5414
result_list_focus_down_extend_keys=9512,5416
result_list_focus_page_up_extend_keys=9505,5409
result_list_focus_page_down_extend_keys=9506,5410
result_list_focus_start_of_list_extend_keys=9508
result_list_focus_end_of_list_extend_keys=9507
result_list_focus_result_list_keys=
result_list_focus_highest_run_count_result_keys=
result_list_focus_last_run_result_keys=
result_list_toggle_path_column_keys=1330
result_list_toggle_size_column_keys=1331
result_list_toggle_extension_column_keys=1332
result_list_toggle_type_column_keys=1333
result_list_toggle_date_modified_column_keys=1334
result_list_toggle_date_created_column_keys=1335
result_list_toggle_attributes_column_keys=1336
result_list_toggle_file_list_filename_column_keys=
result_list_toggle_run_count_column_keys=
result_list_toggle_date_recently_changed_column_keys=1337
result_list_toggle_date_accessed_column_keys=
result_list_toggle_date_run_column_keys=
result_list_size_all_columns_to_fit_keys=8555
result_list_size_result_list_to_fit_keys=
result_list_context_menu_keys=9337
result_list_scroll_left_or_thumbnail_left_keys=8229
result_list_scroll_right_or_thumbnail_right_keys=8231
result_list_scroll_page_left_or_thumbnail_focus_left_keys=8485
result_list_scroll_page_right_or_thumbnail_focus_right_keys=8487
result_list_left_extend_keys=9253
result_list_right_extend_keys=9255
result_list_focus_left_extend_keys=9509
result_list_focus_right_extend_keys=9511
result_list_select_focus_keys=8224
result_list_toggle_focus_selection_keys=8480
result_list_copy_as_csv_keys=
preview_focus_preview_keys=
result_list_font=
result_list_font_size=
search_edit_font=
search_edit_font_size=
status_bar_font=
status_bar_font_size=
header_font=
header_font_size=
normal_background_color=
normal_foreground_color=
normal_bold=
highlighted_background_color=
highlighted_foreground_color=
highlighted_bold=
current_sort_background_color=
current_sort_foreground_color=
current_sort_bold=
current_sort_highlighted_background_color=
current_sort_highlighted_foreground_color=
current_sort_highlighted_bold=
selected_background_color=
selected_foreground_color=
selected_bold=
selected_highlighted_background_color=
selected_highlighted_foreground_color=
selected_highlighted_bold=
selected_inactive_background_color=
selected_inactive_foreground_color=
selected_inactive_bold=
selected_inactive_highlighted_background_color=
selected_inactive_highlighted_foreground_color=
selected_inactive_highlighted_bold=
drop_target_background_color=
drop_target_foreground_color=
drop_target_bold=
drop_target_highlighted_background_color=
drop_target_highlighted_foreground_color=
drop_target_highlighted_bold=
mouseover_background_color=
mouseover_foreground_color=
mouseover_bold=
mouseover_highlighted_background_color=
mouseover_highlighted_foreground_color=
mouseover_highlighted_bold=
mouseover_current_sort_background_color=
mouseover_current_sort_foreground_color=
mouseover_current_sort_bold=
mouseover_current_sort_highlighted_background_color=
mouseover_current_sort_highlighted_foreground_color=
mouseover_current_sort_highlighted_bold=
alternate_row_background_color=
alternate_row_foreground_color=
alternate_row_bold=
alternate_row_highlighted_background_color=
alternate_row_highlighted_foreground_color=
alternate_row_highlighted_bold=
current_sort_alternate_row_background_color=
current_sort_alternate_row_foreground_color=
current_sort_alternate_row_bold=
current_sort_alternate_row_highlighted_background_color=
current_sort_alternate_row_highlighted_foreground_color=
current_sort_alternate_row_highlighted_bold=
hot_background_color=
hot_foreground_color=
hot_bold=
hot_highlighted_background_color=
hot_highlighted_foreground_color=
hot_highlighted_bold=
selected_hot_background_color=
selected_hot_foreground_color=
selected_hot_bold=
selected_hot_highlighted_background_color=
selected_hot_highlighted_foreground_color=
selected_hot_highlighted_bold=
selected_inactive_hot_background_color=
selected_inactive_hot_foreground_color=
selected_inactive_hot_bold=
selected_inactive_hot_highlighted_background_color=
selected_inactive_hot_highlighted_foreground_color=
selected_inactive_hot_highlighted_bold=
thumbnail_mouseover_background_color=
thumbnail_mouseover_foreground_color=
thumbnail_mouseover_bold=
thumbnail_mouseover_highlighted_background_color=
thumbnail_mouseover_highlighted_foreground_color=
thumbnail_mouseover_highlighted_bold=

PS: Concerning the topic "short filenames" I haven't done anything yet.
________________________________________________
Windows 10 Pro (x64) Version 1903 (OS build 18362.418)
Intel(R) Core(TM) i7-4770 CPU @ 3.4OGHz, 16 GB RAM
Everything - Version 1.4.1.949 (x64) - Beta Nightly Builds
Portable on Systemdrive C: in C:\Tools\Everything

tuska · Post by **tuska** » Sat Oct 19, 2019 9:50 am

Today I did an additional test with the 32 bit version of Everything.
The 32 bit and the 64 bit version behave identical in my opinion.

Only after a "Force Rebuild" all data are available in the file Everything.db.
In this case the difference was 2,255 entries and it was again the above mentioned paths.

For me this is no problem, because I never search for files in these folders on drive C:.
If I should search for it, my file manager will find these files (tested).

________________________________________________
Windows 10 Pro (x64) Version 1903 (OS build 18362.418)
Intel(R) Core(TM) i7-4770 CPU @ 3.4OGHz, 16 GB RAM
Everything - Version 1.4.1.954 (x64) - Beta Nightly Builds
Portable on Systemdrive C: (x64) and portable on drive D: (x32)

Post by **void** » Tue Oct 22, 2019 7:09 am

Thanks for the information.

It sounds like the NTFS monitors are working, except some files are missed.

Could you please run Everything in debug mode, you don't have to send me any of the logs..

Just look for lines in red like the following:
unable to update ntfs folder: ABC: folder not found
unable to update ntfs folder: ABC: parent 0123456701234567 not found
folder name ABC already exist, yet FRN 0123456701234567 doesn't
unable to remove ntfs folder: 0123456701234567: file not found
unable to rename ntfs folder: 0123456701234567: move into child
unable to rename ntfs folder: 0123456701234567: not found
unable to rename ntfs folder: 0123456701234567: new parent not found
unable to add ntfs file: ABC.txt: parent 0123456701234567 not found
unable to remove ntfs file: ABC: file not found

Also look for the following errors in grey:
read NTFS usn journal failed x
read usn journal: bad cb x
read usn journal: record overflow
read usn journal: bad file name offset x, length y
read usn journal: bad record length x

If you see any of these messages, please let me know.

tuska · Post by **tuska** » Tue Oct 22, 2019 12:27 pm

Thank you for this detailed description for troubleshooting.

I did the following with Version 1.4.1.955 (x64) under
Windows 10 Pro (x64) Version 1903 (OS build 18362.418):

1. viewtopic.php?p=25825#p25825
    /debug_log=1
    /restart
2. Force Rebuild
3. %TEMP%\Everything Debug Log.txt
    Check for entries listed above
4. /debug_log=0
    /restart
---------------------------------
- 941984 entries
- Force Rebuild
- 943234 entries --- Difference: 1250 entries
- %TEMP%\Everything Debug Log.txt (55,1 KB) --> copied to another location and renamed to Everything Debug Log_Test1.txt
- Force Rebuild again on 22.10.2019 13:46
- 943234 entries --- same as before
- %TEMP%\Everything Debug Log.txt (128 KB)
   Everything Debug Log.txt -> File size is constantly growing...

Lines in red and errors in grey: No colors available in Everything Debug Log.txt.
I did not find any of the above entries (hints for errors) in the Everything Debug Log.txt file.

The only thing I noticed was a couple of those entries:
2019-10-22 13:36:36.014: CreateFileW(): GetLastError(): 5: Failed to open volume \\?\Volume{126add44-5607-11e3-ba64-806e6f6e6963}
but I think you had classified such entries as irrelevant in another case.

Did I maybe do something wrong on the test?

Post by **NotNull** » Tue Oct 22, 2019 5:49 pm

I guess you also have to enable the debug console ( Ctrl-` to enable /disable it )

That will open a console window. Do not close it with the [X] in that window, but use a Ctrl-` in the Everything window instead.

tuska · Post by **tuska** » Tue Oct 22, 2019 7:00 pm

Thanks for the advice!

I'm out of practice again.

I'll get back to that soon.

fury · Post by **fury** » Wed Oct 23, 2019 3:32 am

Boy, I'd love to be a forum mod. Anyway...

void wrote: ↑Tue Oct 22, 2019 7:09 am Thanks for the information.

It sounds like the NTFS monitors are working, except some files are missed.

Could you please run Everything in debug mode, you don't have to send me any of the logs..

Just look for lines in red like the following:
unable to update ntfs folder: ABC: folder not found
unable to update ntfs folder: ABC: parent 0123456701234567 not found
folder name ABC already exist, yet FRN 0123456701234567 doesn't
unable to remove ntfs folder: 0123456701234567: file not found
unable to rename ntfs folder: 0123456701234567: move into child
unable to rename ntfs folder: 0123456701234567: not found
unable to rename ntfs folder: 0123456701234567: new parent not found
unable to add ntfs file: ABC.txt: parent 0123456701234567 not found
unable to remove ntfs file: ABC: file not found

Also look for the following errors in grey:
read NTFS usn journal failed x
read usn journal: bad cb x
read usn journal: record overflow
read usn journal: bad file name offset x, length y
read usn journal: bad record length x

If you see any of these messages, please let me know.

I reinstalled 1.4.1.935 and enabled debug logging. There are many "not found" entries; here is a small excerpt:

2019-10-22 23:21:29.750: unable to add ntfs file: CachedImage_2560_1440_POS1.jpg: parent 04df000000006bf0 not found
2019-10-22 23:21:29.752: unable to remove ntfs folder: 04df000000006bf0: file not found
2019-10-22 23:21:29.756: folder name CachedFiles already exist, yet FRN 0x057f000000002c47 doesn't
2019-10-22 23:21:29.759: folder name CachedFiles already exist, yet FRN 0x057f000000002c47 doesn't
2019-10-22 23:21:29.762: unable to add ntfs file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.764: unable to add ntfs file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.767: unable to update file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.770: unable to add ntfs file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.773: unable to update file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.776: unable to add ntfs file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.780: unable to update file: CachedImage_2560_1440_POS1.jpg: parent 057f000000002c47 not found
2019-10-22 23:21:29.846: unable to add ntfs file: $ILDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.849: unable to add ntfs file: $ILDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.857: unable to update file: $ILDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.865: unable to add ntfs file: $ILDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.868: unable to update file: $ILDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.871: unable to rename ntfs folder: 01cb0000000072dd: new parent not found
2019-10-22 23:21:29.873: unable to rename ntfs folder: 01cb0000000072dd: new parent not found
2019-10-22 23:21:29.876: unable to remove ntfs folder: 01cb0000000072dd: file not found
2019-10-22 23:21:29.884: unable to update ntfs folder: $RLDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.887: unable to update ntfs folder: $RLDNWCH.tmp: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.890: unable to add ntfs file: $IM332LF: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.897: unable to add ntfs file: $IM332LF: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.900: unable to update file: $IM332LF: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.903: unable to add ntfs file: $IM332LF: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.905: unable to update file: $IM332LF: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.908: unable to rename ntfs folder: 0332000000002cf6: new parent not found
2019-10-22 23:21:29.916: unable to rename ntfs folder: 0332000000002cf6: new parent not found
2019-10-22 23:21:29.919: unable to remove ntfs folder: 0332000000002cf6: file not found
2019-10-22 23:21:29.922: unable to update ntfs folder: $RM332LF: parent 07e4000000000cd2 not found
2019-10-22 23:21:29.925: unable to update ntfs folder: $RM332LF: parent 07e4000000000cd2 not found

I did not find "usn journal failed", "bad cb", "record overflow", "bad file name offset", or "bad record length" entries, nor "read usn journal". I was a bit confused by your reference to "red" and "grey".

I shut down Everything after 7 minutes, since it was still "Updating database...", and I'm perhaps overly paranoid about wear and tear on my 6-year-old SSD.

Post by **NotNull** » Wed Oct 23, 2019 3:22 pm

fury wrote: ↑Wed Oct 23, 2019 3:32 am Boy, I'd love to be a forum mod.

Please elaborate?

fury · Post by **fury** » Wed Oct 23, 2019 4:25 pm

NotNull wrote: ↑Wed Oct 23, 2019 3:22 pm
fury wrote: ↑Wed Oct 23, 2019 3:32 am Boy, I'd love to be a forum mod.
Please elaborate?

I'm not sure who is talking to who anymore, and I am no longer following this thread. I think I just wasted an hour of my time when I was not the one being addressed.

Post by **NotNull** » Wed Oct 23, 2019 8:03 pm

fury wrote: ↑Wed Oct 23, 2019 4:25 pm I'm not sure who is talking to who anymore

Yeah, some threads here are quite confusing to follow (and sometimes I am partly to blame for that

).
Thanks for clarifying!

tuska · Post by **tuska** » Sat Oct 26, 2019 10:44 am

tuska wrote: ↑Tue Oct 22, 2019 7:00 pm I'll get back to that soon.

Now I did the following...
Start PC, then after some time debug logging (single steps roughly summarized):

Code: Select all

- File - Export... CTRL+S --> C:\Tools\Everything\EFU\953124.efu  (153 MB)
- Completely exited 'Everything'
- Everything.ini debug_log=1
- Regedit.exe --> "C:\Tools\Everything\Everything.exe" -svc -debug-log
- Right clicked Everything and clicked Stop Service
- Right clicked Everything and clicked Start Service
- Restart Everything. -> Database update... running (takes a long time)
- 953137 objects
- 20191026@113019_Everything Debug Log.txt	 97.187	26.10.2019 11:30
- Force Rebuild
- 954751 objects | Difference: 1614 objects;
  Difference only in the .efu files -> drive C: ... *\Norton* and *\WindowsApps* ... Paths as specified above.
- 20191026@113435_Everything Debug Log.txt	161.931	26.10.2019 11:34
- File - Export... CTRL+S --> C:\Tools\Everything\EFU\954751.efu  (153 MB)
- 20191026@114601_Everything Debug Log.txt	413.291	26.10.2019 11:46
- 20191026@115306_Everything Debug Log.txt	500.863	26.10.2019 11:53

Result after checking the *Everything Debug Log.txt files:
None of the above error messages are contained in these files.

Result after checking the file: Everything Service Debug Log.txt ... renamed to:
20191026@115729_Everything Service Debug Log.txt 7.003.185 26.10.2019 11:57
None of the above error messages are contained in this file.

On my other internal (data-)hard disks (D:, I:, J:) I haven't missed any entries yet.
The PC is always used as a user and not as an administrator.

Tested with:
Windows 10 Pro (x64) Version 1903 (OS build 18362.449)
Intel(R) Core(TM) i7-4770 CPU @ 3.4OGHz, 16 GB RAM
Everything - Version 1.4.1.955 (x64) - Beta Nightly Builds
Portable on Systemdrive C: in C:\Tools\Everything

Notnull wrote:I guess you also have to enable the debug console ( Ctrl-` to enable /disable it )
That will open a console window. Do not close it with the [X] in that window, but use a Ctrl-` in the Everything window instead.

I think this variant of debug logging doesn't have a console window.
The last time I saw one was in the topic "Everything - portable use",
in which an Everything.exe was previously specially prepared by the author.

Nevertheless I would be interested in this shortcut:
Could this shortcut be used to extract the content from a console window (with rapidly changing screen content)?
to the clipboard?

Does the shortcut refer to an English keyboard layout? (Here: German keyboard layout).
Which keys would I have to press at the same time on a German keyboard?

Post by **NotNull** » Sat Oct 26, 2019 1:30 pm

tuska wrote: ↑Sat Oct 26, 2019 10:44 am I think this variant of debug logging doesn't have a console window.

This is what I see on my system (file viewer uses tail option to keep up with recent changes to the log file):

: 2019-10-26 15_12_16-Window.png (95.68 KiB) Viewed 78880 times

On a German keyboard, the "`" ist zwischen ß und der Rücktaste.

Alternatively, you can add an extra keyboard shortcut (suggestion: Ctrl-Alt-D) to the debug console by adding it under Menu:Tools > Options > Keyboard : Tools|Console = CTRL+ALT+D (global)

tuska wrote: ↑Sat Oct 26, 2019 10:44 am Nevertheless I would be interested in this shortcut:
Could this shortcut be used to extract the content from a console window (with rapidly changing screen content)?
to the clipboard?

Use the mouse to selct some text in the console window.
Conhost.exe (the program responsible for all input/output to/from console applications) on Win10 will halt all further screen output until you press ESCAPE.

tuska · Post by **tuska** » Sat Oct 26, 2019 2:59 pm

2Notnull
Thank you for your prompt response and support.

I was a bit confused, because e.g. in Winword I need SHIFT+` ...
Now I have set CTRL+ALT+D as shortcut like you suggested.

With CTRL+` the console does not open, but with CTRL+ALT+D.

NotNull wrote: ↑Sat Oct 26, 2019 1:30 pm ... Use the mouse to selct some text in the console window.
Conhost.exe (the program responsible for all input/output to/from console applications) on Win10
will halt all further screen output until you press ESCAPE.

This can work, but my experience has shown that it usually doesn't work for me.
With CTRL+A this works reliably and afterwards with CTRL+C I can get the whole screen content,
and then filter in EmEditor for desired entries.

I assume that the console content is reflected in the Everything Debug Log.txt file.
Everything has now been clarified for me on this topic (debugging console).
Thanks again.

Post by **void** » Mon Oct 28, 2019 10:02 am

Thanks for the logs.

It looks like some files and folders are missing because a parent folder was not indexed or is missing.
It's tricky to say without seeing the full logs as to why these parent folders were not indexed.

folder name CachedFiles already exist, yet FRN 0x057f000000002c47 doesn't

This is the first time I have ever seen this error reported.

This error occurs when a new folder is created, but that folder already exists in the Everything database (by name) with a different FRN.
This may occur if this folder was deleted and recreated and Everything missed the deletion. Again, I would need to see the full logs.
Folder FRNs can not change once a folder is created.
This CachedFiles folder has a very high Sequence Number (0x057f), so it looks like it is deleted and recreated often.

For future release I'll make sure Everything just removes the old inconsistent folder and re-adds it with the correct FRN. This may help pick up new changes, but not fix the underlying issue.

tuska · Post by **tuska** » Mon Oct 28, 2019 2:59 pm

Today I did a verbose debug logging and a debug logging for the 'Everything Service' in 'Everything'.
Single steps roughly summarized:

Code: Select all

Start PC 08:45, then after some time at 14:40 
- File - Export... CTRL+S --> C:\Tools\Everything\EFU\953971.efu  (153 MB)
Verbose debug logging:
- Completely exited 'Everything'
- Everything.ini: debug_log=1 ... verbose=1 <-----------------------------------------------------------------------------------|
- Saved changes and restarted Everything.
- %TEMP%\Everything Debug Log.txt <---------------------------------------------------------------------------------------------|
- 20191028@144516_Everything Debug Log.txt	690.798	28.10.2019 14:45
Verbose debug logging for the 'Everything Service'
- Regedit.exe --> "C:\Tools\Everything\Everything.exe" -svc -debug-log -verbose <-----------------------------------------------|
  ■ Everything Service
- Task Manager: Right clicked Everything and clicked Stop Service
- Task Manager: Right clicked Everything and clicked Start Service
- Restart Everything. -> Updating database... (takes now much longer)
- Admin required to see file "Everything Service Debug Log.txt" in C:\Windows\Temp\Everything Service Debug Log.txt  <----------|
  20191028@145410_Everything Service Debug Log.txt	2.032.747	28.10.2019 14:54
- 953977 objects
- Force Rebuild
- 955211 objects | Difference: 1234 objects
- File - Export... CTRL+S --> C:\Tools\Everything\EFU\955211.efu  (153 MB)
  Difference only in the .efu files -> ONLY drive C: ... ONLY *\Norton* ... Path as specified above. -> No problem at all for me!
- 20191028@150040_Everything Debug Log.txt		2.062.205	28.10.2019 15:00
- 20191028@150021_Everything Service Debug Log.txt	5.193.855	28.10.2019 15:00
- 20191028@151822_Everything Service Debug Log.txt	5.872.403	28.10.2019 15:18
- 20191028@151908_Everything Debug Log.txt		3.232.507	28.10.2019 15:19
- 20191028@160224_Everything Service Debug Log.txt	7.488.521	28.10.2019 16:02
- 20191028@160213_Everything Debug Log.txt		5.956.970	28.10.2019 16:02
- zzz.txt created as admin on drive C: (C:\zzz.txt) --> zzz.txt was found in 'Everything' IMMEDIATELY!

Result after checking the *Everything Debug Log.txt files:
None of the above error messages are contained in these files.

Result after checking the *Everything Service Debug Log.txt files:
None of the above error messages are contained in these files.

I only noticed this lines:

Code: Select all

20191028@150021_Everything Service Debug Log.txt ... (791 lines in total with "bad file record type 00000000)
20191028@151822_Everything Service Debug Log.txt ... (805 lines in total with "bad file record type 00000000)
e.g. 2019-10-28 14:56:14.377: bad file record type 00000000 ...

Tested under same conditions as above.

Post by **void** » Wed Oct 30, 2019 7:07 am

bad file record type 00000000 ...

This is a empty NTFS record. Everything and Windows will ignore this file record.

tuska · Post by **tuska** » Wed Oct 30, 2019 10:20 am

void wrote: ↑Wed Oct 30, 2019 7:07 am
bad file record type 00000000 ...
This is a empty NTFS record. Everything and Windows will ignore this file record.

Thank you for that information.

Due to the number of entries, they probably cannot be related to those entries,
which I miss (usually 1250+ entries - only on drive C: in the above stated directories)
AND receive after a "Force rebuild"(!).

Post by **void** » Fri Nov 08, 2019 11:02 pm

Looking into this some more, it looks like some of these files might be hard links or folder junctions.

When a hard link is created or removed a USN_REASON_HARD_LINK_CHANGE notification is stored in the USN Change Journal.

Everything does not process USN_REASON_HARD_LINK_CHANGE notifications yet.
Adding new hard links is on my TODO list.

I'm also testing a fix that will add missing folders to the index.

tuska · Post by **tuska** » Sat Nov 09, 2019 12:26 pm

On the point "Junctions" it looks like this to me:

: 2019-11-09_Junctions in Everything.png (61.47 KiB) Viewed 78839 times

Thank you for your continued efforts to find a solution.

tuska · Post by **tuska** » Sun Mar 14, 2021 3:08 pm

2void
Everything 1.5 Alpha - Version 1.5.0.1238a (x64) has fixed the problem!

Short summary:

Indexed NTFS drives C:, D:, I:, J:
Only indexed NTFS drive C: (Systemdrive) did not show all files in Everything.
These were usually files from the following folders:
C:\Program Files (x86)\Norton 360\...
C:\ProgramData\Norton\...
C:\Program Files\WindowsApps\...

Only because it was reported above:
Newly created files are also displayed immediately in Version 1.5.0.1238a...and 1.5.0.1241a (newest version of today).

Thank you very much for your efforts!

___________________________________________________
Windows 10 Pro (x64) Version 20H2 (OS build 19042.867)
'Everything' 1.5.0.1241a (x64) | Portable on Systemdrive C:

tuska · Post by **tuska** » Wed Aug 25, 2021 3:24 pm

Hi,

Everything (main instance) + Everything (PRIVATE) - (additional instance)

- Path\to\Everything64.exe -instance "PRIVATE"
- Everything Service: Everything (PRIVATE)

Code: Select all

Everything 1.5.0.1272a (x64) ............... starts daily with Windows ([✓] Start Everything on system startup)
Everything 1.5.0.1272a (x64) - (PRIVATE) ... is started manually when necessary, e.g. to call up another language menu.

I have noticed on several days now that the main instance and another instance in the status bar have a different number of objects.
The difference was between 25 and 62 objects/entries.

In the case of the switched-on Everything (PRIVATE) instance, the entries are updated after the start,
However, even with hours of PC use, the number of entries does not reach the number of entries of the main instance.

Only after a "force rebuild", e.g. for the Everything (PRIVATE) instance, the number of entries corresponds to the main instance.

I have now noticed that there is only a difference in drive C: and not in drives D:, I:, J:
A file comparison brought up the usual "suspects" for me, such as files from Norton 360.

Can there still be a connection to the (solved) issue mentioned above?

_________________________________________________
Windows 10 Pro (x64) Version 21H1 (OS build 19043.1165)
'Everything' 1.5.0.1272a (x64)

Post by **void** » Thu Aug 26, 2021 8:56 am

Are you excluding any files or folders under Tools -> Options -> Exclude for the PRIVATE instance?
-If so, it's possible for Everything to miss changes if the files were ever excluded.

Are you using the include only setting under Tools -> Options -> NTFS -> drive -> Include only for the PRIVATE instance?

Could you please enable debug logging, allow Everything to run for sometime and check for missing files:

In the Everything private instance:
Ensure Everything is up to date. (force a rebuild from Tools -> Options -> Indexes -> Force Rebuild)
Enable debug logging from Tools -> Debug -> Start Debug Logging
Let Everything run for sometime.
Compare Everything with your other instance. (or Create a file list with the current index, force a rebuild and compare the file lists with the new index)
Stop debug logging from Tools -> Debug -> Stop Debug Logging.
---this will open the Everything debug log.txt in Notepad.
For the missing files, please check the debug log output for any associated USN events (and for any errors reported immediately after the USN event)
log entries should look something like:
USN PROCESS 00000102 norton.tmp
unable to add ntfs folder: norton: parent 0123456789abcdef not found
If possible, please send a copy of the USN events/errors (for the missing files only) to support@voidtools.com

tuska · Post by **tuska** » Thu Aug 26, 2021 9:50 pm

2void
thank you for your prompt response.

void wrote: ↑Thu Aug 26, 2021 8:56 am Are you excluding any files or folders under Tools -> Options -> Exclude for the PRIVATE instance?
-If so, it's possible for Everything to miss changes if the files were ever excluded.

No.

void wrote: ↑Thu Aug 26, 2021 8:56 am Are you using the include only setting under Tools -> Options -> NTFS -> drive -> Include only for the PRIVATE instance?

No.

The settings are identical under Tools -> Options -> NTFS -> drive.

The *.ini files are also almost identical Everything.ini <-> Everything-PRIVATE.ini.
In my opinion, there are only a few and irrelevant differences (language, last_options_page, options_x, options_y).

Search queries in TC using 'Everything' (TC = Total Commander)
My procedure is illustrated here under point 15. 'Everything' - Multiple instances -> Named instances
... files created by Everything to be stored -> 2. in a separate (sub)folder, e.g. "Everything_PRIVATE" ...:
C:\Tools\Everything\Everything_PRIVATE\Everything64.exe -instance "PRIVATE"

I.e. I copied all settings files from C:\Tools\Everything\ into the "Everything_PRIVATE" folder and then renamed the files
-> "Everything.ini" ... "Everything-PRIVATE.ini", "Bookmarks.csv" ... "Bookmarks-PRIVATE.csv", etc.,
to have my settings identical in both instances.
The second instance was started as follows: C:\Tools\Everything\Everything_PRIVATE\Everything64.exe -instance "PRIVATE"
____________________________
I will submit debug logging later.
Since the last "Force Rebuild" on both instances, I currently only have 5 differences - 'Everything' 1.5.0.1272a (x64) - 26.8.2021.

27.8.2021

Code: Select all

Everything 1.5.0.1273a (x64)	 Everything 1.5.0.1273a (x64) - (PRIVATE)	Difference
1 303 954 objects .......... <-> 1 303 829 Einträge .....................	+ 125 objects (main instance)

tuska · Post by **tuska** » Fri Aug 27, 2021 9:28 am

Up to now I have done the following:

In the Everything private instance:
Ensure Everything is up to date. (force a rebuild from Tools -> Options -> Indexes -> Force Rebuild)

Everything 1.5.0.1273a (x64) - (PRIVATE) - "Force Rebuild"

Code: Select all

D:, I:, J: .........:  Objects ident						Difference
Everything (Private) - drive C:		900 513 Einträge			+ 31 objects (!)
Everything ......... - drive C:		900 482 objects

I have noticed a few times now that the instance (main or second) always has the most number of entries (only drive C:!),
that I've previously done a "force rebuild" on. However, this could probably also be due to the fact
that there are a lot of movements (new files, deletions, Windows, Norton) take place on drive C:.
On the internal drives D:, I:, J: only a few changes (my files, backup) take place in comparison - objects are always identical(!).

Everything 1.5.0.1273a (x64) - "Force Rebuild" (main instance)

Code: Select all

D:, I:, J: .........:  Objects ident						Difference
Everything (Private) - drive C:		900 514 Einträge			   0 objects
Everything ......... - drive C:		900 514 objects

Enable debug logging from Tools -> Debug -> Start Debug Logging

Done for Everything (Private) instance because here the next thing to expect is that there will be fewer entries.

tuska · Post by **tuska** » Sat Aug 28, 2021 10:03 am

2void

Since Everything 1.5.0.1273a (x64) from 27.08.2021 the number of objects in the Everything main instance and
further instance ("Everything 1.5.0.1273a (x64) - (PRIVATE)") is always the same.
(After a "Force Rebuild" has been carried out on both instances).
Even if the number of objects changes over time, the number of objects is the same for both instances!

Code: Select all

27.08.2021 22:58
D:, I:, J: .........:  Objects ident						Difference
Everything (Private) - drive C:		902 108 Einträge (German)		   0 objects
Everything ......... - drive C:		902 108 objects

28.08.2021 11:29
D:, I:, J: .........:  Objects ident						Difference
Everything (Private) - drive C:		901 792 Einträge (German)		   0 objects
Everything ......... - drive C:		901 792 objects

28.08.2021 11:40
D:, I:, J: .........:  Objects ident						Difference
Everything (Private) - drive C:		901 862 Einträge (German)		   0 objects
Everything ......... - drive C:		901 862 objects

I.e. since this version, the problem has been solved.
Thank you very much!

Post by **void** » Sun Aug 29, 2021 12:44 am

Thanks for testing, please let me know if the issue occurs again.

Post by **void** » Thu Sep 09, 2021 5:36 am

Thanks for the logs tuska,

Everything was not caching hardlink filenames correctly.

Everything 1.5.0.1274a fixes an issue with detecting changes to hardlinks.

Post by **void** » Thu Sep 16, 2021 4:59 am

Thank you for the issue report tuska,

Everything 1.5.0.1275a fixes an issue with detecting all hardlink filenames when there was many hardlink filenames in a fragmented NTFS MFT record.

After installing, please force a rebuild from Tools -> Options -> Indexes.
Everything should not have any missing files after the rebuild.

Please let me know if Everything is still missing changes with this version.

tuska · Post by **tuska** » Tue Sep 21, 2021 10:08 am

2void
I have now carried out tests under the following conditions.

Code: Select all

Test scenario - 16.9.2021 ... 21.9.2021:
--------------------------------------------------------------------------------------------------------------------
Windows 10 Pro (x64) Version 21H1 (OS build 19043.1237)    https://aka.ms/Windows/ReleaseInfo

Everything 1.5.0.1275a (x64)
Instances: 3				        Start: Automatic / manual			Everything Service
--------------------------------------------------------------------------------------------------------------------
- Everything 1.5.0.1275a (x64) .... "MAIN"      [✓] Start Everything on system startup		Everything
- Everything 1.5.0.1275a (x64) - (PRIVATE)       *) Start Everything on system startup		Everything (PRIVATE)
- Everything 1.5.0.1275a (x64) - (PRIVAT)       --- Start manually				Everything (PRIVAT)

*) C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\Everything (x64) - (PRIVATE).lnk
   Link target: C:\Tools\Everything\Everything_PRIVATE\Everything64.exe -instance "PRIVATE" -startup

********************************************************************************************************************
Test results:
- Tested (multiple):	  Junction (Folder), Symlink (Folder, File), Hardlink (File) - create/delete
  Results:		  OK ... for Main instance, Everything (PRIVATE) and Everything (PRIVAT) instance

- Drive D:,I:,J:	  ) Always the same number of objects for these 3 instances.
- Drive C:	  	  ) Always the same number of objects for these 3 instances. !!!
********************************************************************************************************************

I have sent all the test details to you by email on 21.9.2021.

For me, the issue is solved.

Thank you very much for your efforts and support!

Post by **void** » Wed Sep 22, 2021 6:47 am

I am glad to hear Everything 1.5a is now picking up all hard links.

Thank you for testing Everything 1.5a.

Post by **void** » Tue Nov 09, 2021 5:28 am

Everything 1.5.0.1284a fixes an issue with missing changes to hard links.

Everything 1.5.0.1284a will no longer attempt to track hard link removal.
Forcing a rebuild will be required to remove stale hard links.