Why do various programs show various number of .wav?

[frew]

Thanks for the new update!

Just curious about something.

Any ideas about why a simple search for all .wav on C: would return varied results
from one program to another?

Windows 10
All searches done withing minutes, and no adding or removing of .wav in the process.

I wanted to see how many .wav files were on my computer so in Everything I searched
C: *.wav
and the results show
4,783

Then I did a Windows search, opening C: root drive, F3, *.wav
and the results show
5427

Next I used a variety of other programs using the same basic technique
searching from root drive C: all .wav, using *.wav.

So here's the various results from various programs:
4751
5017
5427 (Windows)
4746 (a handful of programs returned 4746)
4243
4783 (Everything)

I'm just wondering how many .wav files are indeed on my computer.

And what may account for the variations in the search results numbers?

Thanks for any ideas on this.

frew

[NotNull]

* Some programs search through zip files, others don´t
* depends on the useraccount the search program runs under. A restricted account can't see all the files on your system

It's hard to even define how many (.wav) files are on your system. A file that exists 1 time on disk, can have several names in several folders (hardlinks, symbolic links and other variations of reparse points).
I guess all these programs use different definitions of what to count as a file ...

(Fun fact: every file on your disk already has at least 2 names: a "normal" filename and an 8.3 filename for backwards compatibility with older programs. (*). This is done through hard-links)







(*) NTFS filesystem with out of the box settings; it is possible to get rid of the 8.3 filenames

[frew]

I guess all these programs use different definitions of what to count as a file ...

Wow, I did not know that.

Thank you for the insights.

frew

[NotNull]

Do you mean this in a cynical way? (can't tell ...)

[frew]

Do you mean this in a cynical way? (can't tell ...)

No, actually I did not know all about any of that.
But I knew something was up.

Recently got Windows 10, and never had to deal with these various kinds of
symbolic links, etc. Coming from XP. It seemed on XP that all the search tools
returned the same results when doing a simple search for the number of .wav
on my whole computer. Now with the various results i don't know what's what.

I work with music and I'd like a way to simply show me all the .wav files on my computer.

Thanks,

frew

[NotNull]

Ok, thanks for clarification! Either answer would be OK, but being on different wavelengths makes communicating on a forum even harder than it already is.

Obviously I can't tell you the right number of files or a definitive method to get this count (and it's a little too much to count them manually ). Maybe (probably!) someone else here is more knowledgeable in this area ...

Maybe one other suggestion: what you could do, is export the filenames found in the different utilities and let some file compare utility show you the differences (you will probably have to sort the files first).
Maybe that will give you some extra insight.

Good luck!

[void]

Another reason for why you may be seeing different results is wildcard usage:

Everything uses Windows XP style wildcards, that is using wildcards will match the entire filename, whereas in Windows Vista or later, wildcards match anywhere in the filename.

Searching for *.wav in Windows search may give you the following results:
foo.wav.mp3

To match wildcards anywhere in the filename:

• In Everything, from the Tools menu, click Options.
• Click the Search tab on the left.
• Uncheck Match whole filename when using wildcards.

[frew]

Okay, thanks for pointing that out about wildcard searches.

Good to know.

I think that explains a lot about some of the varied search results.

Still I wish there were a simple way to just show me all the .wav on my computer.

Note that the following is not a rant.
Rather, I'm just painting a picture of what I go through,
and then a couple questions.

With all the junctions, etc, etc...(not even really sure what they are)....symlinks,
hard links, soft links...Windows 10 seems very confusing to me. No actual
Documents and Setting folder....but instead an AppData folder? I even had one program
hang on some sort of search loop where is search log said something like not able to resolve
a search with something in the log like
C:\Documents and Settings\All Users\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data....etc.
As a matter of fact I just got that path by clicking on the Application Data folder there a few times.
Then there's hidden files, etc...and all the permissions stuff...
I don't know...kind of hard to tell what's on my computer really.

The reason I mention this is that, regarding searching for things on my computer,
I'm wondering if I'm overlooking something basic that I need to know about Windows 10 searches,
having come from having no problems at all knowing what was on my computer in XP.

Thanks for any ideas for me to consider.

frew

[NotNull]

I think that if you exclude the c:\Windows folder (and subdirs) from your utilities, the results will be more in line with each other. There are a lot of "special" files (amongst which WAV files) in Windows itself which might cause double-counting (678 WAV's on my Win7 test system)

[frew]

I think that if you exclude the c:\Windows folder

Thanks for the idea.

The thing is though that I do actually wish to return search results that
simply show me every single .wav file on C:

So here's my question:

Is it possible to generate a list of every single .wav file on C:
without showing any duplicate .wav files?
(unless of course they are actually duplicates found in separate folders,
in which case they count as an individual separate distinct .wav file)

By duplicate .wav files here I refer to any .wav that are showing up
in the results list more than one time, due to some sort of double counting
of any particular .wav file.

Thanks.

[Stamimail]

He will probably need a script similiar to this to get it more easier.

[frew]

He will probably need a script similiar to this to get it more easier.

Thanks. I'll check that.

[NotNull]

So here's my question:

Is it possible to generate a list of every single .wav file on C:
without showing any duplicate .wav files?
(unless of course they are actually duplicates found in separate folders,
in which case they count as an individual separate distinct .wav file)

By duplicate .wav files here I refer to any .wav that are showing up
in the results list more than one time, due to some sort of double counting
of any particular .wav file.

Thanks.

The thing is, how do you decide which one is the original and which on is the duplicate? To the filesystem they are the same.

He will probably need a script similiar to this to get it more easier.

I'll see if I can come up with something. Seems like a challenge ...

[NotNull]

I'll see if I can come up with something. Seems like a challenge ...

Well, that was easier than I thought it would be ...
It's a bit brute force (45 seconds for 1500 files), but it gets the job done
Output is a spreadsheet with matching files on the same line (inclusing a count of those, so you can sort the results).
Something like this: (tested with TXT files)

2017-06-11 23_10_51-temp3.csv - LibreOffice Calc.png (82.47 KiB) Viewed 19885 times

Will post the script if @frew is done with the other "things" related to this subject. Otherwise it gets confusing (for me, at least ...)

[frew]

Bear with me, I got a little mixed up.
I think it's worth copying and pasting the following,
which I mistakenly wrote in this related recent topic:
wondering how to see what paths contain no .wav files
I copy this here just so that if ever any others run into this problem,
and a solution is found, this restatement of my basic goal here
will be worth having in this thread I think.
Pardon the bit of tedium on my part here.



"Still I wonder, back to my basic question...
Okay, I've got this computer, and I know it has a bunch of .wav files on it:

What may be the best way, given that it's Windows 10, to
definitively show me every single separate .wav file on my computer?

As it is now, I'm just not sure how to get that information precisely,
largely I'm thinking now because of all the Junctions stuff on Windows 10
that may be causing various sort of "twice found" .wav on C:

Or for that matter perhaps "thrice found", or more?

Just out of curiosity now, I wonder how the built in
Windows 10 Windows Search service indexes all the
.wav on my computer? I wonder if I can trust that to show
me what's really going on with regards to showing me all the
separate individual .wav on my computer, ie without showing me
any of the "twice founds", etc.

This curious situation arose because, having just switched to
Windows 10 form XP, I noticed that the various tools I was using
on XP that gave me consistent search results returns from a simple search
for all .wav on C: now began ( on Win10) to show various results
amongst themselves with that same simple search for all .wav on C:"



Okay, thanks for the help.

I'm looking forward to trying the new ideas you're presenting.

frew

[therube]

every file on your disk already has at least 2 names: a "normal" filename and an 8.3 filename for backwards compatibility with older programs. (*). This is done through hard-links

"Normal" vs 8.3 has nothing to do with "hard links".

[void]

NotNull wrote:every file on your disk already has at least 2 names: a "normal" filename and an 8.3 filename for backwards compatibility with older programs. (*). This is done through hard-links

"Normal" vs 8.3 has nothing to do with "hard links".

NotNull is correct, 8.3 filenames are essentially a hard link.
Everything chooses to ignore 8.3 names.

[therube]

possible to generate a list of every single .wav file on C:
without showing any duplicate .wav files?

As it was, older versions of Everything (which, not sure offhand) only enumerated 1 filename of a hardlinked file.

So what currently shows as:
C:\Windows\Media\chimes.wav
and
C:\Windows\winsxs\x86_microsoft-windows-shell-sounds_31bf3856ad364e35_6.1.7600.16385_none_16e8d25616dd2c98\chimes.wav

(which are hardlinks of one another), in older version of Everything, would have shown only 1 of those files (& which particular one, was not determinable).

So using an older Everything, for that purpose, might get you closer.


Maybe a future Everything will have a hardlinkonly1: function .

[therube]

(All directory-based file systems must have (at least) one hard link giving the original name for each file. ... The term “hard link” is usually only used in file systems that allow more than one hard link for the same file.

So if every file is a hard link, & if 8.3 points to the same file as a LFN... still to me, its a stretch .)

[NotNull]

(All directory-based file systems must have (at least) one hard link giving the original name for each file. ... The term “hard link” is usually only used in file systems that allow more than one hard link for the same file.

So if every file is a hard link, & if 8.3 points to the same file as a LFN... still to me, its a stretch .)

It *is* more complicated than "just a hardlink". In fact it is a MS-DOS filename attribute where "regular"hardlinks are stored as an NTFS filename attribute. But to the "outside world" (programs, API's) it is the same.

I didn't want to overcomplicate things ... :-S

[therube]

(I just want to say...

Couldn't even figure out how do a Windows 7 Find, to search for file names at all?
In XP, Winkey-F, All files & folders, All or part of the filename; .wav.

And with that, went to the bathroom (& washed my hands), came back, only to hear the HDD running, so copied some (paper) files & filed them away, came back, only to hear the HDD running... Figured enough was enough & canceled the Find.

LONG LIVE Everything .)

[NotNull]

I just tested the script with the WAV's on my local system (they're all Windows WAV's; my music is on a network share).

I didn't expect to see that some of those files have 20 (!!) hardlinks ...
(721 filenames result in 346 files on disk)

[frew]

I just tested the script with the WAV's on my local system (they're all Windows WAV's; my music is on a network share).

I didn't expect to see that some of those files have 20 (!!) hardlinks ...
(721 filenames result in 346 files on disk)

Wow. I'm not sure what to do now.

By the way, what method did you use to determine
"(721 filenames result in 346 files on disk)"

Thanks.

[NotNull]

[
Wow. I'm not sure what to do now.
Thanks.

There are still some loose ends throughout these threads, I believe.

By the way, what method did you use to determine
"(721 filenames result in 346 files on disk)"

That's what this "duplicate" script does: feed it a list of all the WAV files and it will sort it out.

[therube]

That's what this "duplicate" script does

Do show.

So, what, you're pulling the FRN (File ID), counting the number links, if NUMBER > 1 ...


Related (even if it may or may not happen):

"I'm even considering indexing file frns, which would allow for finding hardlinks (duplicated frns).

Everything tries to avoid system32 (and other) redirections. I'm aiming to show the file system exactly as it is, which does cause some issues with gathering unindexed file information or opening certain system files. Everything disables windows folder redirection where possible.
I'll consider an option to show the redirected paths."

viewtopic.php?p=17968#p17968

[NotNull]

That's what this "duplicate" script does

Do show.

These threads (including this one: viewtopic.php?f=2&t=6204) go a bit like me asking "Where is the end of the universe?":
In itself a simple question (just keep walking straight ahead and you'll get there, right?), but the answers will probably fry my brain and lead to a lot of new questions. And it's hard to answer these questions without introducing further technicalities and complications, which would lead to even more questions ...

I got the feeling @frew (the original poster) is now at a point that he/she even regrets asking this
I think it would be best to let the dust settle (if that is the right expression?) before moving to the next step, especially since @frew has troubles handling scripts.

But for your amusement,you can find the script attached so I don't have to explain what it does

"I'm even considering indexing file frns, which would allow for finding hardlinks (duplicated frns).
[...]
viewtopic.php?p=17968#p17968

Thanks for that link! I already concluded that FRN's are not indexed. Tried and got no results.
This was (highly) expected as I run Everything with Folder indexes (and no NTFS Indexes) under a a restricted non-admin useraccount without the Everything service.
(although, as far as I know, you don't need administrative privileges to read the FRN's)

[NotNull]

@frew: now the related thread is sort of closed, we can continue here (never a dull moment ... )

Earlier in this thread you created a file, containing an enormous list of all your WAV files (let's assume it is called MyWAVs.txt). You will need that.

[*] Extract GetDuplicates.cmd from the previously attached ZIP file and put it in a folder (easiest would be the same folder where MyWAVs.txt is)
[*] Drag/drop MyWAVs.txt to GetDuplicates.cmd
[*] Wait a couple of minutes (rough estimation: 5 to 10 minutes)

This will create MyWAVs_duplicates.csv

[*] Open this file. It will be recognized as a spreadsheet ( ";" as delimiter if asked)

On any line you will see the ID (FRN) of the file *ON DISK* (second column), the number of names this file has (first column) and every name of the file (third column and beyond)


The number of lines in MyWAVs_duplicates.csv will give you the number of unique files on disk. No duplicates. (Ok, ok ... subtract 1 for the header ...)

[NotNull]

@therube: If it wasn't already clear:

- for every filename in file: Export FRN;filename
- sort this, so FRN's are grouped together
- for every FRN: put filename on stack (and keep count)
- New FRN: write count, old FRN and all filenames on stack to CSV file

(That's why the "call :NODOUBLE dummy dummy": to push the last FRN out of the pipeline. This is just a quick script; no algorithm optimization, etc..)

[frew]

On any line you will see the ID (FRN) of the file *ON DISK* (second column), the number of names this file has (first column) and every name of the file (third column and beyond)

Okay, I opened the MyWAVs_duplicates.csv in LibreOffice and everything looks pretty good,
but I'm wondering about a few unusual lines that show up. Any idea what these mean?
For example, line 537:

42 0x00000000000000000001000000048d47 C:\Users\main\Local Settings\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Users\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\Local Settings\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav C:\Documents and Settings\main\AppData\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Programs\Python\Python35-32\Lib\test\sndhdrdata\sndhdr.wav

Sorry for all that, just wanted to show exactly what was on that line.
What does all that mean I wonder?


Edit: Sorry, and what would this mean again, here at line 853?

2 0x00000000000000000003000000011679 C:\Windows\WinSxS\amd64_microsoft-windows-shell-sounds_31bf3856ad364e35_10.0.15063.0_none_2826556bee58508f\Alarm01.wav C:\Windows\Media\Alarm01.wav


Thanks

[NotNull]

Edit: Sorry, and what would this mean again, here at line 853?

2 0x00000000000000000003000000011679 C:\Windows\WinSxS\amd64_microsoft-windows-shell-sounds_31bf3856ad364e35_10.0.15063.0_none_2826556bee58508f\Alarm01.wav C:\Windows\Media\Alarm01.wav
Thanks

Like said before:

On any line you will see the ID (FRN) of the file *ON DISK* (second column), the number of names this file has (first column) and every name of the file (third column and beyond)

So, file with ID 0x....679 exists under 2 names:
[1] C:\Windows\WinSxS\amd64_microsoft-windows-shell-sounds_31bf3856ad364e35_10.0.15063.0_none_2826556bee58508f\Alarm01.wav
[2] C:\Windows\Media\Alarm01.wav

Those 2 files are the same. If you edit the first one and open the second one later on, it has all the changes you made to the first one.
The file 0x...679 is only deleted from disk when you delete the first filename AND the second filename.



The very long entry on line 537 is a perfect example why different programs count different number of files:

Those folders are JUNCTIONS; entering them will "warp" you to another folder on your disk. Unfortunately, there are some loops in there: A "warps" you to B; B:\folder "warps" you back to A (a simple example).

Tool X recognizes a JUNCTION and stops there
Tool Y recognizes a loop in the JUNCTIONS and stops there
Tool Z doesn't all that and enters the loop and will keep on running in circles and will only exit this loop when maximum path length has been reached.

The list of WAV files is created with a tool of type Z.

[frew]

Once again, thank you very much.

That really cleared things up, and I appreciate it.

frew

[NotNull]

... and thank YOU! One of the main reasons (*) I hang out here is because I wanted to improve on my English (not my native language). And you certainly made me practice a lot

(*) Other reasons: I think one of the best ways to learn a new tool thoroughly is by answering questions (or trying to) that are posted on it's forum, regardless if you post your answers or not. And I like solving puzzles. A lot.