---
timer.exe (from igor's, 7bench) vhash .
T:\XXX\+ok.xxx-old-o-XXX\DOWN_XXX
so it runs the hash's in parallel (or at least to some extent)?
(or at least it generates/writes goes to next hash, generates/writes...)
oh, it automatically does recursion on an entire directory tree* (see below)
- in that case, option is need to not recurs
Kernel Time = 10.732 = 7%
User Time = 87.688 = 62%
Process Time = 98.421 = 69% Virtual Memory = 1028 MB
Global Time = 141.319 = 100% Physical Memory = 1030 MB
---
timer fchash --non_stop --sha1 * > 0sha1
Total: 1115files, 11501.6MiB, 86.3sec, 133.2MiB/s
Kernel Time = 0.873 = 1%
User Time = 27.159 = 31%
Process Time = 28.033 = 32% Virtual Memory = 10 MB
Global Time = 86.338 = 100% Physical Memory = 13 MB
--
ah, you've got to disregard (entirely) the above results,
as voidhash did not complete all of the files
T:\XXX\+ok.xxx-old-o-XXX\DOWN_XXX>wc -l ..*
651 ..md5
637 ..sfv
638 ..sha1
647 ..sha256
2573 total
T:\XXX\+ok.xxx-old-o-XXX\DOWN_XXX>wc -l 0hash 0sha1
1113 0hash
1125 0sha1
2238 total
---
fchash, 7200 HDD (local SATA):
TOSHIBA HDWN180
sha1: Total: 1115files, 11501.6MiB, 86.3sec, 133.2MiB/s
xxh3: Total: 1110files, 11501.3MiB, 53.8sec, 213.9MiB/s
---
for the 638 file that vhash did hash, the hashes agreed between vhash & fchash
(had to do a little tweaking of fchash' results as they are "non-standard")
wonder if timer interfered with vhash ?
- nope
-0 was with timer, !-0 was without
T:\XXX\+ok.xxx-old-o-XXX\DOWN_XXX>wc -l ..*
653 ..md5
651 ..md5-0
642 ..sfv
637 ..sfv-0
641 ..sha1
638 ..sha1-0
649 ..sha256
647 ..sha256-0
5158 total
btw: 25% cpu (100% of 1 core), 1 GB or RAM, i/o is variable max looked to be 118 MB
heh, oh, that particular directory had (at least) a 254-char file name-part
(which again is why i'm using fchash)
& if i rename it (with Everything [next largest name-part is 219-char]...)
T:\XXX\+ok.xxx-old-o-XXX\DOWN_XXX>wc -l ..*
1121 ..md5
651 ..md5-0
653 ..md5-2
1121 ..sfv
637 ..sfv-0
642 ..sfv-2
1121 ..sha1
638 ..sha1-0
641 ..sha1-2
1121 ..sha256
647 ..sha256-0
649 ..sha256-2
9642 total
- that's better
T:\XXX\+ok.xxx-old-o-XXX\DOWN_XXX>timer vhash .
voidhash 1.0 (c) voidtools 2022
parse .
parse ....
parse .\X...
Kernel Time = 14.695 = 8%
User Time = 144.004 = 78%
Process Time = 158.699 = 86% Virtual Memory = 1028 MB
Global Time = 182.643 = 100% Physical Memory = 1030 MB
given you're computing 4 (slower at that) hashes per file, would
seem you're quite efficient.
(sha1) hashes agree (as expected) between vhash & fchash
Salamander verified the same (*but* it had to be dealing with cached data)
(so... after all that, do you mind if i get back to actually doing something with my hash'd data
, heh.)---
T:\XXX\CE\cel\X>timer vhash .
voidhash 1.0 (c) voidtools 2022
parse .
parse ....
parse .\Y...
parse .\Y\X...
Kernel Time = 5.413 = 5%
User Time = 67.704 = 73%
Process Time = 73.117 = 79% Virtual Memory = 1028 MB
Global Time = 92.210 = 100% Physical Memory = 1030 MB
T:\XXX\CE\cel\X>timer hash --non_stop --recur * > 0sha1
Total: 263files, 5430.0MiB, 25.5sec, 213.3MiB/s
Kernel Time = 0.390 = 1%
User Time = 0.717 = 2%
Process Time = 1.107 = 4% Virtual Memory = 10 MB
Global Time = 25.479 = 100% Physical Memory = 12 MB
T:\XXX\CE\cel\X>timer hash --non_stop --recur --sha1 * > 0sha1
using "timer.exe" affects cache usage (or not)
i.e. (after having already run fchash on said directory):
> timer fchash --use_cache * > 0hash
- --use-cache, should use the cached (data)
but instead of it being any quicker, i was still running 25.seconds
vs. without "timer.exe"
> fchash --use_cache fchash * > 0hash
Total: 268files, 5430.1MiB, 2.3sec, 2320.6MiB/s
2.3 vs 25.seconds, so a 10x difference
(without --use-cache, he invalidates the cache - which is probably a good thing to do.
wonder how he does that? as a "clearchache.exe" could be very useful for various [generally benchmark] purposes)
now... why did you NOT recurs, when before you did ?
oh, you DON'T actually recurs, you just /enumerate/ ("parse") subdirectories,
but you don't actually do anything with them
T:\XXX\CE\cel\X\Y\X>timer vhash .
voidhash 1.0 (c) voidtools 2022
parse .
parse ....
parse .\Y...
Kernel Time = 0.312 = 136%
User Time = 0.000 = 0%
Process Time = 0.312 = 136% Virtual Memory = 1028 MB
Global Time = 0.228 = 100% Physical Memory = 1030 MB
well, that blows that comparison out of the water (unless i NOT --recurs, which probably makes sense, for now)
X: (only)
hash --sha1 --non_stop * > 0hash.Xonly
Total: 75files, 1378.1MiB, 6.4sec, 213.9MiB/s
so 6.sec vs. 92.sec for vhash ... ???
yep. odd, but does look to be correct.
Y: (only)
timer vhash .
Global Time = 52.609 = 100% Physical Memory = 1030 MB
timer hash --non_stop --sha1 * > 0sha1
Total: 198files, 4052.0MiB, 19.1sec, 211.9MiB/s
Global Time = 19.147 = 100% Physical Memory = 12 MB
so 19.sec vs 52.sec
hash --non_stop --use_cache * > 0hash
Total: 199files, 4052.1MiB, 20.3sec, 199.6MiB/s
so hash method will also depend on the data it's hashing
cause with these particular directories/files, there is no speed diff between sha1 & xxh128
(this all assumes i am doing things "correctly" & "accurately"
)