Having trouble with CaSe in preprocessor eval:#exact:

[raccoon]

So I'm trying to compare an SHA hash in a file's Alternate-Data-Stream versus the value in its sha512sum digest file. That is, verifying the sameness of the file's digest to its ads data. I am easily able to transpose the hash value from the ads via regex-match-1: so I can compare it to its sha512sum-SHA512: column. Both columns populate.

My problem: Everything makes the sha512sum-SHA512 column data UPPERCASE, while the ads is in lowercase, and I apparently don't know how to crack this.

regex:ads-ansi:"sha512=(\w+)" !eval:#<:#exact:#regex-match-1:,#sha512sum-sha512:#>:

I've tried mashing all sorts of syntax to ignore or augment the case of either property, but #exact: doesn't care what I try. Suggestions?

Side question: Why 'sha512sum-sha512' and not just 'sha512sum' ? <cipher>-<extension> for future sidecar file extensions?

[void]

Please try the following:

Code: Select all

regex:ads-ansi:"sha512=(\w+)" !eval:#<:#stricmp:<#regex-match-1:,#sha512sum-sha512:>==0#>:

#stricmp:

[raccoon]

I'm afraid this isn't working either.

I have created a test set for this, attached. It is a .7z (7-Zip) archive of a .wim (windows image) archive. 7-Zip can extract both archives. I chose .wim because it is the only container I know of that can store ADS data.

Contains 2 text files and 2 .sha512 digest files. All 4 files contain ADS data supplied by FastCopy. The 'mismatch' file's .sha512 digest no longer matches the file contents, because I altered the file's contents prior to using FastCopy. The mismatch file's ADS is a correct match. You ought to just ignore the ADS data on the .sha512 files themselves because they are not pertinent to this experiment.

[void]

Thanks for your reply raccoon,

Lots of bugs in Everything here..
eval: doesn't understand regex-match-1:
sha512sum-sha512: is leaking memory.
regular-expression-match-1 will cause a stack overflow in eval.

I will get this going and simplify the process for the next alpha update.

There's a lot of changes coming in the next alpha update for comparing properties.
I would like to phase out eval.

In the next alpha update you'll be able to search for:
regex:ads-ansi:"sha512=(\w+)" #regex-match-1:!=#sha512sum-sha512:

[raccoon]

<3

Do you have a match operator to determine case sensitivity and case insensitivity? I've seen '===' used to mean case-sensitive, and '~=' and '~==' can mean case-insensitive, while '=' and '==' can mean obey-match-case-setting?

[void]

Case insensitive by default.

For case sensitive:
case:#regex-match-1:!=#sha512sum-sha512:

I will have an update soon.

[void]

Everything 1.5.0.1304a improves comparing two properties.

You can now search for:
regex:ads-ansi:"sha512=(\w+)" regmatch1:==sha512sumsha512:
width:<height:
dm:date-taken:

Use the case: search modifier to match case.
Use the diacritics: search modifier to match diacritics.

For example:
case:regmatch1:==sha512sumsha512:



Use double quotes ("") to escape literal text.

For example, find the literal text ==sha512sumsha512: in regmatch1:
regmatch1:"==sha512sumsha512:"



The following operators are supported:
== is equal
!= is not equal
<= less than or equal
>= greater than or equal
< less than
> greater than

Everything will try to match properties with a number primitive first and then fall back to text comparison.
(kind-of like Javascript)

Number primitives support the following additional operators:
(no operator) is equal
= is equal
! is not equal



Everything 1.5.0.1304a also improves the regmatch properties.

regular expression match properties should now update in real-time.
For example: regex:sha512:(.*) should now show the correct capture once the property has been gathered.
Previously, regex:sha512:(.*) would match the empty value before it was gathered and would not update.

[raccoon]

Thanks @void for spending so much time performing surgery on these features. reg-match, ads, sha512, column/property compare. it all adds up to some very powerful stuff now!

Just in my case, now I can quickly detect bit rot when copying copies of copied copy because their digests don't match up to the most recent transfer hash. Such a rare occurrence, but helps me sleep at night, and saving me from 48 ~ 200 hours for a full read verification to know for sure.

Update: This has already paid off. Just found 3 totally corrupt copies that I probably wouldn't have caught otherwise for a very long time.

[Thy Grand Voidinesss]

Would someone be willing to prepare a tutorial on how to use Everything for detecting of corrupted backups mainly due to data fade? Starting with how the whole backup system should be set up?

I mean: in you case @raccoon, you have found 3 bad backups - and then what? Were those 3 single but huge archive files that became inaccessible - and thus you knew you could / should delete? Or were those 3 big folders with plethora of items in them - and you knew which items precisely, thus being able to salvage most of their content?

[I am asking somewhat in relation to this problem: viewtopic.php?p=56665#p56665]

[raccoon]

Thanks for the private message. Sorry, but I don't want to design backup systems for people for free.

What I use in one setup is the program FastCopy (its been around like 25 years), and recently it added the ability to store hash digests (ie, SHA512) of each file copied to that destination file's ADS. It can also perform a verification read of the destination file to verify that the hash matches from the original copying read. What it cannot do is tell you the file that was copied has bit rotted or not, just that the copy is good.

I also use HashCheck (search the forums) whenever I download or originate a piece of media, to create a hash digest file (filename.sha512) sidecar file that lives along side that piece of media.

The combination of having a sidecar digest, and an ADS digest, allows me to ask Everything to compare these two digests for sameness. If the ADS hash differs from the sidecar hash, then I know the recently copied file (more accurately its source file) have rotted since I downloaded or created the file years ago. This tells me to dig into the vaults to pull out and replace this file.

You can certainly write a better program to do all of this for you, but nobody has yet.

(In this system, sidecar files are copied with permanence across several pieces of media, while ADS streams are never copied and persist only on that one piece of media to reflect information of that specific copy instance.)
(The act of copying a file is identical to the act of scanning a file to verify its integrity, since both require a full time-consuming media read, so it's best to create new backups if you're going to spend days verifying integrity anyway.)

[Thy Grand Voidinesss]

Thanks for the private message. Sorry, but I don't want to design backup systems for people for free.

I do not expect [anyone] to

But I was hoping for that kind of answer

What I use is
[...]
also use
[...]
This tells me to
[...]

which I will have to chew through

[And FastCopy I already use - for removing of problematical files from Recycle Bin, along with the whole folder, as other know to me methods often fail to do do]