Files that fail --keep-foreign-metadata

[H2Swine]

These are WAVE_FORMAT_PCM type files such that the apparent "WAVE_FORMAT_EXTENSIBLE"-type chunks should be ignored. flac.exe --keep-foreign-metadata alters them upon an encode-decode roundtrip.

https://www.hifistatement.info/downloads/14-09-01_peterson/Mellow_Tone_16-44.wav
https://www.hifistatement.info/downloads/14-09-01_peterson/Mellow_Tone_24-192.wav

Two discussions on whether they are compliant (and one more file): dbry/WavPack#218 and https://hydrogenaudio.org/index.php/topic,128635

Addendum: some of the test files from that source behave this way, others don't. The person(s) who run the site might possibly know what software was used.

[ktmf01]

Does the decoder at least return an error that it didn't restore the metadata correctly?

[H2Swine]

Yes, sorry for not including that:
ERROR verifying foreign metadata restore from Mellow_Tone_16-44.flac to Mellow_Tone_16-44.flac.wav: stored main chunk length differs from written length

But it gives no warnung upon encoding that you won't get your file back this way.

[gstephens77]

Hi guys. 1.4.2 will work with these files. 1.4.3 and 1.5.0 throw this error. I also have wave files that use a custom chunk for storing radio automation software cues and info. It verified successfully until I replaced 1.4.2 today with 1.5.0. BSI Simian radio automation system are the files I was previously archiving as flac with metadata. Here is a test file of such wave file attached. Note, not all meta data fields are filled in for this track, just basic ones such as title, artist, audio category, no fade at segue, and an intro and sec cue tone of 1 sec in, and 1 sec out respectively. These used to encode and decode and verify fine. Like the other user above, it throws that error now since 1.4.3. I'm using Flac on Windows. Adobie Audition and the older Cool Edit Pro can also save wave files in what it calls broadcast wave format which is the same thing that BSI products use on the wav files.

test_file.wav

[H2Swine]

Tested by round-tripping wav -> flac -> wav using 9 combinations: each of 1.4.2, 1.4.3. 1.5.0 to encode and the same three to decode.

All nine give bit-identically the same .wav out in the end, but it differs from the original for the When_Its_Sleepy_Time_16-44.wav download. For your test_file.wav they do return the same as the original.

All reservations for human error. Wouldn't be my first.

[gstephens77]

My bad. The test file with the tone I assumed would play up like all the other files tagged from this system. But it didn't. I should have checked first.

However, when I encode an actual song (any one of the 932 on this system) rather than a test tone in to a flac file with 1.5.0 64-bit, I get this on decode with verify option enabled:

flac 1.5.0
Copyright (C) 2000-2009 Josh Coalson, 2011-2025 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

1000357.flac: done
ERROR verifying foreign metadata restore from 1000357.flac to 1000357.wav: store
d main chunk length differs from written length
Press any key to continue . . .

If I encode the test file instead with 1.5.0 32-bit, and then try and decode it with the 32-bit version this is what I get as an alternative message about 1 out of 5 goes:

flac 1.5.0
Copyright (C) 2000-2009 Josh Coalson, 2011-2025 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

*.flac: ERROR reading foreign metadata: can't initialize iterator (001)
Press any key to continue . . .

Swap back to 1.4.2 and no problem:

flac 1.4.2
Copyright (C) 2000-2009 Josh Coalson, 2011-2022 Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

1000357.flac: done
Press any key to continue . . .

edit: I can't post the file for copyright reasons here.

[gstephens77]

To add to the above, comparing the files in a hex editor, 1.4.2 makes changes to the start of the decoded file (RIFF section) different from the original, but it doesn't report it as being different when verifying it. 1.5.0 does the same changes, but it reports it as not the same as written. The solution for me was to try wavpack, the result was a 100% bit for bit file to the original, RIFF headers identical to original too.

[tobinus]

@H2Swine mentioned this issue in #878 :

Might I ask whether this has anything to do with the issue reported in #858 , or maybe it is so "close to" what you have fixed, that you might do this too?

I performed a test, and the pull request does not fix that issue. However, you get a new error message, since the size is the same now:

ERROR verifying foreign metadata restore from Mellow_Tone_24-192.flac to rt2.wav: stored foreign format block differs from written block. Perhaps the file is being restored to a different format than that of the original file

This is because FLAC writes zeroes to the restored cbSize field (0x00 00), while the original file has some nonsensical bytes there (0x40 FD).

I guess encoders are free to store whatever they want in those two bytes, since the documentation states:

For WAVE_FORMAT_PCM formats (and only WAVE_FORMAT_PCM formats), this member is ignored.

A proper fix for all files would be to copy the cbSize field from the original file when decoding, which is technically possible since we have access to the original fmt chunk. But it would mean that we introduce "keep foreign metadata" logic into the code that writes the fmt chunk. How "clean" do we wish to keep this code? Is it acceptable to dig up the original cbSize field from the foreign metadata in or close to the fmt chunk writing?