tests: find binaural goldens named <prefix>_rendered_binaural.wav - #177
Open
trsonic wants to merge 1 commit into
Open
tests: find binaural goldens named <prefix>_rendered_binaural.wav#177trsonic wants to merge 1 commit into
trsonic wants to merge 1 commit into
Conversation
get_test_combination_metadata() constructs golden filenames only in the `<prefix>_rendered_id_<id>_sub_mix_<n>_layout_<n>.wav` form. Several binaural test vectors instead ship their reference render as `<prefix>_rendered_binaural.wav`, so the harness logged "golden wav file not found" and silently skipped those binaural layouts entirely -- even when run with --test_binaural. Fall back to `<prefix>_rendered_binaural.wav` for binaural layouts when the layout-indexed name does not exist. This lets 11 previously untested binaural vectors (object-based, advanced-profile, expanded-layout, and the binaural rendering-mode / filter-profile cases test_001100-001103) participate in the PSNR comparison; all pass at ~99 dB with --test_binaural. Non-binaural layouts and the existing layout-indexed binaural goldens (test_000095/000096) are unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
get_test_combination_metadata()builds golden filenames in only one form:Several binaural test vectors instead ship their reference render under a
layout-agnostic name,
<prefix>_rendered_binaural.wav. For those the harnesstakes the
elsebranch, logsand
continues — so the layout is skipped entirely, even when--test_binauralis passed. The warning is easily read as "this vector isnot valid to decode", which is why the gap went unnoticed.
Affected vectors:
test_000812,test_000825,test_000826,test_000912,test_000925,test_000931,test_001013,test_001100–test_001103—11 in total, covering object-based, advanced-profile, expanded-layout, and the
binaural rendering-mode / filter-profile cases.
Fix
When the layout-indexed golden does not exist and the layout is
LAYOUT_TYPE_BINAURAL, fall back to<prefix>_rendered_binaural.wavif thatfile is present.
The fallback is gated so it can only rescue a case that previously bailed out:
test_000095/test_000096, whose binaural goldens already use thelayout-indexed convention, are untouched because the primary path exists and
the first condition short-circuits.
Because
golden_wav_file_nameis reassigned beforeTestCombinationMetadatais constructed,
base_name_to_generatefollows it, so the-o3output pathgiven to
iamfdecand the reference used bycompute_metrics()stayconsistent.
bit_depthandsample_rateare read from the file actually beingcompared against.
Effect
The 11 vectors above now participate in the PSNR comparison under
--test_binaural; at the currently pinned OAR revision(
3d1d23b807543f993a1d0cf0a9839c7f0746d94b) they all score ~99 dB. Defaultruns are unchanged — binaural remains opt-in via
--test_binaural(
run_decode_and_psnr_test.py:261).Note on the naming assumption
The fallback assumes at most one binaural render per test-vector prefix. That
holds across the current corpus: of 333 textprotos, 13 declare
LAYOUT_TYPE_BINAURALand none declares more than one. If a future vector evercarries two binaural layouts, both combinations would resolve to the same
golden and the same generated filename; a guard can be added at that point, or
now if reviewers prefer.
Relevance to #175
This also makes the discussion in #175 verifiable. Six of the vectors this PR
un-skips —
test_000812,test_000825,test_000826,test_000912,test_000925,test_000931— are exactly the ones reported there as changingwhen OAR is advanced past the pinned revision. Until the harness can see them,
any regeneration of those reference files cannot be checked by the suite.