Skip to content

tests: find binaural goldens named <prefix>_rendered_binaural.wav - #177

Open
trsonic wants to merge 1 commit into
AOMediaCodec:mainfrom
trsonic:fix/harness-binaural-golden-name
Open

tests: find binaural goldens named <prefix>_rendered_binaural.wav#177
trsonic wants to merge 1 commit into
AOMediaCodec:mainfrom
trsonic:fix/harness-binaural-golden-name

Conversation

@trsonic

@trsonic trsonic commented Aug 21, 2026

Copy link
Copy Markdown

Problem

get_test_combination_metadata() builds golden filenames in only one form:

<prefix>_rendered_id_<id>_sub_mix_<n>_layout_<n>.wav

Several binaural test vectors instead ship their reference render under a
layout-agnostic name, <prefix>_rendered_binaural.wav. For those the harness
takes the else branch, logs

Warning: golden wav file not found, sometimes this is because the mix
presentation is invalid to decode: ...

and continues — so the layout is skipped entirely, even when
--test_binaural is passed
. The warning is easily read as "this vector is
not valid to decode", which is why the gap went unnoticed.

Affected vectors: test_000812, test_000825, test_000826, test_000912,
test_000925, test_000931, test_001013, test_001100test_001103
11 in total, covering object-based, advanced-profile, expanded-layout, and the
binaural rendering-mode / filter-profile cases.

Fix

When the layout-indexed golden does not exist and the layout is
LAYOUT_TYPE_BINAURAL, fall back to <prefix>_rendered_binaural.wav if that
file is present.

The fallback is gated so it can only rescue a case that previously bailed out:

  • non-binaural layouts are untouched;
  • test_000095 / test_000096, whose binaural goldens already use the
    layout-indexed convention, are untouched because the primary path exists and
    the first condition short-circuits.

Because golden_wav_file_name is reassigned before TestCombinationMetadata
is constructed, base_name_to_generate follows it, so the -o3 output path
given to iamfdec and the reference used by compute_metrics() stay
consistent. bit_depth and sample_rate are read from the file actually being
compared against.

Effect

The 11 vectors above now participate in the PSNR comparison under
--test_binaural; at the currently pinned OAR revision
(3d1d23b807543f993a1d0cf0a9839c7f0746d94b) they all score ~99 dB. Default
runs are unchanged — binaural remains opt-in via --test_binaural
(run_decode_and_psnr_test.py:261).

Note on the naming assumption

The fallback assumes at most one binaural render per test-vector prefix. That
holds across the current corpus: of 333 textprotos, 13 declare
LAYOUT_TYPE_BINAURAL and none declares more than one. If a future vector ever
carries two binaural layouts, both combinations would resolve to the same
golden and the same generated filename; a guard can be added at that point, or
now if reviewers prefer.

Relevance to #175

This also makes the discussion in #175 verifiable. Six of the vectors this PR
un-skips — test_000812, test_000825, test_000826, test_000912,
test_000925, test_000931 — are exactly the ones reported there as changing
when OAR is advanced past the pinned revision. Until the harness can see them,
any regeneration of those reference files cannot be checked by the suite.

get_test_combination_metadata() constructs golden filenames only in the
`<prefix>_rendered_id_<id>_sub_mix_<n>_layout_<n>.wav` form. Several
binaural test vectors instead ship their reference render as
`<prefix>_rendered_binaural.wav`, so the harness logged "golden wav file
not found" and silently skipped those binaural layouts entirely -- even
when run with --test_binaural.

Fall back to `<prefix>_rendered_binaural.wav` for binaural layouts when
the layout-indexed name does not exist. This lets 11 previously untested
binaural vectors (object-based, advanced-profile, expanded-layout, and
the binaural rendering-mode / filter-profile cases test_001100-001103)
participate in the PSNR comparison; all pass at ~99 dB with
--test_binaural. Non-binaural layouts and the existing layout-indexed
binaural goldens (test_000095/000096) are unaffected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant