Skip to content

fix: validate modalities#1774

Open
dbirman wants to merge 5 commits intodevfrom
1688-validate-modality-between-acquisition-data-streams-and-data_descriptionmodalities
Open

fix: validate modalities#1774
dbirman wants to merge 5 commits intodevfrom
1688-validate-modality-between-acquisition-data-streams-and-data_descriptionmodalities

Conversation

@dbirman
Copy link
Copy Markdown
Member

@dbirman dbirman commented Mar 10, 2026

This PR validates modalities between:

  • data_description and acquisition.data_streams/stimulus_epochs. I.e. all data_description modalities must appear in the acquisition streams or epochs and vice versa
  • data_description and instrument, i.e. instrument must be a superset of data_description
  • data_description and quality_control, i.e. QC must be a subset of data_description

Errors are raised as warnings. In V3 we will uplevel to validation errors.

While writing tests I also fixed a bug where the "BEFORE" and "AFTER" time validators weren't allowing identical times. This causes problems when people put in their instrument a calibration that occurred at the same time as the acquisition_start_time. I think it's fine for them to be equal since people are just using the start time as a short-hand for "happened as part of this acquisition".

if dd_modalities - acq_modalities:
parts.append(f"in data_description but not acquisition: {dd_modalities - acq_modalities}")
warnings.warn(
f"Modality mismatch between acquisition.data_streams and data_description. {'; '.join(parts)}"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but something like "behavior" won't necessarily show up in the data_stream as a modality. How do we handle that?

Copy link
Copy Markdown
Collaborator

@saskiad saskiad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to handle the behavior exception

@dbirman
Copy link
Copy Markdown
Member Author

dbirman commented Mar 30, 2026

We've decided to hold on this PR until v3 upgrades. At that time we'll handle via the upgrader repairing all the assets that have behavior in data_description but no corresponding DataStream object.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Validate modality between acquisition data streams and data_description.modalities

2 participants