Skip to content

Delayed postprocessing timeout logic#104

Merged
ShyamB97 merged 14 commits intodevelopfrom
dte/pprocsched_timeout_logic
Nov 13, 2025
Merged

Delayed postprocessing timeout logic#104
ShyamB97 merged 14 commits intodevelopfrom
dte/pprocsched_timeout_logic

Conversation

@denizergonul
Copy link
Copy Markdown
Contributor

@denizergonul denizergonul commented Oct 7, 2025

  • Items can get stuck in buffer without getting postprocessed if no new items arrive. To fix this, we implemented a timeout logic that takes into consideration how many timeouts occurred since an item has been waiting to get processed.
  • Trigger has to define a policy on what to do in case of a timeout. For this, we introduced a new function in DataHandlingModel that will be overriden by TriggerDataHandlingModel. The function is called by postprocess scheduler.

DUNE-DAQ/trigger#416

Copy link
Copy Markdown
Contributor

@alessandrothea alessandrothea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me so far!

Comment thread include/datahandlinglibs/models/detail/DataHandlingModel.hxx Outdated
Comment thread include/datahandlinglibs/models/detail/DataHandlingModel.hxx Outdated
Comment thread include/datahandlinglibs/models/detail/DataHandlingModel.hxx Outdated
Copy link
Copy Markdown
Contributor

@alessandrothea alessandrothea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very nice. Just one comment about the postprocessing timeout logic.

Comment thread include/datahandlinglibs/models/DataHandlingModel.hpp Outdated

auto model =
unittest::MockDataHandlingModel<ReadoutType,
DefaultRequestHandlerModel<ReadoutType, SkipListLatencyBufferModel<ReadoutType>>,
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want the same test with a queue?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No... delayed postprocessing only makes sense for skip list.

return 0;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, this is just an optimization, but maybe still worth considering:
After a few timeouts, end_win_ts will settle on newest_ts+1, so maybe one could check here if
end_win_ts >= m_unprocessed_element.get_timestamp() and if so, stop the processing here.
@denizergonul let me know what you think.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest m_processed_up_to.get_timestamp() >= newest_ts + 1

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(so we don't even calculate end_win_ts )

Renamed m_unprocessed_element
Optimization in timeout logic
@denizergonul
Copy link
Copy Markdown
Contributor Author

What's your final opinion on RDT m_processed_up_to{}?

Copy link
Copy Markdown
Contributor

@alessandrothea alessandrothea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ShyamB97 ShyamB97 merged commit 7667de1 into develop Nov 13, 2025
3 checks passed
@ShyamB97 ShyamB97 deleted the dte/pprocsched_timeout_logic branch November 13, 2025 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Delayed postprocessing timeout logic

4 participants