Skip to content

README.md

Latest commit

 

History

History
56 lines (43 loc) · 4.32 KB

README.md

File metadata and controls

56 lines (43 loc) · 4.32 KB

Streaming Data Types

Crates.io PyPI

FlatBuffers is the format chosen for the ISIS data streaming system, derived from the ESS messaging system.

Schema ids

ID File name Description
f144 f144_logdata.fbs Controls related log data, typically from EPICS or NICOS. Note: not to be used for array data
ev44 ev44_events.fbs Multi-institution neutron event data for both single and multiple pulses
df12 df12_det_spec_map.fbs Detector-spectrum map for Mantid
se00 se00_data.fbs Used for storing arrays with optional timestamps, for example waveform data. Replaces senv.
ad00 ad00_area_detector_array.fbs EPICS area detector array data
hs01 hs01_event_histogram.fbs Event histogram stored in n dim array
ep01 ep01_epics_connection.fbs Status or event of EPICS connection. Replaces ep00
json json_json.fbs Carries a JSON payload
pl72 pl72_run_start.fbs File writing, run start message for file writer and Mantid
6s4t 6s4t_run_stop.fbs File writing, run stop message for file writer and Mantid
answ answ_action_response.fbs Holds the result of a command to the filewriter
wrdn wrdn_finished_writing.fbs Message from the filewriter when it is done writing a file
x5f2 x5f2_status.fbs Status update and heartbeat message for any software
fc00 fc00_forwarder_config.fbs Configuration update for Forwarder
al00 al00_alarm.fbs Generic alarm schema for EPICS, NICOS, etc.
da00 da00_dataarray.fbs Pseudo-scipp DataArray with time-dependent and constant Variables
un00 un00_units.fbs Engineering units update

Adding new schemas

Add .fbs file to `schemas directory

Check ess-streaming-data-types first; attempt not to diverge without reason. If a new schema is really needed, then add a new .fbs schema in the schemas directory.

Note: to generate code from .fbs schemas, you will need the flatc tool installed. It can be acquired from https://github.com/google/flatbuffers/releases .

Adding Python bindings

Python bindings have low-level code (autogenerated by flatc) in the fbschemas directory, but also manually-written convenience serializers and deserializers in the top-level of the python module.

When adding or modifying a schema:

  • The low-level code must be generated by manually calling flatc --python schemas\some_schema.fbs and adding the resulting generated code to python/src/streaming_data_types/fbschemas.
  • Manually-written serializers & deserializers will need to be updated, and added to the lists in __init__.py.

Rust bindings

Rust bindings have low-level code in flatbuffers_generated, and a small high-level wrapper to deserialize any message.

When adding or modifying a schema:

  • The low-level code must be generated by calling generate_rust_bindings.py
  • The wrapper (defined in lib.rs) will need to be updated with the new schema.