From edbda0b83a50e1baa49d07aa775925e7de634435 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 11:25:58 +0200 Subject: [PATCH 01/17] add info on sensor configuration --- .../tutorials/working_with_expedition_yaml.md | 28 ++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/user-guide/tutorials/working_with_expedition_yaml.md b/docs/user-guide/tutorials/working_with_expedition_yaml.md index af059b3f..d0825794 100644 --- a/docs/user-guide/tutorials/working_with_expedition_yaml.md +++ b/docs/user-guide/tutorials/working_with_expedition_yaml.md @@ -49,8 +49,13 @@ instruments_config: # <-- 2. instrument configuration section num_bins: 40 max_depth_meter: -1000.0 period_minutes: 5.0 + sensors: + - VELOCITY ship_underwater_st_config: period_minutes: 5.0 + sensors: + - TEMPERATURE + - SALINITY argo_float_config: ... ctd_config: ... drifter_config: ... @@ -88,6 +93,27 @@ You can do multiple `DRIFTER` deployments at the same waypoint by adding multipl This section defines the configuration settings for each instrument used in the expedition. Each instrument has its own subsection where specific parameters can be set. +##### Sensors + +For most users, the most important instrument configuration settings to consider are the **sensors** for each instrument, which control what type of measurements/variables the instrument records in the simulation. For example, for the `CTD` instrument, you can specify which sensors to include in the simulation (e.g., `TEMPERATURE`, `SALINITY`, `OXYGEN`, etc.) by adding or removing entries from the `sensors` list in the `ctd_config` section. These must be added on _new lines_ and be in _uppercase_, for example: + +```yaml +ctd_config: + max_depth_meter: -2000.0 + min_depth_meter: -11.0 + stationkeeping_time_minutes: 50.0 + sensors: + - TEMPERATURE + - SALINITY + - OXYGEN +``` + +```{important} +See [here](../documentation/full_sensor_list.md) for a full list of available sensors for each instrument. Trying to add a sensor to an instrument that does not support it will result in errors in VirtualShip. +``` + +##### Underway Instruments + Because **underway instruments** (e.g., ADCP, Ship Underwater ST) collect data continuously while the ship is moving, their deployment is not tied to specific waypoints. Instead, the presence of their configuration sections in `instruments_config` indicates that they will be active throughout the expedition. This means that if you wish to turn off an underway instrument, you can remove its configuration section or simply set it to `null`, for example: ```yaml @@ -96,7 +122,7 @@ instruments_config: ship_underwater_st_config: null ``` -For **all other instruments**, e.g. CTD, ARGO_FLOAT etc., the parameters can often be left as the default values unless advanced customisations are required. +For **all other instruments**, e.g. CTD, ARGO_FLOAT etc., the parameters can often be left as the default values unless further, advanced customisations are required. #### 3. `ship_config` From cd5b523f83046ca1e3ec5ab42a32af36ee5aa967 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 11:26:26 +0200 Subject: [PATCH 02/17] add list of available sensors to documentation --- .../documentation/full_sensor_list.md | 26 +++++++++++++++++++ docs/user-guide/index.md | 1 + 2 files changed, 27 insertions(+) create mode 100644 docs/user-guide/documentation/full_sensor_list.md diff --git a/docs/user-guide/documentation/full_sensor_list.md b/docs/user-guide/documentation/full_sensor_list.md new file mode 100644 index 00000000..1cc072a7 --- /dev/null +++ b/docs/user-guide/documentation/full_sensor_list.md @@ -0,0 +1,26 @@ +# Full list of available instrument sensors + +The following table provides a comprehensive list of available sensors for each instrument. These sensors can be specified via the `virtualship plan` tool (see the [Quickstart guide](../quickstart.md)) or in the `sensors` section of the respective instrument configuration in the `expedition.yaml` file (see the [working with expedition.yaml tutorial](../tutorials/working_with_expedition_yaml.md)). + +```{note} +Trying to add a sensor to an instrument that does not support it will result in errors in VirtualShip. Always refer to this table to check which sensors are available for each instrument. +``` + +| Instrument | Sensor Name | Description | Units | Category | +| :--------------------- | :----------------- | :-------------------------------------------------- | :----------------------------------- | :-------------- | +| **ADCP** | VELOCITY | Current velocities (eastward (u) and northward (v)) | m/s | Physical | +| **Ship Underwater ST** | TEMPERATURE | Temperature | °C | Physics | +| | SALINITY | Salinity | psu | Physics | +| **CTD** | TEMPERATURE | Temperature | °C | Physics | +| | SALINITY | Salinity | psu | Physics | +| | OXYGEN | Oxygen concentration | mmol m-3 | Biogeochemistry | +| | CHLOROPHYLL | Chlorophyll concentration | mmol m-3 | Biogeochemistry | +| | NITRATE | Nitrate concentration | mmol m-3 | Biogeochemistry | +| | PHOSPHATE | Phosphate concentration | mmol m-3 | Biogeochemistry | +| | PH | pH | - | Biogeochemistry | +| | PHYTOPLANKTON | Phytoplankton concentration in carbon | mmol m-3 | Biogeochemistry | +| | PRIMARY_PRODUCTION | Net primary production | mmol m-3 day-1 | Biogeochemistry | +| **ARGO_FLOAT** | TEMPERATURE | Temperature | °C | Physics | +| | SALINITY | Salinity | psu | Physics | +| **DRIFTER** | TEMPERATURE | Temperature | °C | Physics | +| **XBT** | TEMPERATURE | Temperature | °C | Physics | diff --git a/docs/user-guide/index.md b/docs/user-guide/index.md index fac1c26c..d5d4a124 100644 --- a/docs/user-guide/index.md +++ b/docs/user-guide/index.md @@ -17,4 +17,5 @@ assignments/index documentation/copernicus_products.md documentation/pre_download_data.md documentation/example_copernicus_download.ipynb +documentation/full_sensor_list.md ``` From 4668e548e63dcd27457c6493e7245a8fcea34bc2 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 11:27:16 +0200 Subject: [PATCH 03/17] remove redundant TODO --- src/virtualship/static/expedition.yaml | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/virtualship/static/expedition.yaml b/src/virtualship/static/expedition.yaml index 3be45c4d..acb16dcf 100644 --- a/src/virtualship/static/expedition.yaml +++ b/src/virtualship/static/expedition.yaml @@ -1,7 +1,5 @@ # see https://virtualship.readthedocs.io/en/latest/user-guide/tutorials/working_with_expedition_yaml.html for more details on how to edit this file # -# TODO: add a link to docs where lists what sensors are supported for each instrument -# schedule: waypoints: - instrument: From ae2d528edc38f735d296a0a1577e31195aa33a34 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 11:52:09 +0200 Subject: [PATCH 04/17] improve phrasing --- docs/user-guide/tutorials/working_with_expedition_yaml.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/tutorials/working_with_expedition_yaml.md b/docs/user-guide/tutorials/working_with_expedition_yaml.md index d0825794..6bbf181e 100644 --- a/docs/user-guide/tutorials/working_with_expedition_yaml.md +++ b/docs/user-guide/tutorials/working_with_expedition_yaml.md @@ -95,7 +95,7 @@ This section defines the configuration settings for each instrument used in the ##### Sensors -For most users, the most important instrument configuration settings to consider are the **sensors** for each instrument, which control what type of measurements/variables the instrument records in the simulation. For example, for the `CTD` instrument, you can specify which sensors to include in the simulation (e.g., `TEMPERATURE`, `SALINITY`, `OXYGEN`, etc.) by adding or removing entries from the `sensors` list in the `ctd_config` section. These must be added on _new lines_ and be in _uppercase_, for example: +For most users, the most important instrument configuration setting to consider is the list of **sensors** for each instrument, which controls what type of measurements/variables the instrument records in the simulation. For example, for the `CTD` instrument, you can specify which sensors to include in the simulation (e.g., `TEMPERATURE`, `SALINITY`, `OXYGEN`, etc.) by adding or removing entries from the `sensors` list in the `ctd_config` section. These must be added on _new lines_ and be in _uppercase_, for example: ```yaml ctd_config: From 17d5b40bc335976b229a9cd4610d68039b1a8046 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 11:52:25 +0200 Subject: [PATCH 05/17] add details on sensor configurations to quickstart guide --- docs/user-guide/quickstart.md | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/docs/user-guide/quickstart.md b/docs/user-guide/quickstart.md index cc43b8a0..a4f75997 100644 --- a/docs/user-guide/quickstart.md +++ b/docs/user-guide/quickstart.md @@ -56,7 +56,7 @@ This will create a folder/directory called `EXPEDITION_NAME` with a single file: For advanced users: it is also possible to run the expedition initialisation step without an MFP .xlsx export file. In this case you should simply run `virtualship init EXPEDITION_NAME` in the CLI. This will write an example `expedition.yaml` file in the `EXPEDITION_NAME` folder/directory. This file contains example waypoints, timings, instrument selections, and ship configuration, but can be edited or propagated through the rest of the workflow unedited to run a sample expedition. ``` -## 3) Expedition scheduling & ship configuration +## 3) Expedition scheduling & configuration ```{important} This section describes the process of finalising the expedition schedule and instrument selection using the `virtualship plan` application. This is the recommended way for most users but when expeditions become larger with many waypoints, it can become cumbersome to use the planning tool (note, using VirtualShip in a remote terminal / cloud-based environment can also introduce lag in the user-interface). **In this case, you may prefer to edit the `expedition.yaml` file directly (see [here](./tutorials/working_with_expedition_yaml.md) for more details on how to do so)**. @@ -82,6 +82,23 @@ VirtualShip is capable of taking underway temperature and salinity measurements, For the underway ADCP, there is a choice of using the 38 kHz OceanObserver or the 300 kHz SeaSeven version (see [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#ADCP) for more detail on the two ADCP types). +### Instrument configuration + +The most important instrument configuration setting to consider is the list of **sensors** for each instrument, which controls what type of measurements/variables the instrument records in the simulation and therefore what output data you will receive for each instrument. + +Sensor lists can be configured for each instrument under _Ship Config Editor_ > _Instrument Configurations_. For example, for the CTD instrument, you can specify which sensors to include in the simulation (e.g., `TEMPERATURE`, `SALINITY`, `OXYGEN`, etc.) by toggling the respective switches on or off. + +```{note} +Sensor choices are only relevant for the instruments you plan to deploy as [underway measurements](#underway-measurements) or at waypoints across your expedition schedule [(see below)](#instrument-selection). For example, if you do not select to deploy a CTD at any of your waypoints, the CTD sensor choices will not affect any output data. +``` + +```{tip} +See [here](../documentation/full_sensor_list.md) for more information on the sensors available for each instrument. + +``` + +There are other instrument configurations settings that can be adjusted in the editor as well (e.g. `max_depth` for the CTD), but these are more advanced and in most cases do not need to be changed from the default values. + ### Waypoint datetimes @@ -107,15 +124,11 @@ The MFP route planning tool will give estimated durations of sailing between sit You should now consider which measurements are to be taken at each sampling site, and therefore which instruments need to be selected in the planning tool. ```{tip} -Click [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) for more information on what measurement options are available, and a brief introduction to each instrument. +Click [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) for more information on what instrument options are available, and a brief introduction to each instrument. ``` You can make instrument selections for each waypoint in the same sub-panels as the [waypoint time](#waypoint-datetimes) selection by simply switching each on or off. Multiple instruments are allowed at each waypoint. -```{note} -For advanced users: you can also make further customisations to behaviours of all instruments under _Ship Config Editor_ > _Instrument Configurations_. -``` - ### Save changes When you are happy with your ship configuration and schedule plan, press _Save Changes_. From baa982aa43d99a8080b0b146164a03c81c10cd77 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 12:04:37 +0200 Subject: [PATCH 06/17] update sail_the_ship with sensor configuration instructions --- .../assignments/Sail_the_ship.ipynb | 20 +++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/user-guide/assignments/Sail_the_ship.ipynb b/docs/user-guide/assignments/Sail_the_ship.ipynb index 1e1f439d..b4b94f3d 100644 --- a/docs/user-guide/assignments/Sail_the_ship.ipynb +++ b/docs/user-guide/assignments/Sail_the_ship.ipynb @@ -157,7 +157,7 @@ "The next step is to finalise the expedition schedule plan, including setting times and instrument selection choices for each waypoint, as well as configuring the ship (including any underway measurement instruments). \n", "\n", "
\n", - "**NOTE**: This section describes the process of finalising the expedition schedule and instrument selection using the `virtualship plan` application. For expeditions with many waypoints, it can become cumbersome to use the planning tool (note, using VirtualShip in a remote terminal / cloud-based environment can also introduce lag in the user-interface). **In this case, you may prefer to edit the** `expedition.yaml` **file directly (see [here](../tutorials/working_with_expedition_yaml.md) for more details on how to do so)**.\n", + "**Note**: This section describes the process of finalising the expedition schedule and instrument selection using the `virtualship plan` application. For expeditions with many waypoints, it can become cumbersome to use the planning tool (note, using VirtualShip in a remote terminal / cloud-based environment can also introduce lag in the user-interface). **In this case, you may prefer to edit the** `expedition.yaml` **file directly (see [here](../tutorials/working_with_expedition_yaml.md) for more details on how to do so)**.\n", "
\n", "\n", "\n", @@ -177,6 +177,22 @@ "\n", "For the underway ADCP, there is a choice of using the 38 kHz OceanObserver or the 300 kHz SeaSeven version (see [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#ADCP) for more detail on the two ADCP types).\n", "\n", + "### Instrument/sensor configuration\n", + "\n", + "The most important instrument configuration setting to consider is the list of **sensors** for each instrument, which controls what type of measurements/variables the instrument records in the simulation and therefore what output data you will receive for each instrument.\n", + "\n", + "Sensor lists can be configured for each instrument under _Ship Config Editor_ > _Instrument Configurations_. For example, for the CTD instrument, you can specify which sensors to include in the simulation (e.g., `TEMPERATURE`, `SALINITY`, `OXYGEN`, etc.) by toggling the respective switches on or off.\n", + "\n", + "
\n", + "**Note**: Sensor choices are only relevant for the instruments you plan to deploy as [underway measurements](#underway-measurements) or at waypoints across your expedition schedule [(see below)](#instrument-selection). For example, if you do not select to deploy a CTD at any of your waypoints, the CTD sensor choices will not affect any output data.\n", + "
\n", + "\n", + "
\n", + "**TIP**: See [here](../documentation/full_sensor_list.md) for more information on the sensors available for each instrument.\n", + "
\n", + "\n", + "There are other instrument configurations settings that can be adjusted in the editor as well (e.g. `max_depth` for the CTD), but these are more advanced and in most cases do not need to be changed from the default values.\n", + "\n", "### Waypoint datetimes\n", "\n", "
\n", @@ -200,7 +216,7 @@ "You should now consider which measurements are to be taken at each sampling site (think about those required for your chosen research question), and therefore which instruments need to be selected in the planning tool at each waypoint.\n", "\n", "
\n", - "**Tip**: Click [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) for more information on what measurement options are available, and a brief introduction to each instrument.\n", + "**Tip**: Click [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) for more information on which instruments are available in VirtualShip, and a brief introduction to each.\n", "
\n", "\n", "You can make instrument selections for each waypoint in the same sub-panels as the [waypoint time](#waypoint-datetimes) selection by simply switching each on or off. Multiple instruments are allowed at each waypoint.\n", From a8a4978e8fb1dfd631388af9372fb7c71a33630b Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 12:04:44 +0200 Subject: [PATCH 07/17] enahnce phrasing --- docs/user-guide/quickstart.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/quickstart.md b/docs/user-guide/quickstart.md index a4f75997..7d984111 100644 --- a/docs/user-guide/quickstart.md +++ b/docs/user-guide/quickstart.md @@ -82,7 +82,7 @@ VirtualShip is capable of taking underway temperature and salinity measurements, For the underway ADCP, there is a choice of using the 38 kHz OceanObserver or the 300 kHz SeaSeven version (see [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#ADCP) for more detail on the two ADCP types). -### Instrument configuration +### Instrument/sensor configuration The most important instrument configuration setting to consider is the list of **sensors** for each instrument, which controls what type of measurements/variables the instrument records in the simulation and therefore what output data you will receive for each instrument. @@ -94,7 +94,6 @@ Sensor choices are only relevant for the instruments you plan to deploy as [unde ```{tip} See [here](../documentation/full_sensor_list.md) for more information on the sensors available for each instrument. - ``` There are other instrument configurations settings that can be adjusted in the editor as well (e.g. `max_depth` for the CTD), but these are more advanced and in most cases do not need to be changed from the default values. @@ -124,7 +123,7 @@ The MFP route planning tool will give estimated durations of sailing between sit You should now consider which measurements are to be taken at each sampling site, and therefore which instruments need to be selected in the planning tool. ```{tip} -Click [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) for more information on what instrument options are available, and a brief introduction to each instrument. +Click [here](https://virtualship.readthedocs.io/en/latest/user-guide/assignments/Research_proposal_intro.html#Measurement-Options) for more information on what instrument options are available in VirtualShip, and a brief introduction to each. ``` You can make instrument selections for each waypoint in the same sub-panels as the [waypoint time](#waypoint-datetimes) selection by simply switching each on or off. Multiple instruments are allowed at each waypoint. From 6b3628ed1d123d00fee7f3639753d28c6d0b199b Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 13:05:42 +0200 Subject: [PATCH 08/17] add testing that docs match the code for sensor options, ensure instrument column entries are consistent --- .../documentation/full_sensor_list.md | 38 ++++++----- tests/test_utils.py | 65 ++++++++++++++++++- 2 files changed, 84 insertions(+), 19 deletions(-) diff --git a/docs/user-guide/documentation/full_sensor_list.md b/docs/user-guide/documentation/full_sensor_list.md index 1cc072a7..7e108b67 100644 --- a/docs/user-guide/documentation/full_sensor_list.md +++ b/docs/user-guide/documentation/full_sensor_list.md @@ -6,21 +6,23 @@ The following table provides a comprehensive list of available sensors for each Trying to add a sensor to an instrument that does not support it will result in errors in VirtualShip. Always refer to this table to check which sensors are available for each instrument. ``` -| Instrument | Sensor Name | Description | Units | Category | -| :--------------------- | :----------------- | :-------------------------------------------------- | :----------------------------------- | :-------------- | -| **ADCP** | VELOCITY | Current velocities (eastward (u) and northward (v)) | m/s | Physical | -| **Ship Underwater ST** | TEMPERATURE | Temperature | °C | Physics | -| | SALINITY | Salinity | psu | Physics | -| **CTD** | TEMPERATURE | Temperature | °C | Physics | -| | SALINITY | Salinity | psu | Physics | -| | OXYGEN | Oxygen concentration | mmol m-3 | Biogeochemistry | -| | CHLOROPHYLL | Chlorophyll concentration | mmol m-3 | Biogeochemistry | -| | NITRATE | Nitrate concentration | mmol m-3 | Biogeochemistry | -| | PHOSPHATE | Phosphate concentration | mmol m-3 | Biogeochemistry | -| | PH | pH | - | Biogeochemistry | -| | PHYTOPLANKTON | Phytoplankton concentration in carbon | mmol m-3 | Biogeochemistry | -| | PRIMARY_PRODUCTION | Net primary production | mmol m-3 day-1 | Biogeochemistry | -| **ARGO_FLOAT** | TEMPERATURE | Temperature | °C | Physics | -| | SALINITY | Salinity | psu | Physics | -| **DRIFTER** | TEMPERATURE | Temperature | °C | Physics | -| **XBT** | TEMPERATURE | Temperature | °C | Physics | + + +| Instrument | Sensor Name | Description | Units | Category | +| :------------------------------------- | :----------------- | :-------------------------------------------------- | :----------------------------------- | :-------------- | +| **ADCP** | VELOCITY | Current velocities (eastward (u) and northward (v)) | m/s | Physical | +| **UNDERWATER_ST** (Ship Underwater ST) | TEMPERATURE | Temperature | °C | Physics | +| | SALINITY | Salinity | psu | Physics | +| **CTD** | TEMPERATURE | Temperature | °C | Physics | +| | SALINITY | Salinity | psu | Physics | +| | OXYGEN | Oxygen concentration | mmol m-3 | Biogeochemistry | +| | CHLOROPHYLL | Chlorophyll concentration | mmol m-3 | Biogeochemistry | +| | NITRATE | Nitrate concentration | mmol m-3 | Biogeochemistry | +| | PHOSPHATE | Phosphate concentration | mmol m-3 | Biogeochemistry | +| | PH | pH | - | Biogeochemistry | +| | PHYTOPLANKTON | Phytoplankton concentration in carbon | mmol m-3 | Biogeochemistry | +| | PRIMARY_PRODUCTION | Net primary production | mmol m-3 day-1 | Biogeochemistry | +| **ARGO_FLOAT** | TEMPERATURE | Temperature | °C | Physics | +| | SALINITY | Salinity | psu | Physics | +| **DRIFTER** | TEMPERATURE | Temperature | °C | Physics | +| **XBT** | TEMPERATURE | Temperature | °C | Physics | diff --git a/tests/test_utils.py b/tests/test_utils.py index fde6796f..3196fa03 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -1,12 +1,13 @@ import datetime +import re from pathlib import Path import numpy as np import pytest import xarray as xr -from parcels import FieldSet, JITParticle, ScipyParticle, Variable import virtualship.utils +from parcels import FieldSet, JITParticle, ScipyParticle, Variable from virtualship.instruments.sensors import SensorType from virtualship.instruments.types import InstrumentType from virtualship.models.expedition import Expedition, SensorConfig @@ -399,3 +400,65 @@ def test_build_particle_class_scipy_base(): ParticleClass = build_particle_class_from_sensors(sensors, nonsensor, ScipyParticle) assert issubclass(ParticleClass, ScipyParticle) + + +def test_allowed_sensors_matches_docs(): + """Test that SUPPORTED_SENSORS_MAP (sensors allowed for each instrument) matches the sensor table in full_sensor_list.md.""" + # local imports to trigger instrument registration and avoid potential circular imports + import virtualship.instruments # noqa: F401 - ensures all @register_instrument decorators run + from virtualship.utils import INSTRUMENT_CLASS_MAP, SUPPORTED_SENSORS_MAP + + docs_path = ( + Path(__file__).parent.parent + / "docs/user-guide/documentation/full_sensor_list.md" + ) + content = docs_path.read_text(encoding="utf-8") + + display_name_to_instrument_type: dict[str, InstrumentType] = { + instrument_type.value: instrument_type + for instrument_type in INSTRUMENT_CLASS_MAP + if isinstance(instrument_type, InstrumentType) + } # all instruments should use their enum value as the bold display name in the markdown table + + # parse markdown table rows + row_pattern = re.compile(r"^\|([^|]*)\|([^|]*)\|.*$", re.MULTILINE) + + expected: dict[InstrumentType, set[SensorType]] = {} + current_instrument: InstrumentType | None = None + + for match in row_pattern.finditer(content): + instrument_cell = match.group(1).strip() + sensor_cell = match.group(2).strip() + + # extract only the **bold** text from the cell (e.g. "**UNDERWATER_ST** (Ship Underwater ST)" -> "UNDERWATER_ST") + bold_match = re.search(r"\*\*(.+?)\*\*", instrument_cell) + instrument_name = bold_match.group(1).strip() if bold_match else "" + + if instrument_name and instrument_name in display_name_to_instrument_type: + current_instrument = display_name_to_instrument_type[instrument_name] + if current_instrument not in expected: + expected[current_instrument] = set() + + # skip irrelevant cells + if ( + not sensor_cell + or sensor_cell.startswith(":") + or sensor_cell == "Sensor Name" + ): + continue + + sensor_name = sensor_cell.strip() + if current_instrument is not None and sensor_name: + expected[current_instrument].add(SensorType(sensor_name)) + + # verify each instrument in the docs is registered and has matching sensors + for instrument_type, doc_sensors in expected.items(): + assert instrument_type in SUPPORTED_SENSORS_MAP, ( + f"{instrument_type} is listed in full_sensor_list.md but not found in SUPPORTED_SENSORS_MAP." + ) + registered_sensors = set(SUPPORTED_SENSORS_MAP[instrument_type]) + assert registered_sensors == doc_sensors, ( + f"Sensor mismatch for {instrument_type}:\n" + f" In docs: {sorted(s.value for s in doc_sensors)}\n" + f" In code: {sorted(s.value for s in registered_sensors)}\n" + ) From e23087b61cae509ec7be54be780d57f4af21f48c Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Wed, 20 May 2026 13:10:46 +0200 Subject: [PATCH 09/17] add check that all instruments in code are addressed in sensor table --- tests/test_utils.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/test_utils.py b/tests/test_utils.py index 3196fa03..4ab96654 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -462,3 +462,9 @@ def test_allowed_sensors_matches_docs(): f" In docs: {sorted(s.value for s in doc_sensors)}\n" f" In code: {sorted(s.value for s in registered_sensors)}\n" ) + + # verify each instrument registered in code is also covered in the docs + for instrument_type in SUPPORTED_SENSORS_MAP: + assert instrument_type in expected, ( + f"{instrument_type} is registered in SUPPORTED_SENSORS_MAP but not listed in full_sensor_list.md." + ) From e9cae63693ec54697a63ccdb0c3f390c22883a0a Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:14:12 +0200 Subject: [PATCH 10/17] "physics" consistency --- docs/user-guide/documentation/full_sensor_list.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/documentation/full_sensor_list.md b/docs/user-guide/documentation/full_sensor_list.md index 7e108b67..d120f1e6 100644 --- a/docs/user-guide/documentation/full_sensor_list.md +++ b/docs/user-guide/documentation/full_sensor_list.md @@ -10,7 +10,7 @@ Trying to add a sensor to an instrument that does not support it will result in | Instrument | Sensor Name | Description | Units | Category | | :------------------------------------- | :----------------- | :-------------------------------------------------- | :----------------------------------- | :-------------- | -| **ADCP** | VELOCITY | Current velocities (eastward (u) and northward (v)) | m/s | Physical | +| **ADCP** | VELOCITY | Current velocities (eastward (u) and northward (v)) | m/s | Physics | | **UNDERWATER_ST** (Ship Underwater ST) | TEMPERATURE | Temperature | °C | Physics | | | SALINITY | Salinity | psu | Physics | | **CTD** | TEMPERATURE | Temperature | °C | Physics | From 0b0567f0717d8573c9e57a9649fd73de184aa3fb Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:19:44 +0200 Subject: [PATCH 11/17] make descriptions and categories consistent with copernicus_products doc --- .../documentation/full_sensor_list.md | 36 +++++++++---------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/user-guide/documentation/full_sensor_list.md b/docs/user-guide/documentation/full_sensor_list.md index d120f1e6..56a0fb80 100644 --- a/docs/user-guide/documentation/full_sensor_list.md +++ b/docs/user-guide/documentation/full_sensor_list.md @@ -8,21 +8,21 @@ Trying to add a sensor to an instrument that does not support it will result in -| Instrument | Sensor Name | Description | Units | Category | -| :------------------------------------- | :----------------- | :-------------------------------------------------- | :----------------------------------- | :-------------- | -| **ADCP** | VELOCITY | Current velocities (eastward (u) and northward (v)) | m/s | Physics | -| **UNDERWATER_ST** (Ship Underwater ST) | TEMPERATURE | Temperature | °C | Physics | -| | SALINITY | Salinity | psu | Physics | -| **CTD** | TEMPERATURE | Temperature | °C | Physics | -| | SALINITY | Salinity | psu | Physics | -| | OXYGEN | Oxygen concentration | mmol m-3 | Biogeochemistry | -| | CHLOROPHYLL | Chlorophyll concentration | mmol m-3 | Biogeochemistry | -| | NITRATE | Nitrate concentration | mmol m-3 | Biogeochemistry | -| | PHOSPHATE | Phosphate concentration | mmol m-3 | Biogeochemistry | -| | PH | pH | - | Biogeochemistry | -| | PHYTOPLANKTON | Phytoplankton concentration in carbon | mmol m-3 | Biogeochemistry | -| | PRIMARY_PRODUCTION | Net primary production | mmol m-3 day-1 | Biogeochemistry | -| **ARGO_FLOAT** | TEMPERATURE | Temperature | °C | Physics | -| | SALINITY | Salinity | psu | Physics | -| **DRIFTER** | TEMPERATURE | Temperature | °C | Physics | -| **XBT** | TEMPERATURE | Temperature | °C | Physics | +| Instrument | Sensor Name | Description | Units | Category | +| :------------------------------------- | :----------------- | :---------------------------------------------------------------------------------------- | :----------------------------------- | :------------- | +| **ADCP** | VELOCITY | Current velocities (Eastward Sea Water Velocity (U) and Northward Sea Water Velocity (V)) | m/s | Physical | +| **UNDERWATER_ST** (Ship Underwater ST) | TEMPERATURE | Sea Water Potential Temperature | °C | Physical | +| | SALINITY | Sea Water Salinity | psu | Physical | +| **CTD** | TEMPERATURE | Sea Water Potential Temperature | °C | Physical | +| | SALINITY | Sea Water Salinity | psu | Physical | +| | OXYGEN | Mole Concentration of Dissolved Molecular Oxygen in Sea Water | mmol m-3 | Biogeochemical | +| | CHLOROPHYLL | Mass Concentration of Chlorophyll a in Sea Water | mmol m-3 | Biogeochemical | +| | NITRATE | Mole Concentration of Nitrate in Sea Water | mmol m-3 | Biogeochemical | +| | PHOSPHATE | Mole Concentration of Phosphate in Sea Water | mmol m-3 | Biogeochemical | +| | PH | Sea Water pH | - | Biogeochemical | +| | PHYTOPLANKTON | Mole Concentration of Phytoplankton | mmol m-3 | Biogeochemical | +| | PRIMARY_PRODUCTION | Net Primary Production of Biomass | mmol m-3 day-1 | Biogeochemical | +| **ARGO_FLOAT** | TEMPERATURE | Sea Water Potential Temperature | °C | Physical | +| | SALINITY | Sea Water Salinity | psu | Physical | +| **DRIFTER** | TEMPERATURE | Sea Water Potential Temperature | °C | Physical | +| **XBT** | TEMPERATURE | Sea Water Potential Temperature | °C | Physical | From 2008fb6b042e2f6db584ed9c0afda0ec439c2af1 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:32:16 +0200 Subject: [PATCH 12/17] make reference to the copernicus products docs --- docs/user-guide/documentation/full_sensor_list.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/user-guide/documentation/full_sensor_list.md b/docs/user-guide/documentation/full_sensor_list.md index 56a0fb80..9b22c05d 100644 --- a/docs/user-guide/documentation/full_sensor_list.md +++ b/docs/user-guide/documentation/full_sensor_list.md @@ -2,6 +2,8 @@ The following table provides a comprehensive list of available sensors for each instrument. These sensors can be specified via the `virtualship plan` tool (see the [Quickstart guide](../quickstart.md)) or in the `sensors` section of the respective instrument configuration in the `expedition.yaml` file (see the [working with expedition.yaml tutorial](../tutorials/working_with_expedition_yaml.md)). +The source data associated with these instrument sensor variables are described in the [Copernicus Marine products](copernicus_products.md) page. + ```{note} Trying to add a sensor to an instrument that does not support it will result in errors in VirtualShip. Always refer to this table to check which sensors are available for each instrument. ``` From 0685c905bccc2516a75bec823d2afd41cf6dd1c6 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:33:12 +0200 Subject: [PATCH 13/17] "CMEMS" -> "Copernicus Marine Service --- docs/user-guide/documentation/copernicus_products.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/documentation/copernicus_products.md b/docs/user-guide/documentation/copernicus_products.md index fec42d44..7f18ccb1 100644 --- a/docs/user-guide/documentation/copernicus_products.md +++ b/docs/user-guide/documentation/copernicus_products.md @@ -60,7 +60,7 @@ For biogeochemical variables `ph` and `phyc`, monthly products are required for * Certain BGC variables (`ph`, `phyc`) are only available as monthly products in hindcast and hindcast interim periods. ``` -##### CMEMS variables legend +##### Copernicus Marine Service variables legend | Variable Code | Full Variable Name | Category | | :------------ | :------------------------------------------------------------ | :------------- | From f78b9662eb9438e344baf5500b2e41d765503391 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:34:13 +0200 Subject: [PATCH 14/17] reorder docs pages --- docs/user-guide/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/index.md b/docs/user-guide/index.md index d5d4a124..24eee0c8 100644 --- a/docs/user-guide/index.md +++ b/docs/user-guide/index.md @@ -14,8 +14,8 @@ assignments/index ```{toctree} :maxdepth: 1 +documentation/full_sensor_list.md documentation/copernicus_products.md documentation/pre_download_data.md documentation/example_copernicus_download.ipynb -documentation/full_sensor_list.md ``` From 5ff4eb10f6fbca55d0c87930d9e5185e73b04c49 Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:36:17 +0200 Subject: [PATCH 15/17] "Copernicus Marine" + "Service" --- docs/user-guide/documentation/copernicus_products.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/documentation/copernicus_products.md b/docs/user-guide/documentation/copernicus_products.md index 7f18ccb1..54737fc6 100644 --- a/docs/user-guide/documentation/copernicus_products.md +++ b/docs/user-guide/documentation/copernicus_products.md @@ -1,4 +1,4 @@ -# Copernicus Marine products +# Copernicus Marine Service products VirtualShip supports running experiments anywhere in the global ocean from 1993 through to the present day (and approximately two weeks into the future), using the suite of products available from the [Copernicus Marine Data Store](https://data.marine.copernicus.eu/products). From 0fe134055c403bc3048b27564ae5d45ea445dabd Mon Sep 17 00:00:00 2001 From: j-atkins <106238905+j-atkins@users.noreply.github.com> Date: Thu, 21 May 2026 09:45:04 +0200 Subject: [PATCH 16/17] further tidy up docs --- .../documentation/example_copernicus_download.ipynb | 2 +- docs/user-guide/documentation/pre_download_data.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/user-guide/documentation/example_copernicus_download.ipynb b/docs/user-guide/documentation/example_copernicus_download.ipynb index 1e630520..ae5774f7 100644 --- a/docs/user-guide/documentation/example_copernicus_download.ipynb +++ b/docs/user-guide/documentation/example_copernicus_download.ipynb @@ -7,7 +7,7 @@ "source": [ "# Example Copernicus data download \n", "\n", - "This notebook provides a rough, non-optimised example of how to download Copernicus Marine data using the `copernicusmarine` Python package.\n", + "This notebook provides a rough, non-optimised example of how to download Copernicus Marine Service data using the `copernicusmarine` [toolbox](https://github.com/mercator-ocean/copernicus-marine-toolbox?tab=readme-ov-file).\n", "\n", "This will download:\n", "- Global bathymetry data (static)\n", diff --git a/docs/user-guide/documentation/pre_download_data.md b/docs/user-guide/documentation/pre_download_data.md index f37912e9..d58100cc 100644 --- a/docs/user-guide/documentation/pre_download_data.md +++ b/docs/user-guide/documentation/pre_download_data.md @@ -1,6 +1,6 @@ # Pre-downloading data -By default, VirtualShip will automatically 'stream' data from the Copernicus Marine Service via the [copernicusmarine toolbox](https://github.com/mercator-ocean/copernicus-marine-toolbox?tab=readme-ov-file). However, for users who wish to manage data locally, it is possible to pre-download the required datasets and feed them into VirtualShip simulations. +By default, VirtualShip will automatically 'stream' data from the Copernicus Marine Service via the `copernicusmarine` [toolbox](https://github.com/mercator-ocean/copernicus-marine-toolbox?tab=readme-ov-file). However, for users who wish to manage data locally, it is possible to pre-download the required datasets and feed them into VirtualShip simulations. @@ -74,7 +74,7 @@ The following assumptions are also made about the data: 1. All pre-downloaded data files must be in NetCDF format (`.nc`). 2. Physical data files must contain the following variables: `uo`, `vo`, `so`, `thetao` - Or these strings must appear as substrings within the variable names (e.g. `uo_glor` is acceptable for `uo`). -3. If using BGC instruments (e.g. `CTD_BGC`), the relevant biogeochemical data files must contain the following variables: `o2`, `chl`, `no3`, `po4`, `nppv`, `ph`, `phyc`. +3. If using BGC-enabled instruments (e.g. BGC variables on the `CTD`), the relevant biogeochemical data files must contain the following variables: `o2`, `chl`, `no3`, `po4`, `nppv`, `ph`, `phyc`. - Or these strings must appear as substrings within the variable names (e.g. `o2_glor` is acceptable for `o2`). 4. Bathymetry data files must contain a variable named `deptho`. From 1819d91f52308451a1a3618fbd69ec4b70c20037 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 21 May 2026 08:24:36 +0000 Subject: [PATCH 17/17] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- tests/test_utils.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test_utils.py b/tests/test_utils.py index 4ab96654..2628793d 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -5,9 +5,9 @@ import numpy as np import pytest import xarray as xr +from parcels import FieldSet, JITParticle, ScipyParticle, Variable import virtualship.utils -from parcels import FieldSet, JITParticle, ScipyParticle, Variable from virtualship.instruments.sensors import SensorType from virtualship.instruments.types import InstrumentType from virtualship.models.expedition import Expedition, SensorConfig