v0.13.1

Changelog

Features

• Added first-class read-only support and a migration-status helper to the Database API.

  Database.from_config(..., read_only=True) rewrites file-based SQLite URLs to read-only URI form and skips migrations,
  and Database.migration_status(config) reports the current/head revisions and state. (#624)

Changes

• cddf9f6 Bump version: 0.13.0 → 0.13.1
• 625f0df docs: Changelog
• 6b57d18 fix(db): address PR review comments
• 86b10fd chore: update the notebook
• a16f701 docs: pin openapi-python-client package name in API guide
• 77f77cb docs: add changelog entry for #624
• b169132 feat(db): add read-only mode and migration_status helper

v0.13.0

Announcements

(#582) reworked how the data cache works. It is recommended to rerun ref providers setup after upgrading.

Changelog

Features

• Added timeseries extraction for ESMValTool regional historical and ozone polar cap diagnostics, and updated file patterns to match the current ESMValTool output directory structure. (#607)
• Added ref executions reingest command to re-ingest existing execution results without re-running diagnostics. Creates a new immutable execution record with a timestamped output fragment, leaving the original execution untouched. (#610)
• Added ref db CLI subcommand group for database management. Includes commands for running migrations, checking schema status, viewing migration history, creating backups, executing SQL queries, and listing tables. (#615)

Improvements

• Fetch ESMValTool recipes when installing the provider. (#582)
• Unify facet filter parsing across CLI commands. --filter in executions list-groups and delete-groups now supports multiple values for the same key with OR semantics (e.g., --filter source_id=A --filter source_id=B), consistent with --dataset-filter in datasets list and solve. (#613)
• Improved solver performance by batching dataset finalisation before grouping, avoiding redundant file I/O when multiple groups share overlapping datasets. (#616)

Bug Fixes

• Fixed CLI test isolation by making the invoke_cli fixture depend on the config fixture, ensuring tests use an isolated database rather than the user's real one. Also marked RLIMIT_AS tests as expected failures on macOS where this resource limit is not supported. (#611)
• Copied the scratch directory for the previous execution when reingesting (#612)

Changes

• 66f8c69 Bump version: 0.12.3 → 0.13.0
• 0fa7bf2 refactor: improve cache migration handling in DatasetRegistryManager
• 30e5a7a feat(core): add cache migration helper for registry layout changes
• 0c7a903 chore(deps): bump pyjwt from 2.11.0 to 2.12.0
• 7fc74f8 chore(deps): bump cryptography from 46.0.5 to 46.0.7
• f3dc942 chore(deps): bump tornado from 6.5.4 to 6.5.5
• f07af91 docs: add changelog for #616
• 4cb8034 perf: batch dataset finalisation before grouping in solver
• fb6105c chore: remove ansi
• b37ae2b fix: address PR review comments
• 88e0104 refactor(db): extract _get_sqlite_path helper for URL parsing
• a176627 test(cli): add tests for db commands without an existing database
• 8d4a1dd docs: add changelog entry for db CLI commands
• b857b79 feat(cli): add db subcommand group for database management
• cfcee3e fix: copy then build
• 450759f docs: add changelog entry for PR #613
• 94ac1dc refactor: unify facet filter parsing across CLI commands
• 48ca342 test: Reorganise tests for reingest
• 12b76f3 refactor: add update_dirty param to handle_execution_result
• 283bcf4 fix: changelog
• 7baf0ac docs: add changelog entry for reingest selective copy fix
• ce3f377 fix(reingest): use handle_execution_result for selective file copying
• 52be560 docs: add changelog entry for PR #611
• 624156b fix(test): isolate CLI tests from user database and xfail RLIMIT_AS on macOS
• 7ec0bb4 fix(reingest): always reingest from oldest (original) execution
• fed124c fix: address PR review comments for reingest
• b1eaba3 refactor(fragment): replace collision-checking with timestamp-based output fragments
• 0480279 fix(solver): build definition early to avoid MagicMock key in tests
• 375ef6c test: add coverage for fragment collision, dataset links, and execution state
• df3350d refactor(reingest): simplify to always-versioned immutable flow
• 9f05ba0 fix(test): handle ANSI escape codes in typer missing option error
• 10a19d1 refactor(executor): extract shared ingest_execution_result and add equivalence tests
• d5a4218 refactor(executor): deduplicate reingest and result_handling ingestion logic
• c02beab fix: clean up finalized->finalised
• 3403780 test(reingest): add CLI and unit test coverage for reingest command
• 303a3fb refactor(test): deduplicate reingest test fixtures and helpers
• 557eee4 test(reingest): improve coverage to 74% with targeted unit tests
• 08d8ddb chore: fix update to environs api
• 7d629f1 fix(executor): remove eager reingest import from executor init
• f977aa1 fix(reingest): address PR review comments
• 484d6bc chore: add changelog entry for PR #610
• 42a9e33 style(cli): remove redundant inline comments in reingest command
• 6ec695a refactor(reingest): consolidate duplicated ingestion functions and add scratch cleanup
• 5f06821 feat(cli): add reingest command with enum-based mode selection
• 8383ae6 chore(deps-dev): bump the python-dependencies group across 1 directory with 16 updates
• 7ff0cec chore(deps): bump the github-actions group across 1 directory with 5 updates
• a2ad979 test(esmvaltool): add tests for byte string index decoding and _region_to_filename
• 9752736 chore: add changelog entry for PR #607
• ba5d941 feat(esmvaltool): add timeseries extraction for regional and ozone diagnostics
• a3a65fd fix: changelog
• 65f9ed9 Put everything under ~/.cache/climate_ref
• 06cf52b Add changelog
• 41d07f5 Fetch recipes in advance

v0.12.3

Announcements

Changelog

Features

• Added ref datasets stats and ref executions stats CLI commands for viewing summary statistics without listing individual records. (#584)

Bug Fixes

• Fixed zero-emission-commitment diagnostic failures caused by space-separated activity_id values creating path mismatches, incorrect parent timerange computation when datasets are split across multiple files, and models with esm-1pctCO2 as parent experiment being incorrectly scheduled (see #586). (#585)
• Fixed experiment selection for computing ZEC. (#589)
• Solve regression tests now use the local default_ignore_datasets.yaml instead of downloading from the main branch on GitHub, ensuring tests reflect the current ignore list. (#606)
• Use flexible time stamp for PMP annual cycle. (#465)

Improvements

• Enable multiple-file-input for PMP's variability modes diagnostics. (#583)-

Improved Documentation

• Added a how-to guide on controlling memory use and parallism during diagnostic execution. (#591)

Changes

• a3831a9 Bump version: 0.12.2 → 0.12.3
• cba52e0 chore(docs): remove --strict flag from mkdocs build
• 653d27b test(pmp): add tests for get_wildcard_pattern and validate common prefix
• 6b94c6c chore: add changelog for #606
• 2d9dbfa fix(tests): use local ignore list in solve regression tests
• c362381 clean up
• 8c3bfb4 clean up
• 01ed2a9 clean up
• d85aa24 clean up
• 2c6943e add change log
• 3ddd098 change input checker, enable multi-file input
• 831c60d docs: cleanups
• 2b53834 docs: rename
• 6789c06 docs: Placeholder QA page
• b60b9a2 Add docs on configuring Dask
• eec3610 Correct experiment selection for computing ZEC
• b42bebf chore: Ignore failing datasets in the sea-ice-sensitivity runs
• 1a8b0ac fix(esmvaltool): exclude ZEC models with esm-1pctCO2 parent experiment
• 7d255d9 chore: add changelog for #585
• 50b4620 fix(esmvaltool): resolve ZEC failures from activity_id mismatch and wrong timerange
• fab8b9c chore: add changelog for #584
• 8a13616 feat(cli): add stats subcommand to datasets and executions

v0.12.2

Changelog

Bug Fixes

• Improved conda error logging by capturing stderr in solve logs and avoiding unnecessary stacktraces. (#580)

Trivial/Internal Changes

• #580

Changes

• 7bc2bff Bump version: 0.12.1 → 0.12.2
• 02c2e0c chore: add second EOF
• 78e84db Bump version: 0.12.0 → 0.12.1
• fac866b chore: support mip_id
• d09556c fix: update test to expect CondaCommandError instead of CalledProcessError
• ba662d2 docs: add changelog entries for PR #580
• 29bba46 fix: Log conda stderr to solve logs
• 7d269b6 chore: add extra cvs

v0.12.0

Changelog

Features

• Added ESMValTool ozone diagnostics. (#473)

• Metric for diagnostics of double ITCZ was added: Spatial corrleation of simulated DJF precipitation climatology against reference dataset over the area of 20S to 0 latitude and 100 to 210 longitude. (#557)

• Added lazy loading and finalisation support for CMIP7 datasets via DRS and complete parsers, matching the existing CMIP6 pattern.

  This removes the realm filter for CMIP7 executions as it cannot be parsed from the DRS.
  We strongly recommend the use of branded variables when filtering CMIP7 datasets to properly constrain the expected variables. (#571)

• Update ilamb3, remove the ohc-noaa diagnostic and add the evspsbl-pr diagnostic. (#573)

• Added time_units and calendar metadata to CMIP6 and CMIP7 datasets, enabling proper handling of non-standard CF calendars such as 360_day and noleap. Time values are now stored as cftime.datetime objects instead of datetime.datetime. (#574)

• Added CMIP7 data catalog and ESMValTool recipe variants for CMIP7 diagnostics. (#577)

Improvements

• Implemented a memory constraint using the environment variable MEMORY_LIMIT_PARSL_JOB_GB to set the memory limit (units: GB) for a PARSL worker. (#464)
• Extracted shared finalisation logic into FinaliseableDatasetAdapterMixin, reducing code duplication between CMIP6 and CMIP7 adapters. (#571)
• Use the metadata from the data catalog instead of reading netCDF files when determining ESMValTool branch times (#577)

Bug Fixes

• Fixed fire diagnostic CMIP7 data selection and recipe writing. (#540)
• Migrated to use a different post-processed JSON file for mapping CMIP6 compound names to CMIP7.
  This fixes some errors when dealing with tasmax/tasmin and the removes the out_name attribute which was correctly included in #530. (#547)
• Apply same version filtering logic to the regression tests as loading the data catalog (#570)

Changes

• cb55f38 Bump version: 0.11.1 → 0.12.0
• 7aefb3d chore(deps): bump the github-actions group with 2 updates
• 38a3866 chore: fix test
• d1f47f2 chore: Add regression outputs
• 4e01e96 chore: update catalog
• 46a12ea chore: add obs4mips requirement
• a0d92c6 test: Fix failing tests
• 27bb39f fix(solver): re-finalise group after constraints that add unfinalised rows
• 7bd41af chore: Add empty solves
• 5591831 chore: Support cmip6/7
• a3cddcd fix: sanitize non-YAML-safe values in save_datasets_to_yaml
• 055c3a8 chore: add test_specs
• 9777aa8 chore: ruff
• 1632d39 chore: run all solves
• 380d911 chore: add a cmip7 data catalog and regenerate the ESMValTool tests
• 126781b chore: add CMIP7 data catalog
• 003a2c6 chore: Use a subset of cmip6 source ids
• efc2461 chore: update catalog
• 4aa128e chore: add ozone
• 6f0f36c refactor: add a esgf_data_catalog_trimmed fixture
• dd95c00 chore: add back the catch
• ff5dd49 chore: cleanup is_na checks
• a6c1c91 chore: fetch obs4ref datasets too
• e5c68fb fix: add postgresql_using cast for String->DateTime downgrade
• 3697ab9 fix: use FALSE instead of 0 in migration for PostgreSQL boolean column
• 01c93c9 fix: use enum names in migration UPDATE for PostgreSQL compatibility
• e51352d chore: remove test
• 116520f chore: skip the diagnostic
• fa17b8e chore: Add regression file
• da8cbee docs: add changelog entry for time_units and calendar feature
• 98649d4 fix: handle mixed-calendar cftime comparison in contiguity constraint
• a403a21 chore: rename changelog to match PR #573
• 063e7cb chore: add lossy compression as the thetaO files are large
• 404b25b chore: add other missing variable
• 5e29107 chore: add missing variable
• c55b29c revert: Undo the changes due to new datasets
• b0f18d7 test: update tests
• 3d9560a chore: add migration
• 5a82ac6 feat: parse into cftime at boundary
• 981cb4f chore: add time_units and calendar
• 6741964 chore: read mandatory variables
• 09d9efc fix: Better handling of nan values
• 1b85499 docs: Update documenation about DRS parser
• 78f46df docs: changelog update
• e43f358 fix: correct the order of the constraints
• b044e53 chore: add changelog entries for PR #571
• 761022e chore: Clean up CMIP6 and CMIP7 test overlap
• 866be11 chore: refactor tests
• f90ca0d chore: update tests
• 9a600f8 chore: remove realm from filtering
• b78c97c feat: Add CMIP7 lazy loading
• 650dd6f chore: Fix regression outputs
• 88b9d5b chore: changelog
• 019f870 fix: apply version filtering to the regression analysis
• 911c87e chore: Use variable_name
• c81d78b chore: fix pre-commit
• 999d944 add change log
• 5c69847 rename double_ITCZ to doubleITCZ for consistency in output file naming
• 5e54e72 chore: add test case for double ITCZ
• 8ecfcc3 chore(deps-dev): bump jupyterlab in the python-dependencies group
• e9ad222 chore: clean up datarequirements
• e8fab3e chore: loosen facet filter type
• 5393d00 chore: remove ohc dataset
• 8815a50 chore: fix mypy
• 55ccbca add changelog
• 5fb2f31 add tests for memory limit
• 67d0378 expand ilamb3 ref configure files to allow for dictionaries with data requirement facets
• e5b3ca9 replace the hard-coded memory limit with an env variable
• da7bade initial commit
• 3cc6603 fix: lockfile
• 9f66c29 fix: Explicitly handle extra spurious extra coord
• 9aed0cf fix: correctly parse variant indexes in CMIP6-to-CMIP7 conversion
• 14f1cb3 chore: remove physical_parameter_name as it isn't needed
• 2cf846b changelog
• d6d3b9c bump ilamb3 version
• fe0b9f9 add ilamb model color selector, only 1 model but at least won't be black
• 55553f8 add the evspsbl-pr metric only over land as requested
• c77383e append the obs4REF dataset registry to the ilamb so these keys can be used in configure files
• ade4287 change how models are named in the plots and output filenames
• 2a6f5c8 changes to configure files, data references updated where possible
• 2deee16 Temporarily remove the ocean heat content diagnostic as per email from Forrest (on behalf of the MBTT) on 17-Feb
• 7123b57 fix: test
• 9386b77 chore: use the common cache dir
• abbf776 chore: use correct drs
• 38952dd chore: add missing attributes
• cd33274 Fix fire diagnostic CMIP7 data selection and recipe writing
• 166b744 docs: Changelog
• c713489 refactor: Refactor DReq variable handling and filename generation
• 91df83f added 473 to change log
• 5501551 automatic formatting fixes
• 7402cb7 cleaned up ozone.py
• b16785d update ozone-zonal diagnostic
• cb81694 add ESACCI-OZONE data (variable o3) for ESMValTool
• f1e8588 first draft of esmvaltool ozone diagnostics
• 003cde9 memory limit to a parsl worker in HPCExecutor

v0.11.1

Changelog

Bug Fixes

• Fixed DRS re-ingestion from crashing or regressing already-finalised datasets. Previously, re-ingesting the same directory with the DRS parser would either crash with a TypeError due to pd.NA comparisons, or overwrite finalised metadata with empty values. Finalised datasets are now skipped during DRS ingestion while still adding any new files.

  Reduced memory usage during dataset ingestion by releasing ORM objects from the SQLAlchemy session after each dataset commit, preventing unbounded memory growth on large archives. (#567)

Changes

• 271f0a8 Bump version: 0.11.0 → 0.11.1
• f96176c docs: add changelog for #567
• 56fe6ee fix: reduce memory during ingestion and add solve logging
• 9f57e32 fix: prevent DRS re-ingestion from regressing finalised datasets

v0.11.0

Announcements

This release brings large performance improvements for ingesting datasets, particularly for low IOPs filesystems. The previous ingestion process opened each NetCDF file to extract metadata which is slow for large archives.

A new "drs" CMIP6 parser has been introduced which uses the CMIP6 DRS for the initial ingest. The remaining metadata is then loaded at solve time if a dataset might be needed for a diagnostic. This allows users to ingests a large swath of data and only the datasets needed for diagnostics will be materialised.

This can be enabled by setting the cmip6_parser to drs.

Changelog

Breaking Changes

• Changed get_branding_suffix, get_realm, and get_cmip7_compound_name to require a table_id parameter in addition to variable_id, enabling Data Request compound name lookups. (#530)

• Failed diagnostic executions now clear the execution group's dirty flag,
  preventing automatic retry on subsequent solves.

  Previously, failed executions were retried indefinitely.
  Use ref solve --rerun-failed or ref executions flag-dirty to explicitly retry failed diagnostics.
  The solver also now skips duplicate submissions when an execution with the same dataset hash is already in progress. (#552)

Features

• Added a constraint to add the parent experiment. (#214)

• Added lazy dataset ingestion with two-phase finalisation. Datasets are now bootstrapped from directory structure metadata only (no file I/O), with full metadata extracted lazily at solve time after filtering narrows candidates. This dramatically reduces ingestion time for large CMIP6 archives on HPC parallel file systems. (#515)

• Added CMIP7 support to all ESMValTool diagnostics using OR-logic data requirements, enabling automatic evaluation of CMIP7 datasets alongside existing CMIP6 support. (#519)

• Added CMIP7 data requirements and test data specifications for all PMP diagnostics (annual cycle, ENSO, and variability modes). (#526)

• Added esgf_data_catalog test fixture and per-provider solver regression baselines using pre-generated parquet catalogs, enabling solver regression testing without requiring sample data downloads. (#529)

• Added structured CMIP6-to-CMIP7 variable mappings from the CMIP7 Data Request, with a typed DReqVariableMapping class for reliable branding suffix, realm, and output name lookups. (#530)

• TOML file for QAQC requirement from REF added. (#532)

• Added dimensions to files produced by ESMValTool diagnostics. (#534)

• Added CMIP7 data support to ILAMB diagnostics, enabling dual CMIP6/CMIP7 data requirements with branded variable name lookups and dynamic source type detection. (#535)

• Distinguished system errors (OOM, disk full, worker crash) from diagnostic logic errors when handling execution failures.
  System errors leave the execution group dirty so they are automatically retried on the next solve,
  while diagnostic errors clear the dirty flag to prevent retrying indefinitely with the same data.

  The solver also now skips duplicate submissions when an execution is already in progress for the same dataset hash.

  Added --rerun-failed and --no-wait flags to ref solve, and a new ref executions fail-running command for marking stuck executions as failed. (#552)

• Added validation that prevents DataRequirements from filtering or grouping on columns that require dataset finalisation, raising a clear error instead of silently producing empty results. (#561)

• Added --limit flag to the solve command to cap the number of executions, and --dataset-filter option to both solve and datasets list commands to filter input datasets by facet values before solving. (#562)

Improvements

• Improved solver performance by caching slug column lookups and avoiding expensive DataFrame string representation in debug logging. (#533)
• Removed the ecgtools dependency and 16 transitive dependencies (intake, intake-esm, joblib, zarr, etc.) by replacing it with a focused internal catalog builder module. This also eliminates the pydantic v1 deprecation warning that ecgtools was causing. File parsing now shows a tqdm progress bar. (#558)
• Replaced xarray with netCDF4 for metadata-only reads during dataset ingestion, significantly reducing per-file parsing overhead. (#559)
• Parallelised the finalise_datasets operation for CMIP6 datasets, mirroring the threaded approach used during ingest.
  The number of worker threads is controlled by the existing n_jobs parameter on CMIP6DatasetAdapter. (#564)

Bug Fixes

• Removed deprecated mix_stderr parameter from CliRunner in the test fixture, fixing compatibility with Click 8.3+. (#528)
• Fixed a ValueError in AddSupplementaryDataset when the data catalog contained duplicate index labels for supplementary datasets. (#537)
• Fixed confusion between variable_id and out_name in fake CMIP7 data. (#539)
• Improved the resiliance of the celery worker configuration to failures (#550)
• Mount a celeryconfig.py via ConfigMap for Flower so accept_content is read correctly by Celery's config loader. The CELERY_ACCEPT_CONTENT env var is not picked up by Flower/Kombu directly. The config is user-configurable via flower.celeryConfig in Helm values. (#556)
• Fixed REF_CMIP6_PARSER and REF_LOG_FORMAT environment variables not being applied because the Config class was missing the post-init hook for environment variable overrides. (#561)
• Resolved pandas FutureWarnings to support both pandas 2 and 3, including fixes for DataFrame concatenation with empty or all-NA entries and null-type mismatches in parquet round-trips. (#565)

Trivial/Internal Changes

• #538, #555

Changes

• 3bfd343 Bump version: 0.10.0 → 0.11.0
• ca8d500 chore: Update comment
• 4ffeaba chore: upgrade pins for ilamb
• 4396148 fix: revert compat=override on open_mfdataset
• a2b86e3 docs: add changelog for #565
• 4f5d1b0 chore: Upgrade lockfile and fix some errors
• e1a3b27 chore: add coverage
• 6d25567 chore: add default separator in alembic
• eef3980 fix: time_coder warning
• bc4e6ed chore: Pin to use tas
• d96d0fb fix(solver): preserve DataCatalog wrapper in apply_dataset_filters
• 95ae318 fix(tests): use to_frame() when accessing DataCatalog in solver tests
• ca4f8e2 docs: Changelog
• d89b359 chore: run the finalise in threads
• a8cf780 chore: clean up
• 51f8443 chore: add fix changelog entry for PR #561
• 4cb0432 feat(cli): add --dataset-filter option to datasets list command
• 22085b5 chore: add changelog entry for PR #561
• 17c47bd feat(solver): add --dataset-filter option to filter input datasets when solving
• 3142e2b chore: Support env variables for parser
• d335227 feat(solver): add optional --limit flag to solve command
• 23ca55b feat(solver): validate DataRequirement columns against adapter finalisation
• 30bd383 test: improve test coverage
• 1fff6b9 test: add an integration test comparing the lazy loading to complete runs
• 58e2044 chore: fix regression output
• b49ae0f chore: enforce known converted file
• 3e97a10 chore: log exception
• 8dfc89e perf(datasets): replace xarray with netCDF4 for metadata-only reads
• 10be217 chore: extract dates directly
• 445a7fc docs: add changelog entry for ecgtools removal
• 2056c2a refactor: use tqdm for parsing progress in catalog builder
• 0e6aa7f feat: add progress logging to catalog builder
• b3dae9e refactor: remove ecgtools dependency
• ef366f5 test: Fix failing test
• b425c42 chore: Move the check location
• 8a4c3d7 chore: actually run the tests...
• bca97bf chore: Remove checks
• 084ac07 fix(testing): raise clear error when dataset paths are missing
• 41102b2 chore(helm): add init container to wait for Dragonfly before starting Flower
• 32a45e8 fix: the fetch has to happen in the test-cases job to set the paths
• 52c54f0 chore: add more logging
• 5df80d7 chore: add workflow
• 41b14cc chore: don't show tqdm on downloads
• fdab615 chore: add esgf cache info
• 80b183c chore: retry checking if the flower instance is healthy
• 3cbf143 chore: remove previous fetches
• 4e415ed chore: wait for healthcheck
• 0e9c24a docs: add changelog for flower celeryconfig fix
• 5d1c98c feat(helm): add Flower celeryconfig and CI worker health check
• 7af141a doc: update docs for in...

v0.10.0

Announcements

This release add initial support for CMIP7 datasets.
This is still a work in progress so the list of the attributes that are tracked may change as we integrate the providers.

Changelog

Features

• Added database support for CMIP7 datasets based on the CMIP7 Global Attributes v1.0 specification. (#503)
• Added CMIP7 data requirements support, enabling providers to fetch CMIP6 data from ESGF and translate it to CMIP7 format using the CMIP7 CV converter. (#510)
• Added diagnostic summary introspection and auto-generated documentation for all providers. The ref providers show command now defaults to detailed list format and supports --columns for filtering table output. (#518)

Improvements

• Separated model from observation runs for regional historical diagnostics. (#460)
• Made listing the changed files from a regression test faster. (#514)
• Prepared the monorepo for splitting diagnostic provider packages into independent repositories. Extracted shared test fixtures into a climate-ref[test] pytest plugin, decoupled climate-ref-core from application-level types, and added API surface documentation, versioning strategy, provider compatibility CI, and a copier template for bootstrapping new provider repositories. (#520)

Bug Fixes

• Fixed CMEC bundle dimension validation to use a subset check instead of exact equality, allowing diagnostics with multiple data requirements to have varying output dimensions. (#523)

Trivial/Internal Changes

• #516

Changes

• b31c8a5 Bump version: 0.9.1 → 0.10.0
• 2ff4f58 chore: add changelog entry for PR #523
• 9b809ee fix: use subset check for CMEC bundle dimension validation
• ce56f34 fix(helm): use distinct chart version tags per build context
• 9a04820 chore: remove unnecessary conftest.py from climate-ref-core tests
• 38454c9 fix: address PR review comments
• 0ea9510 fix: improve diff coverage and restrict CI permissions
• a461179 fix(ci): use coverage run for proper entry-point plugin tracking
• 603dae7 fix(ci): skip conftest_plugin in imports-without-extras check
• d22d9d8 chore: add changelog entry for #520
• 9feddb4 ci: add provider compatibility workflow and copier template
• 491912a docs: add API surface, versioning strategy, and contributor guide
• 2121d1c refactor: extract shared test fixtures into climate-ref[test] pytest plugin
• 0953583 chore: add main_log_debug.txt to gitignore and remove existing files
• 57ee249 Apply suggestion from @lewisjared
• f6b876b docs: add changelog entry for PR #518
• 52fdcf4 feat(cli): make list format default for providers show and add --columns option
• 8145dee feat: improve diagnostic docs with admonitions, tabs, and overview by source type
• 7f087aa feat: add diagnostic summary introspection and documentation generation
• 8a758f6 test: improve coverage for CMIP7, solver, and test_cases CLI
• 887242e fix: resolve merge conflict in solver OR-logic after InvalidDiagnosticException removal
• ebd8a0f fix(tests): set up context manager mocks for xr.open_dataset in CMIP7 tests
• 04b7e3f Add changelog
• e81fdef Add codecov.yml with relaxed coverage thresholds
• 7aa967c feat(cmip7): add license_id and external_variables to CMIP7 model
• ce39225 Also update output collection of regional historical trend diagnostic
• f42fac2 Add changelog
• f942b93 Faster listing of regression data
• 9e334b9 Add another variable to test case
• a8fd54b Do not try to push to container registry from forks
• 2bb22bc Update regression test output
• fda40b4 Add changelog
• 3a2083b Split trends recipe and fix the other two regional recipes
• d787258 Update recipes
• c83a5c5 Update regional historical annual cycle and timeseries
• 97242bf fix: address PR review comments
• 32a9b7f test: increase test coverage for CMIP7 data requirements
• 5d09eba docs: add changelog for PR #510
• bcc1754 feat: add CMIP7 data requirements support for Example provider
• 8377a0a style: reorganize import statements for clarity
• 8e47b3e refactor: address PR review comments for CMIP7 support
• 624f40e Add tests for CMIP7 adapter with CMIP6-converted files
• d70e0a2 Add changelog for PR #503
• 0614379 Add CMIP7 database support

v0.9.1

Changelog

Features

• Added ingest_data() lifecycle hook to providers, enabling automatic dataset ingestion during ref providers setup. PMP climatology data is now ingested automatically, eliminating the need for a separate manual ingestion step. (#508)

Improvements

• Implemented coupled versioning for Helm chart: chart version, appVersion, and default image tag now stay in sync with the application version and are updated automatically by bump-my-version. (#507)
• Improved CLI performance by skipping database backup for read-only commands like config list and datasets list. (#511)
• Improved CLI startup time by deferring heavy imports until needed. (#512)

Bug Fixes

• Fixed Helm chart CI to use correct image tag override path (defaults.image.tag instead of invalid climate-ref.image.tag). (#507)

Improved Documentation

• Updated getting started documentation with clearer configuration and dataset download instructions. (#508)

Changes

• b06b4cc Bump version: 0.9.0 → 0.9.1
• c579e31 docs: fix admonition syntax and typo in getting-started guides
• 0d71679 fix(tests): update patch paths for lazy imports
• 6311473 chore: add changelog and fix lint after merge
• 4d500fd test: add unit tests for providers, datasets, config, and models
• ec2352f ci: only run helm jobs when helm/ changes or on main
• ab47504 perf: move SourceDatasetType to lightweight module and add more lazy imports
• 5100872 perf(cli): optimize startup time with lazy imports
• 5b546b0 chore: add changelog for PR #511
• bb10a39 feat(cli): skip database backup for read-only commands
• 758e672 fix: Use the right path
• 8df2043 docs: add changelog entries for PR #508
• c371c79 feat(providers): add ingest_data hook for provider-level dataset ingestion
• 4335793 docs: update configuration and dataset download instructions; remove empty tutorials section
• ba9be9d chore: add skip-validate
• 0086ade docs: add changelog entries for Helm versioning fixes
• a368352 chore: update Helm chart versioning and defaults to 0.9.0
• f2ed7dd chore: bump to latest image
• 7189d4a test(packaging): use the setup command

v0.9.0

Announcements

This release brings some upgrades to the testing framework we use and a streamlined setup process.
This should make it easier to add new diagnostics.

We have temporarily pinned pandas<3 until we can verify that it everything works as expected.

Changelog

Features

• Added test data management infrastructure for diagnostic development:

  • New ref test-cases CLI commands (fetch, list, run) for managing and running diagnostic test cases.
  • ESGF data fetching utilities with support for CMIP6 and obs4MIPs datasets.
  • TestDataSpecification and TestCase classes for defining reproducible test scenarios.

  (#475)

• Add functionality to translate a CMIP6 dataset to the new CMIP7 conventions (#484)

• Added test automation infrastructure for diagnostic testing using test-cases (#485)

• Added CMIP6 to CMIP7 format conversion command-line script to translate CMIP6 datasets into CMIP7-compatible format. (#489)

• Added RegistryRequest class for fetching datasets from pooch registries (pmp-climatology, obs4ref) instead of ESGF. (#490)

• Added Helm chart for Kubernetes deployment with automated CI/CD pipeline for building and publishing the chart to GitHub Container Registry, including deployment templates for provider workloads (ESMValTool, PMP, ILAMB), Flower monitoring UI, Dragonfly Redis dependency, and comprehensive integration testing in minikube. (#492)

• Add CI workflow to verify solve works without network access (ci-offline-solve.yaml).
  This test uses Docker with --network none to block all network access including
  subprocesses. Runs every other day and can be triggered manually. (#497)

• Added provider lifecycle hooks for offline execution setup. Providers can now implement setup_environment(), fetch_data(), and post_setup() methods to prepare for execution on HPC compute nodes without internet access. A new ref providers setup CLI command runs all provider setup hooks, fetching required reference data to the local cache before offline solving. (#498)

Improvements

• Clean up the open database connections in the test suite (#482)
• Improved catalog handling with hash-based change detection and multi-file dataset support. Enhanced CLI test-cases commands with new flags: --only-missing, --force, --dry-run, --if-changed, and --clean. (#490)
• Updated ESMValTool to v2.13.0 (#500)

Trivial/Internal Changes

• #480, #506

Changes

• 2eb538f Bump version: 0.8.1 → 0.9.0
• c435cb4 docs: Changelog
• f8caea7 chore: pin pandas to <3
• 6d3d884 chore(deps-dev): bump the python-dependencies group across 1 directory with 9 updates
• 6e98652 chore: add more test coverage
• eb655a5 Add changelog
• db4ca16 Regenerate example output
• d7ed855 Update ESMValTool to v2.13.0
• b8f7ce6 refactor: address PR review feedback for provider lifecycle
• 2368a2d Update docs/how-to-guides/hpc_executor.md
• 5dcead9 refactor: address PR review feedback for provider lifecycle hooks
• 690f759 docs: update provider setup documentation
• 439a85d revert: remove blocking
• a6c47ae docs: add changelog for provider lifecycle hooks
• 0892d14 fix(config): handle network errors when fetching ignore datasets file
• 665ccd6 feat(providers): add lifecycle hooks for offline execution setup
• c4323d4 revert: makefile
• e477331 test: add Docker-based offline solve test workflow
• 524ea64 chore: don't download all data
• 089de09 fix: add conda environment setup before offline tests
• 2c1a436 fix: fetch test data before running offline solve tests
• 28f1070 test: add CI job to verify solve works without network access
• b0dc7e2 chore: ignore all .ref
• 850c823 chore(deps): bump the github-actions group with 4 updates
• 0d7e81b docs: add documentation for the helm chart
• 62e7768 fix: Set HOME dir to a writable location
• c9aedb0 revert: undo livliness checks for now
• 34b2e16 feat: add liviness probe
• 950b17f chore(deps-dev): bump the python-dependencies group with 3 updates
• 2002e40 chore: Re-enable CI/CD workflows
• 51b163e feat: Build and publish Helm chart in CI
• fd730d3 chore: Add pragma: no cover for display-only CLI code
• 7322e25 refactor: Extract utilities from test_cases.py and add tests
• f878721 docs: Add changelog entries for PR #490
• 149e302 feat: Add RegistryRequest for pooch registries and improve catalog handling
• 724be90 docs: Changelog
• b3d0135 feat: added script to translate CMIP6 to CMIP7
• 22d1eab fix: Updates to understanding from Zeb
• 06a119e chore(deps-dev): bump the python-dependencies group with 4 updates
• d407a89 chore: remove the unneeded files
• 018b53c test: Fix udpates to paths
• a8c5f44 chore: Updates after rereading the global attributes file
• 69b8ec1 chore: use parameterised tests instead
• cb246c0 test: improve coverage
• 0beb3fa Update packages/climate-ref-core/src/climate_ref_core/cmip6_to_cmip7.py
• bb8fefc doc: Changelog
• b2bae88 chore: run test-cases separately
• a4ee861 chore: ignore pytest warning
• bd408dc chore: add example regression output
• 58fd3c9 feat: Add a helper class for tracking test case paths
• 3ea8367 test: offload current testing to the validate cmec bundles routine
• 683fdee chore: run the example diagnostic
• a75e074 chore: add example catalog
• 7ac134a chore: move test case tests to their respective packages
• ce706b4 chore: move catalog to a provider-specific path
• a5e2659 chore: remove doctest as the datafile isn't available
• 83147da docs: Changelog
• daf3c8a feat: Add cmip6 to cmip7 translation
• 7227336 fix: Allow fetching multiple variables
• 5a8b72e fix: typos in docs
• 2e4a7d3 fix: remove unused function
• 236c0b3 chore: improve coverage
• 281c2aa chore: clean up tests
• a27870f feat: Generate a data catalog on fetch
• 6be929d chore: Move ESGF_DATA_DIR definition to testing
• 1f0e89b Update packages/climate-ref/tests/unit/datasets/test_cmip6.py
• 779f974 chore: Fix more warnings
• 51c9b49 chore: allow to fetch and run
• 39d0ef7 chore: Clean up db in the cli
• d9dac70 chore: use pytest 9's strict mode
• e84660c test: Fix old testing value
• 20aeda6 docs: Changelog
• edb2965 chore: Fix sqlite resource errors
• bb2f52a refactor: rename commands
• 7777f24 test: Add more coverage
• 6b850b1 test: test testing tests
• 3b39fab docs: Add documentationf for the new diagnostics
• 77202fc test: improve coverage
• f365d06 doc: Changelog
• 27dde70 revert: Go back to separating stderr and stdout
• 57ea206 chore: pin xarray for ilamb3 due to issue
• 3dc9ca1 Potential fix for pull request finding 'Explicit returns mixed with implicit (fall through) returns'
• 6f70658 fix: handle a regression in xarray
• 84ec4fe fix: Clean up some type hints from updating mypy
• 5326654 docs: fix typo
• e3b2541 feat: Add a harness to run a set of specs
• aafd16d chore(deps-dev): bump the python-dependencies group with 31 updates
• 3b6d9e1 fix: use uv not pip
• 4084648 docs: Changelog
• b7384fb ci: Ignore dependabot PRs
• a1e0121 chore(deps): bump the github-actions group with 5 updates
• bccb710 chore: add ipython dep to fix CI
• 35f8a05 test: Add coverage for the new modules
• d189868 feat: add test spec
• 1a3894c feat: Add esgf fetching utlities from sample repo
• 50ca6bc chore: add required packages