v0.2.0

Announcements

This project is being renamed from CMIP-REF to Climate-REF to better reflect the goal for wider use within the climate modelling community.

This release also brings an initial prototype of asynchronously running metrics with the new Celery executor.
Documentation and tutorials for its usage will be developed during the upcoming Hackathon.

Changelog

Breaking Changes

• Refactor cmip_ref.env module to cmip_ref_core.env (#60)

• Removed cmip_ref.executor.ExecutorManager in preference to loading an executor using a fully qualified package name.

  This allows the user to specify a custom executor as configuration
  without needing any change to the REF codebase. (#77)

• Renamed the $.paths.tmp in the configuration to $.paths.scratch to better reflect its purpose.
  This requires a change to the configuration file if you have a custom configuration. (#89)

• The REF now uses absolute paths throughout the application.

  This removes the need for a config.paths.data directory and the config.paths.allow_out_of_tree_datasets configuration option.
  This will enable more flexibility about where input datasets are ingested from.
  Using absolute paths everywhere does add a requirement that datasets are available via the same paths for all nodes/container that may run the REF. (#100)

• An [Executor][cmip_ref_core.executor.Executor] now supports only the asynchronous processing of tasks.
  A result is now not returned from the run_metric method,
  but instead optionally updated in the database.

  The run_metric method also now requires a provider argument to be passed in. (#104)

Features

• Adds a cmip-ref-celery package to the REF that provides a Celery task queue.

  Celery is a distributed task queue that allows you to run tasks asynchronously.
  This package will be used as a test bed for running the REF in a distributed environment,
  as it can be deployed locally using docker containers. (#60)

• Add metric_providers and executor sections to the configuration which loads the metric provider and executor using a fully qualified package name. (#77)

• Implemented Pydantic data models to validate and serialize CMEC metric and output bundles. (#84)

• Add the cmip_ref_celery CLI commands to the ref CLI tool.
  These commands should be available when the cmip_ref_celery package is installed.
  The commands should be available in the ref CLI tool as ref celery .... (#86)

• Add fetch-sample-data to the CLI under the datasets command.

  ref datasets fetch-sample-data --version v0.3.0 --force-cleanup
  ``` ([#96](https://github.com/CMIP-REF/cmip-ref/pulls/96))

• Add a Celery-based executor
  to enable asynchronous processing of tasks. (#104)

• Add ref executions list and ref executions inspect CLI commands for interacting with metric executions. (#108)

Improvements

• Move ILAMB/IOMB reference data initialization to a registry-dependent script. (#83)

• ILAMB gpp metrics added with full html output and plots. (#88)

• Saner error messages for configuration errors (#89)

• Centralised the declaration of environment variable overrides of configuration values.

  Renamed the REF_OUTPUT_ROOT environment variable to REF_RESULTS_ROOT to better reflect its purpose.
  It was previously unused. (#92)

• Sample data is now copied to the test/test-data/sample-data instead of symlinked.

  This makes it easier to use the sample data with remote executors as the data is now self-contained
  without any links to other parts of the file system. (#96)

• Integrated the pycmec validation models into ref core and metric packages (#99)

• Added ILAMB relationship analysis to the current metrics and flexibility to define new metrics in ILAMB via a yaml file. (#101)

• Sped up the test suite execution (#103)

Improved Documentation

• Added an excerpt from the architecture design document (#87)
• Adds a roadmap to the documentation (#98)

Trivial/Internal Changes

• #97, #102, #116, #118

Changes

• 73877d0 Bump version: 0.1.6 → 0.2.0
• bb26cab docs: Changelog
• d8e2bea chore: Update sample data to v0.4.1
• f31eec5 chore: Update sample data to v0.3.3
• 009b1d2 test: Improve coverage
• 45842df test: Cleanup test
• b6d1540 test: add executions to the list of expected commands
• ad456bb docs: Changelog
• 7f05298 chore: Add some tests
• 2b7b1cd feat: Add datasets panel
• d1d8938 feat: Add inspect entry
• abcbd5f chore: Add execution list cli
• efb12ba chore: Correctly clean up celery results
• cbf7907 chore: Fix last tests
• 2e82758 chore: Cleanup tests
• 4ab78f5 chore: Remove copy
• 4c9e08c made the ILAMB DataCollection relevant again
• 59a5662 return failed metric if no comparison dataset is generated
• 11bd046 rephrase fail test
• 440044b Update packages/ref-celery/src/cmip_ref_celery/executor.py
• 1924e0a Update packages/ref-celery/src/cmip_ref_celery/executor.py
• 3a3e866 chore: Fix tests broken by the merge
• c70b57e fix a typo in metrics.py
• 52ce942 skip incompletes
• 8a8de0b skip coverage on a block just for logging
• 8dd2b6b skip coverage on a block just for logging
• 57b2409 added tests for missing coverage
• cd3da00 dependency updates
• fbbc7a0 fix dataframe grouping logic for ilamb test with relationships
• 9d18afe added gridded ilamb test
• db49ad5 changelog
• aa1dab7 add relationship analysis and flexibility to define new metrics via a yaml file
• 49a24d1 docs: Changelog
• 991580f chore: Also set results backend
• 7df08cb chore: Check port
• 41df9a7 test: Increase coverage of executor
• 0d252f4 test: Increase coverage of executor
• 1d294b1 chore: Fix indent
• efc3cc5 test: Wrap adding metric execution results in a transaction
• 18b07eb test: Fix tests
• 48fbb2f test: Increase coverage
• 239bfae test: Add an integration test
• 941a130 feat: Add a REF_METRIC_PROVIDERS environment variable
• 10e3af8 chore: Cleanup from refactoring
• ae1e4e6 feat: Add a celery executor
• 7df64b0 chore: Loosen unique constraints on MetricExecutionResult
• 0e627cc fix an error when building the how-to-guides
• 0bf967e add changelog for pycmec integration
• a14938f modify metric packages for pycmec validation and the changes of build_from_output_bundle
• 9183a77 remove the type guard for json_file in pycmec
• 0a0f695 fix: missing rename of the make target
• e107ff8 docs: Changelog
• 32a5380 feat: Added a test-quick make target
• b553cff chore: Ignore pkg_resources error from intake_esm
• 3d65a97 chore: Speed up tests and clean up some errors
• b43f6d2 update catalog test files
• 6161eff update collection/dataset hashes
• 2638cac bump sample data version
• 3a73b9d keep the orig. MetricResult build method with one more dict for the CMEC metric bundle
• 52dc830 chore: Changelog
• 6c20eed chore: Create directory
• 1e02b1b test: make ESMVal output directory
• e13c709 test: Update tests
• 25476be fix: Use absolute paths everywhere
• 198ddfc initial pycmec integration in ref
• ae89101 chore: Roll back adding some of the obs4mips data due to errors
• 954df96 docs: Fixes to roadmap
• 788e241 docs: Changelog
• 3fa917d fix: Fix up some doc issues
• 8183cdf feat: Add a roadmap
• dc7da02 chore: Update rtd's ingest path
• f839721 docs: Changelog
• 54ee7d0 fix: Update makefile name
• 1e49fef chore: Update regression files
• 5a79957 chore: Bump sample data to v0.3.1
• ac8b88f fix: Use CMIP6 paths explicitly in tests
• cce81f1 test: Fix the assertion call attempt 2
• 9158f38 test: Fix the assertion call
• c55c9ca docs: Changelog
• b4b63c4 fix: Copy sample files rather than symlinking them
• d505d9d docs: update docs
• 03cde11 chore: Changelog
• 4765bd8 chore: Use correct exception
• d387037 feat: Enable environment variables to override configuration values
• 4d95fbd add a simple usage in the cmec docstring
• abc6c2c fix a error in cmec doc generation
• 700ffc1 add more inline comments in methods of CMEC classes
• eb48a19 move the cmec sample json data to cmec_testdata
• 0e986ac use pytest data_regression in cmec tests
• 408f380 add pydantic to the ref-core
• d522882 docs: Fix crossref
• f820826 fix: Support different log message in python3.12+
• 660b7e3 relax type constraints of CMC output class
• 266161d update the cmec output sample json using PMP output
• 6832514 chore: Better handle configuration errors
• b3e2fcf change the index as a optional keyword
• ddc47ed change validation methods to private methods
• 669df6b allow other data in the top CMECMetric dict and optional 'attributes' in each level of RESULTS tree
• a6e3058 more tests, methods...

v0.1.6

Announcements

This release adds packages for the key metrics providers that the REF will support for CMIP7 FastTrack.

• ESMValTool
• ILAMB3
• PMP/PCMDI

These integrations are being actively worked on and new metrics will be added over time.

Changelog

Features

• Added Equilibrium Climate Sensitivity (ECS) to the ESMValTool metrics package. (#51)
• Added Transient Climate Response (TCS) to the ESMValTool metrics package. (#62)
• Added the possibility to request datasets with complete and overlapping timeranges. (#64)
• Added a constraint for selecting supplementary variables, e.g. cell measures or
  ancillary variables. (#65)
• Added a sample metric to the ilamb metrics package. (#66)
• Added a sample metric to the PMP metrics package. (#72)
• • Added the standard ILAMB bias analysis as a metric. (#74)

Bug Fixes

• • Added overlooked code to fully integrate ilamb into ref. (#73)
• Correct the expected configuration name to ref.toml as per the documentation. (#82)

Improved Documentation

• Update the package name in the changelog.

  This will simplify the release process by fixing the extraction of changelog entries. (#61)

Trivial/Internal Changes

• #68

Changes

• fa788ac Bump version: 0.1.5 → 0.1.6
• 0526418 docs: Changelog
• e5605b2 fix: Make config name consistent
• 4279599 ignore coverage of some safety lines
• 3d3997e changelog
• c675aa1 successful testing of standard ilamb analysis, json not verified
• d4fa20d added bundling
• d574e63 implement phase 2
• 9a9a46f update
• 56a3827 Add constraint for adding cell measures and ancillary variables (#65)
• 68b4037 Update packages/ref-metrics-pmp/tests/unit/test_metrics.py
• 918d95a pre-commit fix
• af24daf adjust data requirement
• 1576ae4 update uv.loc
• 471845d phase of standard ilamb analysis
• dbb3bdd add change log
• ad6f408 Update packages/ref-metrics-pmp/tests/unit/test_metrics.py
• d78093e Update packages/ref-metrics-pmp/pyproject.toml
• 2817d10 unique slug
• c492cc0 changelog
• e7797a6 missing integration stuff
• 6ea4833 chore: update the number of timesteps
• 63eeda8 chore: Make test indifferent to the providers
• c4ba29b update
• 2edf0e3 update
• 0fca644 follow ilamb example
• f699a7c add pmp
• d32e1c5 update
• 3aae785 initial commit
• 065c88b Add temporal constraints (#64)
• 4520795 test: Update tests
• f6b470f docs: Add documentation for the steps to regenerate the sample data
• 260bc41 case changes
• 9aa219a ilamb3 to ilamb
• 8e7dfc1 test: Make the test suite more resilient to changes in the sample data
• c0420ea chore: Update sample data to v0.3.0
• d719cf9 missing dep
• a291dd6 Add ESMValTool Transient Climate Response recipe (#62)
• f5b8ac8 added changelog
• 4a53866 ilamb3 implementation of the example metric
• 6dd6995 Add ESMValTool Equilibrium Climate Sensitivity recipe (#51)
• 0b0885f docs: Update release docs
• 0523b7a docs: Changelog
• 889359f fix: Correct the changelog to use the _ separated name

v0.1.5

Changelog

Trivial/Internal Changes

• #56

Changes

• 7b64e2d Bump version: 0.1.4 → 0.1.5
• 227d7cf chore: Changelog
• b20c5a6 chore: Include directory to place the outputs
• e71847d chore: Fix deploy action
• 875d83d fix: Attempt a fix on the deploy action

v0.1.4

Announcements

This is the first release of the cmip_ref package.
The project is still a work in progress so there is more to come!

Changelog

Breaking Changes

• Adds an ingest CLI command to ingest a new set of data into the database.

  This breaks a previous migration as alembic's render_as_batch attribute should have been set
  to support targeting sqlite. (#14)

• Renames ref ingest to ref datasets ingest (#30)

• Prepend package names with cmip_ to avoid conflicting with an existing PyPI package.

  This is a breaking change because it changes the package name and all imports.
  All package names will now begin with cmip_ref. (#53)

Features

• Migrate to use UV workspaces to support multiple packages in the same repository.
  Adds a ref-metrics-example package that will be used to demonstrate the integration of a metric
  package into the REF. (#2)

• Adds the placeholder concept of Executor's which are responsible for running metrics
  in different environments. (#4)

• Adds the concept of MetricProvider's and Metrics to the core.
  These represent the functionality that metric providers must implement in order to be part of the REF.
  The implementation is still a work in progress and will be expanding in follow-up PRs. (#5)

• Add a collection of ESGF data that is required for test suite.

  Package developers should run make fetch-test-data to download the required data for the test suite. (#6)

• Adds the ref package with a basic CLI interface that will allow for users to interact with the database of jobs. (#8)

• Add SqlAlchemy as an ORM for the database alongside alembic for managing database migrations. (#11)

• Added a DataRequirement class to declare the requirements for a metric.

  This provides the ability to:

  • filter a data catalog
  • group datasets together to be used in a metric calculation
  • declare constraints on the data that is required for a metric calculation

  (#15)

• Add a placeholder iterative metric solving scheme (#16)

• Extract a data catalog from the database to list the currently ingested datasets (#24)

• Translated selected groups of datasets into MetricDatasets.
  Each MetricDataset contains all of the dataset's needed for a given execution of a metric.

  Added a slug to the MetricDataset to uniquely identify the execution
  and make it easier to identify the execution in the logs. (#29)

• Adds ref datasets list command to list ingested datasets (#30)

• Add database structures to represent a metric execution.
  We record previous executions of a metric to minimise the number of times we need to run metrics. (#31)

• Added option to skip any datasets that fail validation and to specify the number of cores to
  use when ingesting datasets.
  This behaviour can be opted in using the --skip-invalid and --n-jobs options respectively. (#36)

• Track datasets that were used for different metric executions (#39)

• Added an example ESMValTool metric. (#40)

• Support the option for different assumptions about the root paths between executors and the ref CLI.

  Where possible path fragments are stored in the database instead of complete paths.
  This allows the ability to move the data folders without needing to update the database. (#46)

Improvements

• Add a bump, release and deploy flow for automating the release procedures (#20)

• Migrate test data into standalone CMIP-REF/ref-sample-data repository.

  The sample data will be downloaded by the test suite automatically into tests/test-data/sample-data,
  or manually by running make fetch-test-data. (#49)

Bug Fixes

• Adds version field to the instance_id field for CMIP6 datasets (#35)
• Handle missing branch times.
  Fixes #38. (#42)
• Move alembic configuration and migrations to cmip_ref package so that they can be included in the distribution. (#54)

Improved Documentation

• Deployed documentation to https://cmip-ref.readthedocs.io/en/latest/ (#16)

• General documentation cleanup.

  Added notebook describing the process of executing a notebook locally (#19)

• Add Apache licence to the codebase (#21)

• Improved developer documentation. (#47)

Trivial/Internal Changes

• #41, #44, #48, #52

Changes

• 8590fdf chore: Update release process
• e82eba8 Bump version: 0.1.2 → 0.1.3
• 124013f Bump version: 0.1.1 → 0.1.2
• 6b5031e chore: Update changelog
• ca5ce02 test: Remove unused tests
• eafabc0 chore: Update lockfile as part of the precommit step
• 99e0369 Bump version: 0.1.0 → 0.1.1
• 3ef7072 chore: Fix towncrier configuration
• 7d6a086 chore: Ignore migration files
• 70cf609 chore: Changelog
• 9ac1c65 fix: Move alembic migrations into the the cmip_ref package
• 08db461 chore: Add a quick check that the package is installable
• cd72e3f chore: Fix command
• 8ce2344 chore: Verify that packages are installable
• 548ff2c docs: Changelog
• 4a976ed chore: Fix imports
• 5967b0c fix: Incorrectly migrated strings
• 6dfd93c chore: Verify builds
• cc5fb7a refactor: Rename packages
• a5c4864 refactor: Prepend packages with cmip_
• fc46e38 fix: Remove unneeded CI tasks
• 08a9bb4 chore: Changelog
• 8515dbf chore: Always run mypy check
• 8e8f553 chore: Changelog
• eaec985 chore: Update pre-commit version
• 310fae5 feat: Add mypy pre-commit hook
• 454fbd9 chore: Fix coverage
• 12c7248 chore: Changelog
• df475d5 test: Fix integration test
• 324a23b chore: Update stubs for pooch
• 1590ebe chore: Update tests again
• 6899cc2 chore: Update documentation
• 3f977f7 chore: Update test-data
• 7523159 docs: Update sample data commands
• fd3b319 chore: Fetch sample data automatically
• f9d57a4 Run all tests
• 6f9e039 Add changelog entry
• 915dca1 Use GitHub term "pull request" instead of "merge request"
• c2ce544 Add a few more updates
• 4c05fda test: Update tests
• 0cd3cab chore: remove esgpull from CI
• 53530f7 chore: remove esgpull
• b813dbe chore: Don't use esgf data anymore
• 1f7ef11 feat: Fetch data from https://github.com/CMIP-REF/ref-sample-data
• fb7aab2 refactor: Move executor definition to the ref package
• 9f7a917 test: Improve coverage
• 036a961 chore: Move pytest-mock as dependency of root package
• 5504eda refactor: Rename output_filename to to_output_path
• c21e4ff refactor: Rename output_bundle to output_fragment
• 24c5585 Update developer docs
• 35c4afe Add ESMValTool example metric (#40)
• 478479e docs: Changelog
• 21d4b6b refactor: Use a build method for failed executions
• 0f08335 feat: Use a executor specific output directory
• 2459349 docs: Changelog
• 103c77c fix: Update codecov version
• 508c1f4 docs: Changelog
• dcb26f5 fix: Handle none values
• 5fa348c test: Fix integration test
• e68e36c chore: Clarify that stdout is expected
• e4faf07 chore: Split stderr from stdout
• c246553 docs: Changelog
• 62f8f87 refactor: Add a fixture for running and asserting the results from the CLI
• c3ea68b docs: Add message about importance of the index
• ea99083 test: Add test to verify that the right dataset is used
• 6bad36e chore: Undo some code I didn't mean to include
• 208f857 feat: Register datasets/execution links into the database
• b36e493 chore: Fix mypy
• 0f9d7d4 docs: Changelog
• 78127a9 test: Add tests for failing datetimes
• 69f4627 feat: Fixes from ingesting more datasets
• b0f439c chore: Changelog
• 5c4bc08 fix: Update tests
• 5b55427 fix: Add version to the instance_id
• 16db7fb chore: Force LLNL index
• d14e486 docs: Changelog
• 43c5b93 test: Fix integration test
• bf3b1cb test: Fix integration test
• f172d54 chore: Add log messages
• 5047b81 chore: Flatten migrations
• 3408154 fix: mypy
• 1f9577e fix: Added a nested session to handle registering providers
• 0b1a3df chore: Fix flakey test
• 05cdfb1 chore: Fix flakey test
• 2ba6500 chore: Fix flakey test
• 504ec9c test: Add tests for new CLI commands
• 136c51f chore: Changelog
• c1c6ac3 docs: Add doc for ingesting datasets
• 53f8d05 refactor: Move ingest to datasets ingest
• 2245d36 feat: First cli run
• d5c1ee6 refactor: Rename slug key
• 1b1c5b4 feat: Clean up the tables and relationships
• 043567c chore: Add migration
• 31c68a9 fix: Increase coverage
• 98c0020 chore: Fix d...