From 763eb0f3cb494237baad636d6fd02c6c80e5d47f Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Wed, 8 Jul 2026 11:48:35 +0200 Subject: [PATCH 01/22] Add a note on community code contributions --- docs/contributing/index.md | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/contributing/index.md b/docs/contributing/index.md index ecefbf50..fcd7ef38 100644 --- a/docs/contributing/index.md +++ b/docs/contributing/index.md @@ -23,6 +23,13 @@ When you open a bug report, please include: - a description of the encountered behavior, - and any additional information that you think may be relevant, such as which environment you encountered it in, or when you first encountered the bug. +### Documentation + +For minor fixes such as correcting a typo or a broken link, it's fine to make the changes and submit them through a pull request. +For larger changes, please make sure you are assigned to an issue first as is described +in the "[Code](#code)" section of this page. Then, visit the "[Documentation](documentation.md)" +page to learn how to contribute documentation changes. + ### Code If you want to make code contributions, please first make sure there is an open issue @@ -33,14 +40,18 @@ everything is clear, it's not out of date, and "officially" assign you to the is that others know its being worked on. Making sure there is a clear description and clear assignment helps prevent a situation where someone makes a large contribution that is unwanted, or is simultaneously developed by someone else. With an issue assigned, -please head over to the "[Development](contributing.md)" section. +please head over to the "[Setting up the development environment](setup.md)" section. -### Documentation +!!! info "Help Wanted?" + + We learned that working with community contributions during the development of + the first feature-complete release of the REST API is complicated. + While we are entirely grateful that so many people take an interest and want to help, + the majority of the work is figuring out what the desired behavior and interface of the + REST API should be. Making these decisions requires a lot of context about the OpenML platform, + ecosystem, and its history. For this reason, we restrict community contributions in this + project to issues that are marked "Good First Issue" or "Help Wanted" for now. -For minor fixes, it's fine to make the changes and submit them through a pull request. -For larger changes, please make sure you are assigned to an issue first as is described -in the "[Code](#code)" section of this page. Then, visit the "[Documentation](documentation.md)" -page to learn how to contribute documentation changes. ### Other From 3684075c5751ca7701302076af0f1cece1ce23c8 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Wed, 8 Jul 2026 13:00:07 +0200 Subject: [PATCH 02/22] Switch to Zensical. Restructure the landing page and roadmap. --- docker/docs/Dockerfile | 5 ++--- docs/index.md | 40 ++++++++++++++-------------------------- docs/roadmap.md | 28 ++++++++++++++++++++++++++++ mkdocs.yml | 21 ++++++++++++--------- pyproject.toml | 3 +-- 5 files changed, 57 insertions(+), 40 deletions(-) create mode 100644 docs/roadmap.md diff --git a/docker/docs/Dockerfile b/docker/docs/Dockerfile index ae6d376e..a4363ab6 100644 --- a/docker/docs/Dockerfile +++ b/docker/docs/Dockerfile @@ -1,4 +1,3 @@ -FROM squidfunk/mkdocs-material -RUN python -m pip install mkdocs-section-index -ENTRYPOINT ["mkdocs"] +FROM zensical/zensical +ENTRYPOINT ["zensical"] CMD ["serve", "--dev-addr=0.0.0.0:8000"] diff --git a/docs/index.md b/docs/index.md index 68a6631d..e3ccc19b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,6 +4,20 @@ This is the Python-based OpenML REST API server. It's a rewrite of our [old backend](http://github.com/openml/openml) built with a modern Python-based stack. +### What's changed? +For an overview about how the REST API is changing, visit the +[migration guide](migration.md). + + +### Why a new REST API? +The current production REST API has been with us for over a decade, and is still trotting along. +However, it's built on a number of technologies which fell out of favor and some of its design decisions do not match current-day expectations. +The current REST API would need significant refactoring to bring it up-to-date. + +Instead, we chose to reimplement the REST API in Python, since most of our core developers are more familiar with it than PHP, it has good packages and tooling for modern software development, and it's more familiar to the ML community. +We hope that a Python-based REST API will be easier to understand, maintain, and extend by future contributors. + + !!! question "Looking to access data on OpenML?" @@ -16,29 +30,3 @@ modern Python-based stack. If you are looking to interface directly with the REST API, and are looking for documentation on the REST API endpoints, visit the [APIs](https://openml.org/apis) page. - - This documentation is for developing and hosting your own OpenML REST API. - -## Development Roadmap - -First we will mimic current server functionality, relying on many implementation details -present in the current production server. We will implement all endpoints using the SQL -text queries based on PHP implementation, which should give near-identical responses to -the current JSON endpoints. Minor exceptions are permitted but will be documented. - -At the same time we may also provide a work-in-progress "new" endpoint, but there won't -be official support for it at this stage. After we verify the output of the endpoints -are identical (minus any intentional documented differences), we will officially release -the new API. The old API will remain available. After that, we can start working on a -new version of the JSON API which is more standardized, leverages typing, and so on: - - - Clean up the database: standardize value formats where possible (e.g., (un)quoting - contributor names in the dataset's contributor field), and add database level - constraints on new values. - - Redesign what the new API responses should look like and implement them, - API will be available to the public as it is developed. - - Refactor code-base to use ORM (using `SQLAlchemy`, `SQLModel`, or similar). - - Officially release the modernized API. - -There is no planned sunset date for the old API. This will depend on the progress with -the new API as well as the usage numbers of the old API. diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 00000000..5bb95e71 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,28 @@ +The REST API is currently under development. +It's does not yet have feature parity with the PHP-based REST API. +We are currently working on Phase 1 and Phase 2 in parallel, and to deploy to production before the end of 2026. +The sunset date of the PHP-based REST API will be depend on the progress of this reimplementation and that of some other required server services. + +## Phase 1: Achieve Feature Parity + +The first goal is _rough_ feature parity. +This means that we want to expose the same endpoints with the same general responses. +We may deviate in case the original implementation had an oversight or a bug, and fix those along the way. +We can be pragmatic about this, for example, copying over a complex query from the PHP API. +This phase is considered complete when the output of the new REST API can be mapped to output of the old REST API for all endpoints the PHP-based API has and the [migration guide](migration.md) is done. + + +## Phase 2: Modernizing the API + +We are also working on incorporating modern standards like [RFC9457](https://www.rfc-editor.org/rfc/rfc9457.html) for error reporting, better use of JSON types, including authentication in the request's HTTP header instead of the URL, or changing HTTP methods or paths to better reflect the behavior of the endpoint. + +## Phase 3: Revisiting the Data(base) + +After the PHP-based API has sunset, the Python-based API will be the only one connecting to the database. +This means we can start cleaning the database data and schema more freely, such as standardizing value formats where possible (e.g., (un)quoting contributor names in the dataset's contributor field), use more appropriate data types, drop deprecated tables or columns, and more. + +We can also start supporting different file formats for e.g., task splits or prediction files. + +## Phase 4: New Features + +Richer metadata for assets on OpenML (e.g., structured citation data), collaboration features, OAuth support, and support for other data modalities are just some of the features we want to work on in 2027 and beyond. diff --git a/mkdocs.yml b/mkdocs.yml index 1acd2313..17eafa8e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -17,18 +17,21 @@ theme: nav: - OpenML Server: - index.md - - Getting Started: installation.md - - Changes: migration.md - - Database: - - database/index.md - - openml: database/openml.md - - openml_expdb: database/openml_expdb.md - - Contributing: - - contributing/index.md + - Migration Guide: migration.md + - Roadmap: roadmap.md + - Deployment: + - installation.md + - Development: - Setup: contributing/setup.md - Tests: contributing/tests.md - - Documentation: contributing/documentation.md - Project Overview: contributing/project_overview.md + - Database: + - database/index.md + - openml: database/openml.md + - openml_expdb: database/openml_expdb.md + - Contributing: + - contributing/index.md + - Documentation: contributing/documentation.md markdown_extensions: - attr_list diff --git a/pyproject.toml b/pyproject.toml index e1b7e609..1ddbcc0e 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -40,8 +40,7 @@ dev = [ "pytest-xdist", ] docs = [ - "mkdocs-material", - "mkdocs-section-index", + "zensical", ] [project.urls] From 41df332e597a813c7d20957ad71efb637471b94d Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Wed, 8 Jul 2026 13:55:31 +0200 Subject: [PATCH 03/22] Move Elastic Search notice Normal deployment of the REST API is unaffected by ES. It is only when developers want to use it for migration tests that it matters. --- docs/contributing/setup.md | 5 +++++ docs/installation.md | 17 ++++------------- 2 files changed, 9 insertions(+), 13 deletions(-) diff --git a/docs/contributing/setup.md b/docs/contributing/setup.md index 63433b15..977a6b2b 100644 --- a/docs/contributing/setup.md +++ b/docs/contributing/setup.md @@ -54,6 +54,11 @@ This will spin up 5 services, as defined in the `compose.yaml` file: the REST API documentation. Changes to the code in `src/` will be reflected in this container. +!!! note + + The PHP REST API needs Elasticsearch. In some cases, it also needs the ES indices to be built. + The current set up does not automatically build ES indices, because that takes a long time. + Exposing ports to the host network isn't needed for development, but may be useful to inspect responses directly from the host machine. !!! note diff --git a/docs/installation.md b/docs/installation.md index e0a7c449..351bedcd 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,25 +1,16 @@ # Installation -See also ["Contributing"](contributing/contributing.md). - -The primary way to run this service is through a Docker container. +We currently only support running the REST API through a Docker container. The REST API needs to be able to connect to a MySQL database with the OpenML "openml" and "openml_expdb" databases. The `compose.yaml` file of this project defines these together out of the box. This is useful for development purposes, but the database does not persist between restarts in the current configuration. By default, the current code is also mounted into the Python REST API container (again, for development purposes). -For development, it should suffice to run the services from a fresh clone by running `docker compose up python-api -d`. +It should suffice to run the services from a fresh clone by running `docker compose up python-api -d`. If you want to make sure to bind the exposed container ports to the host machine then you will need to use the `compose.ports.yaml` file too (`docker compose -f compose.yaml -f compose.ports.yaml up python-api -d`). -The REST API will then be exposed on port 8001 on the host machine. To visit the Swagger Docs, visit http://localhost:8001/docs. - -Once the containers are started, you can run tests with `docker compose exec python-api python -m pytest -m "not php_api" tests`. -For migration testing, which compares output of the Python-based REST API with the old PHP-based one, also start the PHP server (`docker compose --profile "apis" up -d`) and include tests with the `php_api` marker/fixture: `docker compose exec python-api python -m pytest tests`. - -!!! note +The REST API will then be exposed on port 8001 on the host machine. +To visit the Swagger Docs which show REST API endpoint documentation, visit http://localhost:8001/docs. - The PHP REST API needs Elasticsearch. In some cases, it also needs the ES indices to be built. - The current set up does not automatically build ES indices, because that takes a long time. - When we start testing more upload functionality, for which the PHP API needs built indices, we'll work on an ES image with prebuilt indices. Information for a production deployment will follow, in a nutshell you need to configure the REST API to connect to a persistent database, which can be the one defined in `compose.yaml` if it has an appropriately mounted volume. From 5491f594d13b64eef5e1ef89a0187871384d58b1 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Wed, 8 Jul 2026 16:47:58 +0200 Subject: [PATCH 04/22] Add code of conduct --- docs/contributing/code_of_conduct.md | 97 ++++++++++++++++++++++++++++ mkdocs.yml | 3 +- 2 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 docs/contributing/code_of_conduct.md diff --git a/docs/contributing/code_of_conduct.md b/docs/contributing/code_of_conduct.md new file mode 100644 index 00000000..6e333572 --- /dev/null +++ b/docs/contributing/code_of_conduct.md @@ -0,0 +1,97 @@ +# Contributor Covenant Code of Conduct + +This is a temporary Code of Conduct until OpenML establishes one for all +repositories in its organisation. + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our +community a harassment-free experience for everyone, regardless of age, body +size, visible or invisible disability, ethnicity, sex characteristics, gender +identity and expression, level of experience, education, socio-economic status, +nationality, personal appearance, race, religion, or sexual identity +and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, +diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our +community include: + +* Demonstrating empathy and kindness toward other people +* Being respectful of differing opinions, viewpoints, and experiences +* Giving and gracefully accepting constructive feedback +* Accepting responsibility and apologizing to those affected by our mistakes, + and learning from the experience +* Focusing on what is best not just for us as individuals, but for the + overall community + +Examples of unacceptable behavior include: + +* The use of sexualized language or imagery, and sexual attention or + advances of any kind +* Trolling, insulting or derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or email + address, without their explicit permission +* Other conduct which could reasonably be considered inappropriate in a + professional setting + +Moreover, we expect participation to be human-driven. +While AI may be used to aid in creating code contributions or to communicate more effectively, +it is a poor substitute for human intelligence. +Additionally, the ease with which such content is generated provides a significant burden on +maintainers if applied without consideration (see also, [The Cost of AI in Open Source Maintenance](https://adrin.info/the-cost-of-ai-in-open-source-maintenance.html) +). +As a result, we expect you to understand the code contributions you submit and +minimize the use of AI in dialogue. + + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of +acceptable behavior and will take appropriate and fair corrective action in +response to any behavior that they deem inappropriate, threatening, offensive, +or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, and will communicate reasons for moderation +decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when +an individual is officially representing the community in public spaces. +Examples of representing our community include using an official e-mail address, +posting via an official social media account, or acting as an appointed +representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported to the community leaders responsible for enforcement at `openmlhq@googlegroups.com`. +All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the +reporter of any incident. + +Enforcement may take the shape of a correction, warning, or (temporary) ban at the discretion +of the community leaders. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], +version 2.0, available at +https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. + +Community Impact Guidelines were inspired by [Mozilla's code of conduct +enforcement ladder](https://github.com/mozilla/diversity). + +[homepage]: https://www.contributor-covenant.org + +For answers to common questions about this code of conduct, see the FAQ at +https://www.contributor-covenant.org/faq. Translations are available at +https://www.contributor-covenant.org/translations. diff --git a/mkdocs.yml b/mkdocs.yml index 17eafa8e..a5570bfa 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,13 +25,14 @@ nav: - Setup: contributing/setup.md - Tests: contributing/tests.md - Project Overview: contributing/project_overview.md + - Documentation: contributing/documentation.md - Database: - database/index.md - openml: database/openml.md - openml_expdb: database/openml_expdb.md - Contributing: - contributing/index.md - - Documentation: contributing/documentation.md + - Code of Conduct: contributing/code_of_conduct.md markdown_extensions: - attr_list From b7eee966cae2c49725be13d0d7ac9ea30d32e730 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Wed, 8 Jul 2026 17:08:28 +0200 Subject: [PATCH 05/22] Move pre-commit to separate dependency category. --- docker/python/Dockerfile | 2 +- docs/contributing/setup.md | 58 +++++++++++++++++++++++--------------- docs/roadmap.md | 7 +++++ pyproject.toml | 4 ++- 4 files changed, 46 insertions(+), 25 deletions(-) diff --git a/docker/python/Dockerfile b/docker/python/Dockerfile index 2837e165..27bc88b4 100644 --- a/docker/python/Dockerfile +++ b/docker/python/Dockerfile @@ -13,7 +13,7 @@ RUN python -m pip install uv WORKDIR /app COPY pyproject.toml . -RUN uv pip install -e ".[dev]" +RUN uv pip install -e ".[dev, tests]" COPY . /app RUN mkdir /config diff --git a/docs/contributing/setup.md b/docs/contributing/setup.md index 977a6b2b..fcd4386e 100644 --- a/docs/contributing/setup.md +++ b/docs/contributing/setup.md @@ -1,14 +1,36 @@ # Setting up the development environment -First, follow the [installation](../installation.md#local-installation) instructions -for contributors to install a local fork with optional development dependencies. -Stop when you reach the section "Setting up a Database Server". +## Getting the Code +If you intend to submit your changes to the REST API in a PR, +then you are required to make your changes in a [fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo). + +Then clone the repository (or your fork). +The remainder of the guide assumes you're working from the cloned repository's root directory (i.e., the one you also see on the repository's homepage). + +## Virtual Environment +We primarily use the docker containers for executing the code and running tests. +However, we need to install some tools on the host machine for development purposes. +To install the development dependencies into an isolated environment, first create +a virtual environment and activate it: + +```bash +python -m venv .venv +source .venv/bin/activate +``` + +We can use [`uv`](https://docs.astral.sh/uv/) to easily install the required dependencies from our [pyproject.toml](). + +```bash +python -m pip install uv +uv pip instal -e ".[dev]" +``` ## Pre-commit -We use [`pre-commit`](https://pre-commit.com) to ensure certain tools and checks are -ran before each commit. These tools perform code-formatting, linting, and more. This -makes the code more consistent across the board, which makes it easier to work with +We use `git` for version control and we use [`pre-commit`](https://pre-commit.com) +to ensure certain tools and checks are ran before each commit. +These tools perform code-formatting, linting, and more. +This makes the code more consistent across the board, which makes it easier to work with each other's code and also can catch common errors. After installing it, it will automatically run when you try to make a commit. Install it now and verify that all checks pass out of the box: @@ -58,10 +80,13 @@ This will spin up 5 services, as defined in the `compose.yaml` file: The PHP REST API needs Elasticsearch. In some cases, it also needs the ES indices to be built. The current set up does not automatically build ES indices, because that takes a long time. + The enable building ES indices on startup set `INDEX_ES_DURING_STARTUP=true` in `docker/php/.env`, + which will trigger the building of ES indices on the next startup of the PHP API service. Exposing ports to the host network isn't needed for development, but may be useful to inspect responses directly from the host machine. !!! note + On arm-based Macs, you need to enable Rosetta emulation for Docker for the Elasticsearch container to work. We can now run the full test suite, which takes about 4 minutes: @@ -71,7 +96,7 @@ docker compose exec python-api python -m pytest tests ``` There are three important [test markers](https://docs.pytest.org/en/7.1.x/example/markers.html) to be aware of: - - `php_api`: all tests that require the PHP API container. These are tests which + - `php_api`: all tests that require the PHP API container. - `python_api`: all tests that require the Python API container. That's almost all of them. - `slow`: for long-running tests. Currently only one test. @@ -80,7 +105,7 @@ The `not slow` is only needed if a slow test would be included in your test sele Examples: - `docker compose exec python-api python -m pytest tests -m "not php_api and not slow"`, here the test selection is made primarily through markers. This command takes a few seconds. - - `docker compose exec python-api python -m pytest tests/routers/openml/dataset_tag_test.py`, here the test selection is made through specifying the file with tests. Since this test file naturally includes neither migration tests (in `tests/routers/openml/migration`) nor the slow test (at `tests/routers/openml/datasets_list_datasets_test.py`), excluding tests through markers is unnecessary. This command takes a few seconds. + - `docker compose exec python-api python -m pytest tests/routers/dataset_tag_test.py`, here the test selection is made through specifying the file with tests. You don't always need every container, often just having a database and the Python-based @@ -93,6 +118,7 @@ docker compose up python-api -d Refer to the `docker compose` documentation for more uses. !!! note + We are working on making it easy to run tests from your local shell instead of the container ([#232](https://github.com/openml/server-api/pull/232)). This will likely be limited to the tests that do not need the PHP API. Our CI pipeline runs all tests. ### Connecting to containers @@ -135,10 +161,6 @@ For files, we follow the convention of _appending_ `_test`. Try to keep tests as small as possible, and only rely on database and/or web connections when absolutely necessary. -!!! Failure "" - - Instructions are incomplete. Please have patience while we are adding more documentation. - ## YAML validation The project contains various [`yaml`](https://yaml.org) files, for example to configure @@ -163,7 +185,7 @@ The following `yaml` files have schemas: === "VSCode" - In VSCode, these can be configured from `settings` > `Extetions` > + In VSCode, these can be configured from `settings` > `Extensions` > `JSON` > `Edit in settings.json`. There, add mappings per file or file pattern. For example: @@ -178,13 +200,3 @@ The following `yaml` files have schemas: ] ``` - -## Connecting to another database -In addition to the database setup described in the [installation guide](../installation.md#setting-up-a-database-server), -we also host a database on our server which may be connected to that is available -to [OpenML core contributors](https://openml.org/about). If you are a core contributor -and need access, please reach out to one of the engineers in Eindhoven. - -!!! Failure "" - - Instructions are incomplete. Please have patience while we are adding more documentation. diff --git a/docs/roadmap.md b/docs/roadmap.md index 5bb95e71..feb20aac 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -26,3 +26,10 @@ We can also start supporting different file formats for e.g., task splits or pre ## Phase 4: New Features Richer metadata for assets on OpenML (e.g., structured citation data), collaboration features, OAuth support, and support for other data modalities are just some of the features we want to work on in 2027 and beyond. + + +## Other Services +Besides the REST API, there are a few other components which need to be developed: + + - A logstash pipeline to populate ES indices based on the database data and/or a message queue. Currently, the PHP-based REST API directly puts documents into ES indices, which makes it prone to getting out of sync, and requires an ES server to run to use/test the API in ways which otherwise to not require an ES index. + - Evaluation Engine. A service which computes meta-features for datasets, including the "qualities" and "features" you find with datasets. diff --git a/pyproject.toml b/pyproject.toml index 1ddbcc0e..3ff160e5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -27,9 +27,11 @@ dependencies = [ [project.optional-dependencies] dev = [ + "pre-commit", +] +tests = [ "coverage", "pytest-cov", - "pre-commit", "pytest", "pytest-mock", "pytest-asyncio", From 94975a4e4d97b47b02fdbfa605c517b5780226f0 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Wed, 8 Jul 2026 17:24:57 +0200 Subject: [PATCH 06/22] Update the migration guide --- docs/migration.md | 68 +++++++++++++++++++++++++++++------------------ 1 file changed, 42 insertions(+), 26 deletions(-) diff --git a/docs/migration.md b/docs/migration.md index 85606910..1db29620 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -17,8 +17,12 @@ If there is a behavioral change which was not documented but affects you, please It is possible this migration guide is out of sync for endpoints not yet deployed to production (currently that includes them all). Before an endpoint is deployed to production we will ensure that the documentation is up-to-date to the best of our knowledge. -# RFC 9457 Errors -Errors will follow the RFC9457 standard. However, the original "code" is preserved through a custom field. +## Standardized Errors +The Pyhon-based API will return different HTTP status codes depending on the error (for example, returning `404 NOT FOUND` if a requested dataset does not exist). +Moreover, the body of the response will contain a JSON body that adheres to the RFC9457 standard. + +### Error Response Body + Take for example the "Dataset not found" response for trying to access a dataset that does not exist. ```diff title="CURL Commands" @@ -32,15 +36,22 @@ curl -i https://www.openml.org/api/v1/json/data/1000000 + {"type":"https://openml.org/problems/dataset-not-found","title":"Dataset Not Found","status":404,"detail":"No dataset with id 100000 found.","code":"111"} ``` -You will notice that the response still contains a "code" of "111" (though as a top level property not embedded in the "error" scope). -This field is included to support the migration of clients, but should be considered deprecated. -As per the RFC9457 standard, the "type" field now includes the unique code for the error. -The "title" field is a human readable summary of the general issue and the "detail" field may provide additional information for the specific request. -They _will_ be resolvable URIs in the future, providing a page with more information. +As per the RFC9457 standard, the `type` field now includes the unique code for the error. +You will notice that the response still contains a `code` of "111" (though as a top level property not embedded in the "error" scope). + +The `title` field is a human readable summary of the general issue and the `detail` field may provide additional information for the specific request. +The URIs provided through `type` _will_ be resolvable in the future, providing a page with more information. + +!!! warning + + The`code` field is only included to support the migration of clients, but should be considered **deprecated**. Prefer to use the `type` field to identify the error. -In some cases the JSON endpoints previously returned XML ([example](https://github.com/openml/OpenML/issues/1200)), the new API always returns JSON. -# Appropriate HTTP Status Codes +!!! info + + In some cases the JSON endpoints previously returned XML ([example](https://github.com/openml/OpenML/issues/1200)), the new API always returns JSON. + +### HTTP Status Codes There are several cases where the PHP server did not provide semantically correct status codes. The Python server aims to correct that. The errors that changed which are most likely to occur are probably errors when there is no result, or when the input is incorrect. @@ -69,15 +80,20 @@ For incorrect input (e.g., providing a string instead of an integer identifier): !!! warning "Input validation has been added to many end points" - There are endpoints which previously did not do any input validation. - These endpoints now do enforce stricter input constraints. - Constraints for each endpoint parameter are documented in the API docs. + There are endpoints which previously did not validate input, but now enforce input constraints. + Constraints for each endpoint parameter are documented in the API docs. + +## Endpoint Specific Notes + +!!! warning -# Endpoint Specific Notes + As we develop the REST API, we may make more changes. + Notes in this section are likely _currently_ out of date, + but will be updated before a production-ready release. -## Datasets +### Datasets -### `GET /{dataset_id}` +#### `GET /{dataset_id}` - Dataset format names are normalized to be all lower-case (`"Sparse_ARFF"` -> `"sparse_arff"`). - Non-`arff` datasets will not incorrectly have a `"parquet_url"` ([openml#1189](https://github.com/openml/OpenML/issues/1189)). @@ -94,7 +110,7 @@ For incorrect input (e.g., providing a string instead of an integer identifier): returns a list (which may also be empty or contain a single element). - Fields without a set value are no longer automatically removed from the response. -### `GET /data/list/{filters}` +#### `GET /data/list/{filters}` The endpoint now accepts the filters in the body of the request, instead of as query parameters. ```diff @@ -112,7 +128,7 @@ includes datasets which are private. The `limit` and `offset` parameters can now be used independently, you no longer need to provide both if you wish to set only one. -### `POST /datasets/tag` +#### `POST /datasets/tag` When successful, the "tag" property in the returned response is now always a list, even if only one tag exists for the entity. For example, after tagging dataset 21 with the tag `"foo"`: ```diff @@ -125,17 +141,17 @@ For example, after tagging dataset 21 with the tag `"foo"`: } ``` -## Setups +### Setups -### `GET /{id}` +#### `GET /{id}` The endpoint behaves almost identically to the PHP implementation. Note that fields representing integers like `setup_id` and `flow_id` are returned as integers instead of strings to align with typed JSON. Also, if a setup has no parameters, the `parameter` field is omitted entirely from the response. -### `POST /setup/tag` and `POST /setup/untag` +#### `POST /setup/tag` and `POST /setup/untag` When successful, the "tag" property in the returned response is now always a list, even if only one tag exists for the entity. When removing the last tag, the "tag" property will be an empty list `[]` instead of being omitted from the response. -## Studies +### Studies -### `GET /{id_or_alias}` +#### `GET /{id_or_alias}` Old-style "legacy" studies which are solely based on tags are no longer supported. @@ -172,9 +188,9 @@ Old-style "legacy" studies which are solely based on tags are no longer supporte |50| Hyper-parameter tuning of Decision Trees| |51| ensemble on diabetes | -## Flows +### Flows -### `GET /flow/exists/{name}/{external_version}/` +#### `GET /flow/exists/{name}/{external_version}/` Behavior has changed slightly. When a flow is found: ```diff @@ -191,9 +207,9 @@ and the HTTP header status code is `404` (NOT FOUND) instead of `200` (OK). In the future the successful case will more likely just return the flow immediately instead (see #170). -## Others +### Others -### `GET /estimationprocedure/list` +#### `GET /estimationprocedure/list` The `ttid` field has been renamed to `task_type_id`. All values are now typed. Outer levels of nesting have been removed. From 81faa972d57098fb9ee04e93431602eeecc93be5 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 12:04:31 +0200 Subject: [PATCH 07/22] Move the database documentation to developer documentation --- docs/database/index.md | 27 -------------- docs/development/database/index.md | 36 +++++++++++++++++++ docs/{ => development}/database/openml.md | 0 .../database/openml_expdb.md | 0 .../documentation.md | 0 .../project_overview.md | 0 docs/{contributing => development}/setup.md | 0 docs/{contributing => development}/tests.md | 0 mkdocs.yml | 15 ++++---- 9 files changed, 44 insertions(+), 34 deletions(-) delete mode 100644 docs/database/index.md create mode 100644 docs/development/database/index.md rename docs/{ => development}/database/openml.md (100%) rename docs/{ => development}/database/openml_expdb.md (100%) rename docs/{contributing => development}/documentation.md (100%) rename docs/{contributing => development}/project_overview.md (100%) rename docs/{contributing => development}/setup.md (100%) rename docs/{contributing => development}/tests.md (100%) diff --git a/docs/database/index.md b/docs/database/index.md deleted file mode 100644 index f1f87ec4..00000000 --- a/docs/database/index.md +++ /dev/null @@ -1,27 +0,0 @@ -# Database Schema - -The OpenML server uses two MySQL databases: - -- **[openml](openml.md)** — Core platform database for user accounts, file storage, access control, and forum threads. -- **[openml_expdb](openml_expdb.md)** — Experiment database storing datasets, tasks, flows (implementations), runs, evaluations, and studies. - -These documentation pages describe their current schemas. -There are several tables which are no longer in use, these are mentioned but not described. -The plan is to revise the database schema after we sunset the PHP API, to avoid having to make changes to two APIs. - -When launching the services as described in ["Installation"](../installation.md), you can access the mysql server with both databases using `docker compose exec database mysql -uroot -pok`. - -## Why we use queries instead of an ORM tool -There are two main reasons why we do not use an ORM tool *yet*. - -First, we want to keep queries close to the original PHP implementation. -Not using an ORM makes it natural to stick with similar or identical queries that the PHP API uses. -Introducing an ORM, and having it construct queries for us, adds an extra layer of changes. -Performance issues may be harder to trace down if they arise. - -Second, we will likely revise the database schema significantly after the PHP API is sunset. -The current schema is over a decade old and contains design decisions based on expected future use or features -and other decisions which we may want to revise based on our experience running OpenML. -It seems easier to support those changes when we do not yet use an ORM tool. - -The expectation is that we will move to using an ORM tool once the schema is revised and stable. diff --git a/docs/development/database/index.md b/docs/development/database/index.md new file mode 100644 index 00000000..e73012dc --- /dev/null +++ b/docs/development/database/index.md @@ -0,0 +1,36 @@ +# Database Schema + +The OpenML server uses two MySQL databases: + +- **[openml](openml.md)** — Core platform database for user accounts, file storage, access control, and forum threads. +- **[openml_expdb](openml_expdb.md)** — Experiment database storing datasets, tasks, flows (implementations), runs, evaluations, and studies. + +These documentation pages describe their current schemas. +There are several tables which are no longer in use, these are mentioned but not described. +The plan is to revise the database schema after we sunset the PHP API, to avoid having to make changes to two APIs. + +When launching the services as described in ["Development Environment"](../setup.md#docker), you can access the mysql server with both databases using `docker compose exec database mysql -uroot -pok`. + +## ORM usage + +The project currently uses a mix of ORM and SQL queries to interact with the databases. +The adoption of ORM is recent and work in progress. +We haven't gone back through the codebase yet to ensure the general guidelines below are consistently applied. + +Ultimately we plan to use ORM for all queries, unless we have the data to show that the ORM generated query is significantly slower than a raw SQL query. +However, so far we have used ORM only for operations on a single table or with a single join. +For queries that join multiple tables together, especially those with any kind of run data, we use an SQL query as a poorly generated query may significantly impact performance. +While we can investigate that as it comes up, we took the pragmatic approach of using the "good enough" queries from the PHP API. + +### Why we did not use ORM from the start +When we started the project two years ago, we chose not to use an ORM primarily because: + + - ORM constructed queries may be different from the ones PHP uses. Adding this change may make it harder to debug performance issues. + - We will revise the database schema, and it seemed easier to make these changes if the schema is not also encoded in the code base any more than it needs to be. + - Writing the SQL queries was a good way to get more familiar with the database. + +However, with the mixed approach these concerns are limited: + + - "Complex" queries may continue to use SQL queries. Simple operations such as `SELECT`, `UPDATE`, or `DELETE` statements from single tables should produce identical SQL queries. + - We use [Reflection](https://docs.sqlalchemy.org/en/20/core/reflection.html#reflecting-database-objects) instead of declaring the tables entirely. We declare only as much of the database as we need to support the API, which means we are not encoding the database schema any more than a SQL query would. + - We have gotten more familiar with the database during development already. diff --git a/docs/database/openml.md b/docs/development/database/openml.md similarity index 100% rename from docs/database/openml.md rename to docs/development/database/openml.md diff --git a/docs/database/openml_expdb.md b/docs/development/database/openml_expdb.md similarity index 100% rename from docs/database/openml_expdb.md rename to docs/development/database/openml_expdb.md diff --git a/docs/contributing/documentation.md b/docs/development/documentation.md similarity index 100% rename from docs/contributing/documentation.md rename to docs/development/documentation.md diff --git a/docs/contributing/project_overview.md b/docs/development/project_overview.md similarity index 100% rename from docs/contributing/project_overview.md rename to docs/development/project_overview.md diff --git a/docs/contributing/setup.md b/docs/development/setup.md similarity index 100% rename from docs/contributing/setup.md rename to docs/development/setup.md diff --git a/docs/contributing/tests.md b/docs/development/tests.md similarity index 100% rename from docs/contributing/tests.md rename to docs/development/tests.md diff --git a/mkdocs.yml b/mkdocs.yml index a5570bfa..a4eaf85b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -22,14 +22,15 @@ nav: - Deployment: - installation.md - Development: - - Setup: contributing/setup.md - - Tests: contributing/tests.md - - Project Overview: contributing/project_overview.md - - Documentation: contributing/documentation.md + - Project Overview: development/project_overview.md + - Development: + - "Development Environment": development/setup.md + - "Running Tests": development/tests.md + - Documentation: development/documentation.md - Database: - - database/index.md - - openml: database/openml.md - - openml_expdb: database/openml_expdb.md + - development/database/index.md + - openml: development/database/openml.md + - openml_expdb: development/database/openml_expdb.md - Contributing: - contributing/index.md - Code of Conduct: contributing/code_of_conduct.md From 21c1086a9a9dd054f4d02e19faa2e853cc9fce08 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 12:09:03 +0200 Subject: [PATCH 08/22] Minor changes to the test documentation --- docs/development/tests.md | 12 ++++++------ mkdocs.yml | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/development/tests.md b/docs/development/tests.md index 6724b888..5b222710 100644 --- a/docs/development/tests.md +++ b/docs/development/tests.md @@ -1,7 +1,7 @@ # Testing This page covers running and writing tests for the REST API. -It assumes you already followed the instructions on the ["Setup"](setup.md) page. +It assumes you already followed the instructions to set up your [Development Environment](setup.md). !!! note "Follow the documentation" @@ -37,7 +37,7 @@ in the `docker/php/.env` file to `true` before starting the container. ## Writing Tests -We use the ubiquitous [Pytest](https://docs.pytest.org) framework when writing tests. +We use [Pytest](https://docs.pytest.org) for writing and running tests. ### File Structure When writing tests, we have the following additional conventions on the file structure: @@ -53,10 +53,10 @@ Some guidelines and things to keep in mind when writing tests: - Mark tests that update the database in anyway with the `mut` marker (`@pytest.mark.mut`). - If the test is excessively slow (>0.1 sec) and does not connect to PHP, use a `slow` marker. Tests that include PHP always require roundtrips through other services which makes them slow by default. PHP tests can be filtered out with the automatically generated "php_api" marker. - Four common fixtures you might need when writing tests are: - - py_api: an async client for the Python based REST API - - php_api: an async client for the PHP based REST API - - expdb_test: an AsyncConnection to the "expdb" OpenML database. - - user_test: an AsyncConnection to the "openml" OpenML database. + - `py_api`: an async client for the Python based REST API + - `php_api`: an async client for the PHP based REST API + - `expdb_test`: an AsyncConnection to the "expdb" OpenML database. + - `user_test`: an AsyncConnection to the "openml" OpenML database. - Above fixtures have considerable per-test overhead. Use them only when you need them. - When writing assertions the expected value (a constant, or a php response) should be on the right (`assert response == expected`). diff --git a/mkdocs.yml b/mkdocs.yml index a4eaf85b..0e5416d6 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,7 +25,7 @@ nav: - Project Overview: development/project_overview.md - Development: - "Development Environment": development/setup.md - - "Running Tests": development/tests.md + - "Writing Tests": development/tests.md - Documentation: development/documentation.md - Database: - development/database/index.md From b2de592c8148db2bb3be1310a4145067ba19d936 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 15:20:42 +0200 Subject: [PATCH 09/22] Expand project overview --- docs/development/CI_CD.md | 0 docs/development/code.md | 0 docs/development/project_overview.md | 157 +++++++++++++++++++++++++-- docs/development/tests.md | 19 ++++ mkdocs.yml | 8 +- 5 files changed, 174 insertions(+), 10 deletions(-) create mode 100644 docs/development/CI_CD.md create mode 100644 docs/development/code.md diff --git a/docs/development/CI_CD.md b/docs/development/CI_CD.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/development/code.md b/docs/development/code.md new file mode 100644 index 00000000..e69de29b diff --git a/docs/development/project_overview.md b/docs/development/project_overview.md index 7264fd7d..052e7fd0 100644 --- a/docs/development/project_overview.md +++ b/docs/development/project_overview.md @@ -1,15 +1,154 @@ # Project overview -The Python-based REST API serves several groups of endpoints: +This page describes the context in which the REST API is deployed, +as well as a rough structure of the codebase. - - `/old/`: serves the old-style JSON format, this should mimic the PHP responses exactly with the only deviations recorded in the [migration guide](../migration.md). - - `/mldcat_ap/`: serves datasets in [MLDCAT_AP](https://semiceu.github.io/MLDCAT-AP/releases/1.0.0/) format. - - `/*`: serves new-style JSON format. At this point it is intentionally similar to the old-style format. +## Context of the REST API -The endpoints are specified in subdirectories of `src/routers`. -They pull data from the database through the `src/database` module. -The schemas for each entity, and possible conversions between them, are defined in the `src/schemas` directory. +!!! info "For Core Contributors" -!!! Failure "" + If you are an OpenML core contributor, more information on the services and how they are deployed is available in an internal wiki. + Please reach out to in the `#core` Slack channel for access. - Instructions are incomplete. Please have patience while we are adding more documentation. + +The REST API should always be deployed with access to a database. +In production, these services are deployed in a kubernetes cluster. +A typical request flow would then follow this pattern: + +```mermaid +sequenceDiagram + actor Client + + box Server + participant REST API + actor Database@{ "type" : "database" } + end + + Client->> REST API: `GET /datasets/1` + REST API ->> Database: Query Database + Database -->> REST API: Results + REST API ->> REST API: Check Access + alt Access Granted + opt Fetch More Data + REST API ->> Database: Query Database + Database -->> REST API: Results + end + REST API ->> REST API: Format Response + REST API -->> Client: `HTTP 200 {"dataset_id": ...}` + else Access Denied + alt No Credentials + REST API -->> Client: `HTTP 401 {"type": ...}` + else Not Allowed + REST API -->> Client: `HTTP 403 {"type": ...}` + end + end + +``` + +Note that authentication or authorization is required primarily for operations which mutate the database in some way (uploading, tagging, ...). +Besides the REST API and the database, the server will also have a number of auxiliary services deployed which support background processes on incoming data: + + - An Evaluation Engine analyzes uploaded datasets to compute its "features" and "qualities" , processes uploaded runs to create "evaluations" and creates task splits. Some of these actions update database state (through calling REST API endpoints). + - A MinIO server server for file storage. The REST API will stores uploaded datasets and runs there (not yet implemented). + - An Elastic Search server (see ["Elasticsearch"](#elasticsearch) below). + - A service which creates a [Croissant](https://github.com/mlcommons/croissant) metadata file for a new dataset. + - A service which converts uploaded ARFF datasets to Parquet (to be phased out as ARFF is phased out). + + +### Elasticsearch +We have Elasticsearch (ES) indices which reflect the state of the database and make it easily accessible to e.g., the website. +The PHP-based REST API directly inserts documents into the ES indices. +This is problematic because: + + - The REST API will return errors if the Elasticsearch service is unavailable, depending on the endpoint, this results in: + - the request being stopped early with an error, even if otherwise it would successfully complete the request. + - the request completes the request, but then returns an error anyway because of the failure to update the ES index. The user is unaware that his request actually succeeded. + - The REST API has no job queue for inserting these documents, which means if it fails once it is not inserted again, potentially leading to an out-of-sync index. + - If the ES service is available, the internal roundtrip to update the index adds to server response times. + +For these reasons, this REST API implementation will not directly connect to Elasticsearch for creating or updating indices. +Instead, we will use e.g., a Logstash pipeline to construct and update the indices directly from the database or make use of a Message Bus. +Because a Message Bus keeps some of the disadvantages listed above, we're more likely to use the Logstash approach. +The database schema does not accommodate this well, however. +For example, there is generally no way to tell which data is removed or updated. +In practice, this means that initially the Logstash pipeline will just recreate the indices periodically. + +In [Phase 3](../roadmap.md##phase-3-revisiting-the-database) we can update the schema to work better with Logstash. + +The REST API _may_ make use of ES in the future for e.g., list operations. +These operations which list lots of data aggregated from many different tables would be considerably faster if we query them from a preconstructed index instead. + +For more info, see [issue#233](https://github.com/openml/server-api/issues/233). + + +### Authentication +Authentication is currently supported by providing an api key as a query parameter (mimicking the PHP REST API). +To avoid accidentally leaking credentials (e.g., with an HTTP request, or our server access logs), we will move this form of authentication to the HTTP header, see [issue#203](https://github.com/openml/server-api/issues/203). +Ultimately, it would be good to use a dedicated service for authentication, preferably with OAuth, see [issue#69](https://github.com/openml/server-api/issues/69)(externalizing authentication). + +## Repository File Structure +The diagrams below can be used as a rough indication for the repository file structure. +Many subdirectories and files are omitted to highlight the structure rather than specific content. +Omissions are *not* explicitly mentioned in the diagram. +The `src/` and `tests/` directories are expanded in separate diagrams below. + +```mermaid +treeView-beta + .github/ + workflows/ ## CI/CD scripts + ISSUE_TEMPLATE/ ## GH issue creation prompt and templates + PULL_REQUEST_TEMPLATE.md + docker/ ## For building and configuring containers + docs/ + Dockerfile ## For these doc pages + php/ + .env ## Configuration of the PHP REST API container + python/ + Dockerfile ## For this REST API + docs/ ## Source for this documentation + development/ + project_overview.md ## This file + src/ + main.py ## REST API entrypoint + config.toml ## REST API configuration + tests/ + .pre-commit-config.yaml + codecov.yaml ## codecov CI configuration + compose.yaml + mkdocs.yaml ## Documentation Configuration + pyproject.toml ## Project metadata, dependencies +``` + +The `src/` directory is then organised as follows (omitted files indicated with `...`): + +```mermaid +treeView-beta +core/ + conversions.py ## Common data conversion (e.g., unwrap single element list) + errors.py ## Map for all REST API errors + formatting.py ## Common data formatting (e.g., split csv string to list) + logging.py ## Logging setup and middleware + types.py ## Common types, e.g., Identifier +database/ ## Anything that interacts with the database + models/ ## ORM classes + base.py ## Base classes every ORM class should inherit from + ... + engine.py ## SQLAlchemy Engine Creation + exceptions.py ## Understandable Exceptions + datasets.py ## CRUD operations on datasets + ... +routers/ ## Defines Endpoints + dependencies.py ## FastAPI Dependencies (e.g., authentication, database connections) + datasets.py ## Endpoints under `/datasets` + ... +schemas/ ## Classes that define the REST API schema + common.py ## Used across different subpaths + datasets.py ## Related to `/datasets` endpoints + ... +config.py ## For loading/accessing the configuration +config.toml ## The configuration file +main.py ## API Entrypoint +``` + +The `tests/` directory largely follows the `src/` directory structure, except that each endpoint has a dedicated file (it's possilbe changes to the `src/` structure were made without updating `tests/`). +For more information on tests, see ["Writing Tests"](tests.md). diff --git a/docs/development/tests.md b/docs/development/tests.md index 5b222710..09ec893c 100644 --- a/docs/development/tests.md +++ b/docs/development/tests.md @@ -45,6 +45,25 @@ When writing tests, we have the following additional conventions on the file str - Use a `_test` suffix when naming our files (not a `test_` prefix). Our tests already exist in a `tests` directory, and in common tree list side panels it's likely you can only see the start of file names, so this is more informative. - One dedicated test file per endpoint +The diagram below shows the structure of the `tests/` directory visually. +Omitted files are indicated with `...`. +```mermaid +treeView-beta +database/ + runs_test.py ## Tests for src/database/runs.py + ... +dependencies/ ## Tests for dependencies of src/routers/dependencies.py + ... +resources/ ## Files required for testing purposes + ... +routers/ + dataset_tag_test.py ## tests for src/routers/datasets.py's `/datasets/{IDENTIFIER}/tag` endpoint + ... +config_test.py ## Tests for config.py +conftest.py ## Pytest configuration and fixtures +constants.py ## Constants used for multiple tests (e.g., +users.py ## Stubs and constants for user accounts +``` ### General Test Guidelines Some guidelines and things to keep in mind when writing tests: diff --git a/mkdocs.yml b/mkdocs.yml index 0e5416d6..a47a4f13 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -25,7 +25,9 @@ nav: - Project Overview: development/project_overview.md - Development: - "Development Environment": development/setup.md + - "Writing Code": development/code.md - "Writing Tests": development/tests.md + - "CI/CD": development/CI_CD.md - Documentation: development/documentation.md - Database: - development/database/index.md @@ -39,7 +41,11 @@ markdown_extensions: - attr_list - admonition - pymdownx.details - - pymdownx.superfences + - pymdownx.superfences: + custom_fences: + - name: mermaid + class: mermaid + format: !!python/name:pymdownx.superfences.fence_code_format - pymdownx.tabbed: alternate_style: true - pymdownx.emoji From 7d593307fed6d836b1ed69ffa3f9e1f3ed24ba6b Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 15:37:01 +0200 Subject: [PATCH 10/22] Fix broken links --- docs/contributing/index.md | 4 ++-- docs/development/documentation.md | 24 ++++++++++++++---------- docs/development/project_overview.md | 2 +- 3 files changed, 17 insertions(+), 13 deletions(-) diff --git a/docs/contributing/index.md b/docs/contributing/index.md index fcd7ef38..6329f0ac 100644 --- a/docs/contributing/index.md +++ b/docs/contributing/index.md @@ -27,7 +27,7 @@ When you open a bug report, please include: For minor fixes such as correcting a typo or a broken link, it's fine to make the changes and submit them through a pull request. For larger changes, please make sure you are assigned to an issue first as is described -in the "[Code](#code)" section of this page. Then, visit the "[Documentation](documentation.md)" +in the "[Code](#code)" section of this page. Then, visit the "[Developer Documentation](../development/project_overview.md)" page to learn how to contribute documentation changes. ### Code @@ -40,7 +40,7 @@ everything is clear, it's not out of date, and "officially" assign you to the is that others know its being worked on. Making sure there is a clear description and clear assignment helps prevent a situation where someone makes a large contribution that is unwanted, or is simultaneously developed by someone else. With an issue assigned, -please head over to the "[Setting up the development environment](setup.md)" section. +please head over to the "[Setting up the development environment](../development/setup.md)" section. !!! info "Help Wanted?" diff --git a/docs/development/documentation.md b/docs/development/documentation.md index cbeb1a48..85d1c1cb 100644 --- a/docs/development/documentation.md +++ b/docs/development/documentation.md @@ -1,17 +1,16 @@ # Documentation with mkdocs-material -Our documentation is built using [mkdocs](https://www.mkdocs.org) and the -[mkdocs-material](https://squidfunk.github.io/mkdocs-material/) theme. We also use some -[plugins](https://squidfunk.github.io/mkdocs-material/plugins/). -Please refer to the ["Getting Started"](https://squidfunk.github.io/mkdocs-material/getting-started/) -pages of `mkdocs-material` for a general overview on how to work with `mkdocs-material`. +Our documentation is built using [Zensical](https://zensical.org/). +It's made by the people that created Material for MkDocs, and is "backwards compatible" with that project (in fact, these docs were originally written for that framework). +Please refer to the ["Getting Started"](https://zensical.org/docs/get-started/) +pages of Zensical for a general overview on how to work with Zensical. + All documentation files are in the `docs/` folder, except for the configuration file which is `mkdocs.yml` at the root of the repository. For minor changes, it should be fine to edit the page directly on Github. -That should commit to a separate branch (or fork), and you can set up a pull request. -For larger changes, clone a fork of the repository as described in the -["Local Installation"](../installation.md#local-installation) section. +Commit the change to a separate branch (or fork) and you set up a pull request. +For larger changes, clone the repository as described in ["Getting the Code"](setup.md#getting-the-code). === "Docker" @@ -23,21 +22,26 @@ For larger changes, clone a fork of the repository as described in the ``` + === "Local installation" Instead of installing all dependencies (with `python -m pip install -e ".[docs]"`), you may also install just the documentation dependencies: ```bash - python -m pip install mkdocs-material mkdocs-section-index + python -m pip install zensical ``` You can then build and serve the documentation with ```bash - python -m mkdocs serve + python -m zensical serve ``` This will serve the documentation from the `docs/` directory to [http://localhost:8000/](http://localhost:8000/). Any updates you make to files in that directory will be reflected on the website. When you are happy with your changes, just commit and set up a pull request! + +!! tip "Can't connect to Docker?" + + If the docker container is running but you cannot access it, it's likely that you did not correctly specify the compose files to include `compose.ports.yaml`. diff --git a/docs/development/project_overview.md b/docs/development/project_overview.md index 052e7fd0..7173bb09 100644 --- a/docs/development/project_overview.md +++ b/docs/development/project_overview.md @@ -73,7 +73,7 @@ The database schema does not accommodate this well, however. For example, there is generally no way to tell which data is removed or updated. In practice, this means that initially the Logstash pipeline will just recreate the indices periodically. -In [Phase 3](../roadmap.md##phase-3-revisiting-the-database) we can update the schema to work better with Logstash. +In [Phase 3](../roadmap.md#phase-3-revisiting-the-database) we can update the schema to work better with Logstash. The REST API _may_ make use of ES in the future for e.g., list operations. These operations which list lots of data aggregated from many different tables would be considerably faster if we query them from a preconstructed index instead. From 36a73ceffb26a8f9772c18c2b9f223cfc1ac5b0f Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 15:52:26 +0200 Subject: [PATCH 11/22] Note on CI/CD tooling --- docs/development/CI_CD.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/development/CI_CD.md b/docs/development/CI_CD.md index e69de29b..28967a4f 100644 --- a/docs/development/CI_CD.md +++ b/docs/development/CI_CD.md @@ -0,0 +1,17 @@ +We have the following GitHub actions: + + - `.github/workflows/docker-publish.yml`: Build and publish a new Docker image to [our DockerHub repository](https://hub.docker.com/r/openml/rest-api) on release. + - `.github/workflows/docs.yml`: Build and deploy these documentation pages on a merge to the main branch. + - `.github/workflows/tests.yml`: Run all tests on pull requests and a push to main, pushes code coverage reports to Codecov. + +Additionally, we also have the following services configured: + + - `.pre-commit-config.yaml`: For local pre-commits, but also used by pre-commit.ci on pull requests. + - `codecov.yaml`: For providing code coverage reports on pull requests. + - `CodeRabbit` and `Sourcery AI` for automated code reviews. + + !!! example "On AI code reviews" + + We are currently experimenting with AI code reviews. + For all their flaws in code generation, these bots do regularly point out typos and sometimes bring in context beyond the authors' familiarity. + They also make poor suggestions, but those are typically easy to identify and ignore. From 63544e253aa573d3889d33c4f58f4ed898fb9be8 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 17:10:14 +0200 Subject: [PATCH 12/22] Provide additional information for writing code --- docs/development/code.md | 147 +++++++++++++++++++++++++++ docs/development/database/index.md | 24 ----- docs/development/project_overview.md | 38 +------ mkdocs.yml | 3 +- 4 files changed, 151 insertions(+), 61 deletions(-) diff --git a/docs/development/code.md b/docs/development/code.md index e69de29b..93341d7b 100644 --- a/docs/development/code.md +++ b/docs/development/code.md @@ -0,0 +1,147 @@ +The REST API is built with [FastAPI](https://fastapi.tiangolo.com/) for defining the endpoints, and [SQLAlchemy](https://docs.sqlalchemy.org/en/20/) for interaction with the database. + +In the code base we aim to keep separate the interactions with the database (in the `database/` module) from the application logic and formatting of the endpoint (in `routers/` and `schemas/`, respectively). + +```mermaid +treeView-beta +core/ + conversions.py ## Common data conversion (e.g., unwrap single element list) + errors.py ## Map for all REST API errors + formatting.py ## Common data formatting (e.g., split csv string to list) + logging.py ## Logging setup and middleware + types.py ## Common types, e.g., Identifier +database/ ## Anything that interacts with the database + models/ ## ORM classes + base.py ## Base classes every ORM class should inherit from + ... + engine.py ## SQLAlchemy Engine Creation + exceptions.py ## Understandable Exceptions + datasets.py ## CRUD operations on datasets + ... +routers/ ## Defines Endpoints + dependencies.py ## FastAPI Dependencies (e.g., authentication, database connections) + datasets.py ## Endpoints under `/datasets` + ... +schemas/ ## Classes that define the REST API schema + common.py ## Used across different subpaths + datasets.py ## Related to `/datasets` endpoints + ... +config.py ## For loading/accessing the configuration +config.toml ## The configuration file +main.py ## API Entrypoint +``` + +!!! note "Exceptions" + + The initial development of the code base took place in sparse bursts of activity in a timespan of 3 years. + This means that it wasn't always easy to work with full context, but also that views on the "correct" structure and patterns changed over time. + Because of these factors, you may see deviations from the patterns described in this document, but we aim for consistent application of these ideas (or better ones :-)). + + +## Anatomy of an Endpoint + +We show the general structure of an endpoint by way of example. +The following (simplified) code for tagging a dataset (from `src/routers/datasets.py`) has inline code annotation about what is going on. +Click the (+) icon for information. + +```python title="src/routers/datasets.py" + +router = APIRouter(prefix="/datasets", tags=["datasets"]) # (1)! + +@router.post(path="/tag") # (2)! +async def tag_dataset( + data_id: Annotated[Identifier, Body()], # (3)! + tag: Annotated[TagString, Body()], # (4)! + user: Annotated[User, Depends(fetch_user_or_raise)], # (5)! + expdb_db: Annotated[AsyncConnection, Depends(expdb_connection)], # (6)! +) -> dict[str, dict[str, Any]]: # (7)! + """Add a tag to the dataset, this tag is publicly visible to all users.""" # (8)! + try: + await database.datasets.tag(data_id, tag, user_id=user.user_id, connection=expdb_db) # (9)! + except ForeignKeyConstraintError: + msg = f"Dataset {data_id} not found." + raise DatasetNotFoundError(msg, code=472) from None # (10)! + except DuplicatePrimaryKeyError: + msg = f"Dataset {data_id} already tagged with {tag!r}." + raise TagAlreadyExistsError(msg) from None + + logger.info("Dataset {data_id} tagged '{tag}'.", data_id=data_id, tag=tag) # (11)! + + tags = await database.datasets.get_tags_for(data_id, expdb_db) + + return { # (12)! + "data_tag": {"id": str(data_id), "tag": tags}, + } +``` + +1. All endpoints registered with this router automatically get the `/datasets` prefix and are shown together in the Swagger documentation. +2. To register an endpoint with the router, we need to specify the method (e.g., `get`, `post`) and the path (remember the path `/tag` will get the `/datasets` prefix!). +3. We can specify input parameters to the endpoint through defining parameters for the function. The type hint annotation will specify the type of the input (here, the custom type `Identifier`) and where to expect it (see ["Request Parameters"](https://fastapi.tiangolo.com/reference/parameters/)). +4. Custom types can be convenient for specifying common constraints. For example, an `Identifier` is always a positive integer. The `core/types.py` module is the place for common data types. +5. [Dependencies](https://fastapi.tiangolo.com/reference/dependencies/) may define common functions that are called by FastAPI. They may optionally define parameters that the user should supply. For user authentication, the `fetch_user` and `fetch_user_or_raise` dependencies will automatically request an API key from the client and load the corresponding user. +6. Another common use of dependencies is establishing a connection with the database. +7. The return type annotation is used by FastAPI to parse the return value of the function using Pydantic. You can create complex return types by defining Pydantic classes. These should be defined in the `schemas/` directory. +8. The docstring for the function is used by FastAPI in generating the OpenAPI spec. This means it will be publicly visible in the API documentation! +9. We delegate database operations to a `database/` submodule. Remember to `await` the response! +10. Errors raised by the REST API are defined in `core/errors.py`. The message will be returned to the client. The code matches PHP REST API output and will be deprecated in the future (see also the [Migration Guide](../migration.md#error-response-body)). An exception handler is set up in `main.py` to convert these Python error objects into proper [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457.html) responses. +11. Log statements give observability to what's happening in the REST API (see [Logging](#logging)). +12. Return values are coerced into JSON form by FastAPI. + +## Logging +We use [`loguru`](https://loguru.readthedocs.io/en/stable/) for logging. +A [middleware](https://fastapi.tiangolo.com/reference/middleware/) layer automatically provides the following context to each log entry (see `core/logging.py::add_request_context_to_log`): + + - a unique request identifier + - the request method (such as `GET`, `POST`, ...) + - the request path (such as `/datasets/tag`, ...) + +Through middleware, three additional log entries are automatically written for each request: + + - information about the incoming request before processing the request + - the response to the request after processing the request + - processing time of the request after processing the request + +!!! info "Take a peek" + + If you have a locally running API container, you can have a look at the logging produced with `docker compose logs python-api`. + +It is generally useful to write additional log statements in your code, in particular when: + + - a user tries something they are not allowed to, log a warning message. + - a user permutates the database (create, update, delete), log an info message. + - something unexpected happens. + +Additional messages may be added at your discretion, following these guidelines: + + - a log message should be atomic: it should not require other messages to understand (though context is available through the `request id`). This is necessary because the log may have messages from many concurrent requests so it's unlikely that subsequent log statements occur together in the log. + - exclude sensitive data: these logs are going to be stored on the data in plain text. + +!!! note "Absence of logging" + + During early development, the logging framework wasn't set up yet. + Some endpoints may lack logging (other than the middleware). + +### ORM usage + +The project currently uses a mix of ORM and SQL queries to interact with the databases. +The adoption of ORM is recent and work in progress. +We haven't gone back through the codebase yet to ensure the general guidelines below are consistently applied. + +Ultimately we plan to use ORM for all queries, unless we have the data to show that the ORM generated query is significantly slower than a raw SQL query. +However, so far we have used ORM only for operations on a single table or with a single join. +For queries that join multiple tables together, especially those with any kind of run data, we use an SQL query as a poorly generated query may significantly impact performance. +While we can investigate that as it comes up, we took the pragmatic approach of using the "good enough" queries from the PHP API. + +???- question "Why we did not use ORM from the start" + + When we started the project two years ago, we chose not to use an ORM primarily because: + + - ORM constructed queries may be different from the ones PHP uses. Adding this change may make it harder to debug performance issues. + - We will revise the database schema, and it seemed easier to make these changes if the schema is not also encoded in the code base any more than it needs to be. + - Writing the SQL queries was a good way to get more familiar with the database. + + However, with the mixed approach these concerns are limited: + + - "Complex" queries may continue to use SQL queries. Simple operations such as `SELECT`, `UPDATE`, or `DELETE` statements from single tables should produce identical SQL queries. + - We use [Reflection](https://docs.sqlalchemy.org/en/20/core/reflection.html#reflecting-database-objects) instead of declaring the tables entirely. We declare only as much of the database as we need to support the API, which means we are not encoding the database schema any more than a SQL query would. + - We have gotten more familiar with the database during development already. diff --git a/docs/development/database/index.md b/docs/development/database/index.md index e73012dc..ab9f5c8a 100644 --- a/docs/development/database/index.md +++ b/docs/development/database/index.md @@ -10,27 +10,3 @@ There are several tables which are no longer in use, these are mentioned but not The plan is to revise the database schema after we sunset the PHP API, to avoid having to make changes to two APIs. When launching the services as described in ["Development Environment"](../setup.md#docker), you can access the mysql server with both databases using `docker compose exec database mysql -uroot -pok`. - -## ORM usage - -The project currently uses a mix of ORM and SQL queries to interact with the databases. -The adoption of ORM is recent and work in progress. -We haven't gone back through the codebase yet to ensure the general guidelines below are consistently applied. - -Ultimately we plan to use ORM for all queries, unless we have the data to show that the ORM generated query is significantly slower than a raw SQL query. -However, so far we have used ORM only for operations on a single table or with a single join. -For queries that join multiple tables together, especially those with any kind of run data, we use an SQL query as a poorly generated query may significantly impact performance. -While we can investigate that as it comes up, we took the pragmatic approach of using the "good enough" queries from the PHP API. - -### Why we did not use ORM from the start -When we started the project two years ago, we chose not to use an ORM primarily because: - - - ORM constructed queries may be different from the ones PHP uses. Adding this change may make it harder to debug performance issues. - - We will revise the database schema, and it seemed easier to make these changes if the schema is not also encoded in the code base any more than it needs to be. - - Writing the SQL queries was a good way to get more familiar with the database. - -However, with the mixed approach these concerns are limited: - - - "Complex" queries may continue to use SQL queries. Simple operations such as `SELECT`, `UPDATE`, or `DELETE` statements from single tables should produce identical SQL queries. - - We use [Reflection](https://docs.sqlalchemy.org/en/20/core/reflection.html#reflecting-database-objects) instead of declaring the tables entirely. We declare only as much of the database as we need to support the API, which means we are not encoding the database schema any more than a SQL query would. - - We have gotten more familiar with the database during development already. diff --git a/docs/development/project_overview.md b/docs/development/project_overview.md index 7173bb09..2011e009 100644 --- a/docs/development/project_overview.md +++ b/docs/development/project_overview.md @@ -86,11 +86,11 @@ Authentication is currently supported by providing an api key as a query paramet To avoid accidentally leaking credentials (e.g., with an HTTP request, or our server access logs), we will move this form of authentication to the HTTP header, see [issue#203](https://github.com/openml/server-api/issues/203). Ultimately, it would be good to use a dedicated service for authentication, preferably with OAuth, see [issue#69](https://github.com/openml/server-api/issues/69)(externalizing authentication). -## Repository File Structure +## Repository Directory Structure The diagrams below can be used as a rough indication for the repository file structure. Many subdirectories and files are omitted to highlight the structure rather than specific content. Omissions are *not* explicitly mentioned in the diagram. -The `src/` and `tests/` directories are expanded in separate diagrams below. +The structure of the `src/` directory is documented in ["Writing Code"](code.md) and that of the `tests/` directory in ["Writing Tests"](tests.md). ```mermaid treeView-beta @@ -118,37 +118,3 @@ treeView-beta mkdocs.yaml ## Documentation Configuration pyproject.toml ## Project metadata, dependencies ``` - -The `src/` directory is then organised as follows (omitted files indicated with `...`): - -```mermaid -treeView-beta -core/ - conversions.py ## Common data conversion (e.g., unwrap single element list) - errors.py ## Map for all REST API errors - formatting.py ## Common data formatting (e.g., split csv string to list) - logging.py ## Logging setup and middleware - types.py ## Common types, e.g., Identifier -database/ ## Anything that interacts with the database - models/ ## ORM classes - base.py ## Base classes every ORM class should inherit from - ... - engine.py ## SQLAlchemy Engine Creation - exceptions.py ## Understandable Exceptions - datasets.py ## CRUD operations on datasets - ... -routers/ ## Defines Endpoints - dependencies.py ## FastAPI Dependencies (e.g., authentication, database connections) - datasets.py ## Endpoints under `/datasets` - ... -schemas/ ## Classes that define the REST API schema - common.py ## Used across different subpaths - datasets.py ## Related to `/datasets` endpoints - ... -config.py ## For loading/accessing the configuration -config.toml ## The configuration file -main.py ## API Entrypoint -``` - -The `tests/` directory largely follows the `src/` directory structure, except that each endpoint has a dedicated file (it's possilbe changes to the `src/` structure were made without updating `tests/`). -For more information on tests, see ["Writing Tests"](tests.md). diff --git a/mkdocs.yml b/mkdocs.yml index a47a4f13..d062fcd3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -10,6 +10,7 @@ theme: name: material features: - content.code.copy + - content.code.annotate - navigation.indexes - navigation.sections - navigation.tabs @@ -24,7 +25,7 @@ nav: - Development: - Project Overview: development/project_overview.md - Development: - - "Development Environment": development/setup.md + - "Getting Started": development/setup.md - "Writing Code": development/code.md - "Writing Tests": development/tests.md - "CI/CD": development/CI_CD.md From ece845e2c237f1a22be250397506825dada9a698 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 17:14:00 +0200 Subject: [PATCH 13/22] Parse mkdocs.yml in unsafe fashion due to python marker --- .pre-commit-config.yaml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index ecb56602..368571ce 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -8,6 +8,11 @@ repos: - id: check-ast - id: check-toml - id: check-yaml + exclude: mkdocs.yml + - id: check-yaml + name: check-yaml (mkdocs) + files: mkdocs.yml + args: ['--unsafe'] - id: check-added-large-files - id: check-builtin-literals From 4780d8876b914f72ed0d742709d5c87248b50967 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Thu, 9 Jul 2026 17:36:25 +0200 Subject: [PATCH 14/22] More notes --- docs/development/code.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/development/code.md b/docs/development/code.md index 93341d7b..e6258925 100644 --- a/docs/development/code.md +++ b/docs/development/code.md @@ -121,6 +121,30 @@ Additional messages may be added at your discretion, following these guidelines: During early development, the logging framework wasn't set up yet. Some endpoints may lack logging (other than the middleware). +## Routers + +We use FastAPI to create routers for specific categories of endpoints (e.g., `/datasets`) as shown in the code example in ["Anatomy of an Endpoint"](#anatomy-of-an-endpoint). +Additionally, we also make use of [Middleware](https://fastapi.tiangolo.com/reference/middleware/) functions which operate on each request for logging (see [Logging](#logging)) and [exception handlers](https://fastapi.tiangolo.com/tutorial/handling-errors/#install-custom-exception-handlers) for formatting errors. + +### Endpoint Paths + +The endpoints currently follow the pattern used the PHP-based REST API, but these paths don't always follow modern conventions. +For example, fetching a dataset's qualities requires a request to `datasets/qualities/{identifier}` even though having it available as `datasets/{identifier}/qualities` would be more conventional. + +### Errors + +Error responses of the REST API should follow the [RFC9457](https://www.rfc-editor.org/rfc/rfc9457.html) standard. +Exception handlers take care of the formatting automatically, so long as the exception you raise inherits from the `ProblemDetailError` class defined in `core/errors.py`. +Reference the class documentation for the attributes the errors should have. +The current error classes are largely based on the PHP errors, but we will likely consolidate these errors into broader categories eventually (e.g., there are currently multiple errors defined for users accessing data they are not allowed to see, one for a dataset, one for a study, ...). + +## Database + +Select the [database](database/index.md) pages from the sidebar for information on the database schema. + +### SQLAlchemy + + ### ORM usage The project currently uses a mix of ORM and SQL queries to interact with the databases. From 7ab45c695dff0cc11ae7fcda3804a6f13053e474 Mon Sep 17 00:00:00 2001 From: Pieter Gijsbers Date: Fri, 10 Jul 2026 11:52:57 +0200 Subject: [PATCH 15/22] Update docs/migration.md Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --- docs/migration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/migration.md b/docs/migration.md index 1db29620..b5325090 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -18,7 +18,7 @@ It is possible this migration guide is out of sync for endpoints not yet deploye Before an endpoint is deployed to production we will ensure that the documentation is up-to-date to the best of our knowledge. ## Standardized Errors -The Pyhon-based API will return different HTTP status codes depending on the error (for example, returning `404 NOT FOUND` if a requested dataset does not exist). +The Python-based API will return different HTTP status codes depending on the error (for example, returning `404 NOT FOUND` if a requested dataset does not exist). Moreover, the body of the response will contain a JSON body that adheres to the RFC9457 standard. ### Error Response Body From 8b0bc60a7e026a21584a153df6d61d878480cbab Mon Sep 17 00:00:00 2001 From: Pieter Gijsbers Date: Fri, 10 Jul 2026 11:53:12 +0200 Subject: [PATCH 16/22] Update docs/roadmap.md Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --- docs/roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index feb20aac..fad08eb9 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,7 +1,7 @@ The REST API is currently under development. It's does not yet have feature parity with the PHP-based REST API. We are currently working on Phase 1 and Phase 2 in parallel, and to deploy to production before the end of 2026. -The sunset date of the PHP-based REST API will be depend on the progress of this reimplementation and that of some other required server services. +The sunset date of the PHP-based REST API will depend on the progress of this reimplementation and that of some other required server services. ## Phase 1: Achieve Feature Parity From 15cbbfea9150142fcf177d766825317ddf6311e1 Mon Sep 17 00:00:00 2001 From: Pieter Gijsbers Date: Fri, 10 Jul 2026 11:53:39 +0200 Subject: [PATCH 17/22] fix typo Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --- docs/roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index fad08eb9..4b285836 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -31,5 +31,5 @@ Richer metadata for assets on OpenML (e.g., structured citation data), collabora ## Other Services Besides the REST API, there are a few other components which need to be developed: - - A logstash pipeline to populate ES indices based on the database data and/or a message queue. Currently, the PHP-based REST API directly puts documents into ES indices, which makes it prone to getting out of sync, and requires an ES server to run to use/test the API in ways which otherwise to not require an ES index. + - A logstash pipeline to populate ES indices based on the database data and/or a message queue. Currently, the PHP-based REST API directly puts documents into ES indices, which makes it prone to getting out of sync, and requires an ES server to run to use/test the API in ways which otherwise would not require an ES index. - Evaluation Engine. A service which computes meta-features for datasets, including the "qualities" and "features" you find with datasets. From 6d80cf4801e746e78d40544b0503a5cf5c082bd2 Mon Sep 17 00:00:00 2001 From: Pieter Gijsbers Date: Fri, 10 Jul 2026 11:55:59 +0200 Subject: [PATCH 18/22] fix typo Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --- docs/roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/roadmap.md b/docs/roadmap.md index 4b285836..3fabcb12 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,5 +1,5 @@ The REST API is currently under development. -It's does not yet have feature parity with the PHP-based REST API. +It does not yet have feature parity with the PHP-based REST API. We are currently working on Phase 1 and Phase 2 in parallel, and to deploy to production before the end of 2026. The sunset date of the PHP-based REST API will depend on the progress of this reimplementation and that of some other required server services. From 2470ed6e42c14246ddef41978e4c2189fd654ea6 Mon Sep 17 00:00:00 2001 From: Pieter Gijsbers Date: Fri, 10 Jul 2026 11:56:14 +0200 Subject: [PATCH 19/22] fix typo Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com> --- docs/development/project_overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/development/project_overview.md b/docs/development/project_overview.md index 2011e009..527223d5 100644 --- a/docs/development/project_overview.md +++ b/docs/development/project_overview.md @@ -12,7 +12,7 @@ as well as a rough structure of the codebase. The REST API should always be deployed with access to a database. -In production, these services are deployed in a kubernetes cluster. +In production, these services are deployed in a Kubernetes cluster. A typical request flow would then follow this pattern: ```mermaid From 33d17153f4ce6544811824269b64b6112a181058 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Fri, 10 Jul 2026 11:58:46 +0200 Subject: [PATCH 20/22] Move workflow to Zensical --- .github/workflows/docs.yml | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 7421deea..353026ef 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -1,27 +1,28 @@ -# https://squidfunk.github.io/mkdocs-material/publishing-your-site/?h=#with-github-actions -name: Deploy Docs +name: Documentation on: push: branches: - main - permissions: - contents: write - + contents: read + pages: write + id-token: write jobs: deploy: + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 - - uses: actions/setup-python@v4 + - uses: actions/configure-pages@v6 + - uses: actions/checkout@v7 + - uses: actions/setup-python@v6 with: python-version: 3.x - - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV - - uses: actions/cache@v3 + - run: pip install zensical + - run: zensical build --clean + - uses: actions/upload-pages-artifact@v5 with: - key: mkdocs-material-${{ env.cache_id }} - path: .cache - restore-keys: | - mkdocs-material- - - run: pip install mkdocs-material mkdocs-section-index - - run: mkdocs gh-deploy --force + path: site + - uses: actions/deploy-pages@v5 + id: deployment From e370d0dbdc9f60d9867213bcbc70c5bedc00b1f8 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Fri, 10 Jul 2026 12:04:45 +0200 Subject: [PATCH 21/22] Fix typos --- docs/contributing/index.md | 4 ++-- docs/development/documentation.md | 9 +++++++-- docs/development/setup.md | 4 ++-- docs/index.md | 4 ++-- 4 files changed, 13 insertions(+), 8 deletions(-) diff --git a/docs/contributing/index.md b/docs/contributing/index.md index 6329f0ac..60a88551 100644 --- a/docs/contributing/index.md +++ b/docs/contributing/index.md @@ -27,7 +27,7 @@ When you open a bug report, please include: For minor fixes such as correcting a typo or a broken link, it's fine to make the changes and submit them through a pull request. For larger changes, please make sure you are assigned to an issue first as is described -in the "[Code](#code)" section of this page. Then, visit the "[Developer Documentation](../development/project_overview.md)" +in the "[Code](#code)" section of this page. Then, visit the "[Developer Documentation](../development/documentation.md)" page to learn how to contribute documentation changes. ### Code @@ -49,7 +49,7 @@ please head over to the "[Setting up the development environment](../development While we are entirely grateful that so many people take an interest and want to help, the majority of the work is figuring out what the desired behavior and interface of the REST API should be. Making these decisions requires a lot of context about the OpenML platform, - ecosystem, and its history. For this reason, we restrict community contributions in this + ecosystem, and its history. For this reason, we restrict community code contributions in this project to issues that are marked "Good First Issue" or "Help Wanted" for now. diff --git a/docs/development/documentation.md b/docs/development/documentation.md index 85d1c1cb..bbda195d 100644 --- a/docs/development/documentation.md +++ b/docs/development/documentation.md @@ -1,4 +1,9 @@ -# Documentation with mkdocs-material +# Documentation with Zensical + +!!! tip "Small fixes" + + Small fixes to the documentation can easily be made from GitHub itself. + Navigate to the file, click the pencil icon in the top right, edit the document, and open a pull request. Our documentation is built using [Zensical](https://zensical.org/). It's made by the people that created Material for MkDocs, and is "backwards compatible" with that project (in fact, these docs were originally written for that framework). @@ -42,6 +47,6 @@ This will serve the documentation from the `docs/` directory to [http://localhos Any updates you make to files in that directory will be reflected on the website. When you are happy with your changes, just commit and set up a pull request! -!! tip "Can't connect to Docker?" +!!! tip "Can't connect to Docker?" If the docker container is running but you cannot access it, it's likely that you did not correctly specify the compose files to include `compose.ports.yaml`. diff --git a/docs/development/setup.md b/docs/development/setup.md index fcd4386e..c3f00987 100644 --- a/docs/development/setup.md +++ b/docs/development/setup.md @@ -18,11 +18,11 @@ python -m venv .venv source .venv/bin/activate ``` -We can use [`uv`](https://docs.astral.sh/uv/) to easily install the required dependencies from our [pyproject.toml](). +We can use [`uv`](https://docs.astral.sh/uv/) to easily install the required dependencies from our [pyproject.toml](https://github.com/openml/server-api/blob/main/pyproject.toml). ```bash python -m pip install uv -uv pip instal -e ".[dev]" +uv pip install -e ".[dev]" ``` ## Pre-commit diff --git a/docs/index.md b/docs/index.md index e3ccc19b..edf5fe0d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,12 +4,12 @@ This is the Python-based OpenML REST API server. It's a rewrite of our [old backend](http://github.com/openml/openml) built with a modern Python-based stack. -### What's changed? +## What's changed? For an overview about how the REST API is changing, visit the [migration guide](migration.md). -### Why a new REST API? +## Why a new REST API? The current production REST API has been with us for over a decade, and is still trotting along. However, it's built on a number of technologies which fell out of favor and some of its design decisions do not match current-day expectations. The current REST API would need significant refactoring to bring it up-to-date. From 6645b39ef8b811ddcd011d323f7f814eeb8d7a97 Mon Sep 17 00:00:00 2001 From: PGijsbers Date: Fri, 10 Jul 2026 12:08:52 +0200 Subject: [PATCH 22/22] Edits for clarity --- docs/migration.md | 2 +- mkdocs.yml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/migration.md b/docs/migration.md index b5325090..f2daa49c 100644 --- a/docs/migration.md +++ b/docs/migration.md @@ -36,7 +36,7 @@ curl -i https://www.openml.org/api/v1/json/data/1000000 + {"type":"https://openml.org/problems/dataset-not-found","title":"Dataset Not Found","status":404,"detail":"No dataset with id 100000 found.","code":"111"} ``` -As per the RFC9457 standard, the `type` field now includes the unique code for the error. +As per the RFC9457 standard, the `type` field now includes the unique identifier for the error in the form of a URI. You will notice that the response still contains a `code` of "111" (though as a top level property not embedded in the "error" scope). The `title` field is a human readable summary of the general issue and the `detail` field may provide additional information for the specific request. diff --git a/mkdocs.yml b/mkdocs.yml index d062fcd3..1026eaeb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -24,12 +24,12 @@ nav: - installation.md - Development: - Project Overview: development/project_overview.md - - Development: + - Guides: - "Getting Started": development/setup.md - "Writing Code": development/code.md - "Writing Tests": development/tests.md - "CI/CD": development/CI_CD.md - - Documentation: development/documentation.md + - "Documentation": development/documentation.md - Database: - development/database/index.md - openml: development/database/openml.md