-
-
Notifications
You must be signed in to change notification settings - Fork 50
Add GET /run/trace/{run_id} endpoint #272
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
PGijsbers
merged 8 commits into
openml:main
from
saathviksheerla:feat/get-run-trace-endpoint
Mar 24, 2026
Merged
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
dd83fea
Add GET /runs/trace/{run_id} endpoint
saathviksheerla 966ba52
Address bot review feedback
saathviksheerla 9376c1c
Fix endpoint prefix from /runs to /run to match spec
saathviksheerla 0fb3dda
Rename get to exist, add migration test
saathviksheerla 247408a
Sort trace iterations before comparison in migration test
saathviksheerla a7eabfb
Cover more cases
PGijsbers 5cc7113
Make explicit error comparison per status code, not found always 404
PGijsbers 89797cd
Reflect changes in error response
PGijsbers File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| """Database queries for run-related data.""" | ||
|
|
||
| from collections.abc import Sequence | ||
| from typing import cast | ||
|
|
||
| from sqlalchemy import Row, text | ||
| from sqlalchemy.ext.asyncio import AsyncConnection | ||
|
|
||
|
|
||
| async def get(id_: int, expdb: AsyncConnection) -> Row | None: | ||
| """Check if a run exists by ID.""" | ||
| row = await expdb.execute( | ||
| text( | ||
| """ | ||
| SELECT 1 | ||
| FROM `run` | ||
| WHERE `rid` = :run_id | ||
| """, | ||
| ), | ||
| parameters={"run_id": id_}, | ||
| ) | ||
| return row.one_or_none() | ||
|
|
||
|
|
||
| async def get_trace(run_id: int, expdb: AsyncConnection) -> Sequence[Row]: | ||
| """Get trace rows for a run from the trace table.""" | ||
| rows = await expdb.execute( | ||
| text( | ||
| """ | ||
| SELECT `repeat`, `fold`, `iteration`, `setup_string`, `evaluation`, `selected` | ||
| FROM `trace` | ||
| WHERE `run_id` = :run_id | ||
| """, | ||
| ), | ||
| parameters={"run_id": run_id}, | ||
| ) | ||
| return cast( | ||
| "Sequence[Row]", | ||
| rows.all(), | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| """Endpoints for run-related data.""" | ||
|
|
||
| from typing import Annotated | ||
|
|
||
| from fastapi import APIRouter, Depends | ||
| from sqlalchemy.ext.asyncio import AsyncConnection | ||
|
|
||
| import database.runs | ||
| from core.errors import RunNotFoundError, RunTraceNotFoundError | ||
| from routers.dependencies import expdb_connection | ||
| from schemas.runs import RunTrace, TraceIteration | ||
|
|
||
| router = APIRouter(prefix="/runs", tags=["runs"]) | ||
|
|
||
|
|
||
| @router.get("/trace/{run_id}") | ||
| async def get_run_trace( | ||
| run_id: int, | ||
| expdb: Annotated[AsyncConnection, Depends(expdb_connection)], | ||
| ) -> RunTrace: | ||
| """Get trace data for a run by run ID.""" | ||
| if not await database.runs.get(run_id, expdb): | ||
| msg = f"Run {run_id} not found." | ||
| raise RunNotFoundError(msg) | ||
|
|
||
| trace_rows = await database.runs.get_trace(run_id, expdb) | ||
| if not trace_rows: | ||
| msg = f"No trace found for run {run_id}." | ||
| raise RunTraceNotFoundError(msg) | ||
|
|
||
| return RunTrace( | ||
| run_id=run_id, | ||
| trace=[ | ||
| TraceIteration( | ||
| repeat=row.repeat, | ||
| fold=row.fold, | ||
| iteration=row.iteration, | ||
| setup_string=row.setup_string, | ||
| evaluation=row.evaluation, | ||
| selected=row.selected, | ||
| ) | ||
| for row in trace_rows | ||
| ], | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| """Pydantic schemas for run-related endpoints.""" | ||
|
|
||
| from pydantic import BaseModel | ||
|
|
||
|
|
||
| class TraceIteration(BaseModel): | ||
| """A single trace iteration for a run.""" | ||
|
|
||
| repeat: int | ||
| fold: int | ||
| iteration: int | ||
| setup_string: str | None | ||
| evaluation: float | None | ||
| selected: str | ||
|
|
||
|
|
||
| class RunTrace(BaseModel): | ||
| """Trace data for a run.""" | ||
|
|
||
| run_id: int | ||
| trace: list[TraceIteration] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| """Tests for the GET /runs/trace/{run_id} endpoint.""" | ||
|
|
||
| from http import HTTPStatus | ||
|
|
||
| import httpx | ||
| import pytest | ||
|
|
||
| from core.errors import RunNotFoundError, RunTraceNotFoundError | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("run_id", [34]) | ||
| async def test_get_run_trace_success(run_id: int, py_api: httpx.AsyncClient) -> None: | ||
| """Test that trace data is returned for a run that has trace entries.""" | ||
| response = await py_api.get(f"/runs/trace/{run_id}") | ||
| assert response.status_code == HTTPStatus.OK | ||
| body = response.json() | ||
| assert body["run_id"] == run_id | ||
| assert isinstance(body["trace"], list) | ||
| assert len(body["trace"]) > 0 | ||
| first = body["trace"][0] | ||
| assert isinstance(first["repeat"], int) | ||
| assert isinstance(first["fold"], int) | ||
| assert isinstance(first["iteration"], int) | ||
| assert first["selected"] in ("true", "false") | ||
| assert first["evaluation"] is None or isinstance(first["evaluation"], float) | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("run_id", [24]) | ||
| async def test_get_run_trace_no_trace(run_id: int, py_api: httpx.AsyncClient) -> None: | ||
| """Test that 412 is returned for a run that exists but has no trace.""" | ||
| response = await py_api.get(f"/runs/trace/{run_id}") | ||
| assert response.status_code == HTTPStatus.PRECONDITION_FAILED | ||
| body = response.json() | ||
| assert body["code"] == "572" # RunTraceNotFoundError code | ||
| assert body["type"] == RunTraceNotFoundError.uri | ||
| assert body["title"] == RunTraceNotFoundError.title | ||
| assert body["status"] == HTTPStatus.PRECONDITION_FAILED | ||
|
|
||
|
|
||
| @pytest.mark.parametrize("run_id", [999999]) | ||
| async def test_get_run_trace_run_not_found(run_id: int, py_api: httpx.AsyncClient) -> None: | ||
| """Test that 412 is returned when the run does not exist.""" | ||
| response = await py_api.get(f"/runs/trace/{run_id}") | ||
| assert response.status_code == HTTPStatus.PRECONDITION_FAILED | ||
| body = response.json() | ||
| assert body["code"] == "571" # RunNotFoundError code | ||
| assert body["type"] == RunNotFoundError.uri | ||
| assert body["title"] == RunNotFoundError.title | ||
| assert body["status"] == HTTPStatus.PRECONDITION_FAILED |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.