tests: add API spec test suite #346
Merged
+411
−134
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗒️ Description
Currently the API specs rely on implementers reading the python implementation and opiniately decide what is the spec and what is the leanSpec's own implementation flavor.
This PR creates a new test suite specifically for the API specs so that API conformance test can be run against a target server by running
uv run apitest <server_url>:<server_port>. Implementers can see exactly what is expected by reading & running the tests, e.g. what url structure, what Accept headers, what should be encoded, etc. This way we can reduce the tiring chats and use these tests as reference for discussion and implementation instead.Also as part of the standard
uv run pytest, it will spawn its own server from leanSpec implementation and run it against these same API tests so these API tests that can be used with client implementations are guaranteed to be conformant with the spec (similar to howuv run fill ...also tests against the spec before producing the test vectors).Example
Run
ream(I'll try with other clients as well but want to get the base proposal reviewed & merged first) in ephemeral mode with:cargo run --release -- --ephemeral lean_node \ --network ephemery \ --validator-registry-path ./bin/ream/assets/lean/validators.yamlThen run the tests in leanSpec with:
Results:
🔗 Related Issues or PRs
N/A
✅ Checklist
toxchecks to avoid unnecessary CI fails:uvx tox