Feature Description
Twice now a release has shipped schemas some client rejects: the test-runner env parameter schema that didn't work with OpenAI, and later the registration schema whose missing definitions broke every structured tool until they got inlined. Different bugs, same class: the declared tool surface changed in a way no test observed, and a client found it first.
I maintain an open-source tool called reelier (github.com/seldonframe/reelier) that puts that surface under test: its manifest command snapshots every tool name and JSON schema, diffs the snapshot in CI, and fails closed on drift,
with strict-client validation runnable against the snapshot in one place.
Coming from the org that built error tracking, the framing might resonate: it's observability for the contract the server exposes to agents.
Happy to open a PR so evaluating it costs one review. Feel free to close if it's not a fit, no follow-up from me.
Use Cases
No response
Example Interactions
No response
Feature Description
Twice now a release has shipped schemas some client rejects: the test-runner env parameter schema that didn't work with OpenAI, and later the registration schema whose missing definitions broke every structured tool until they got inlined. Different bugs, same class: the declared tool surface changed in a way no test observed, and a client found it first.
I maintain an open-source tool called reelier (github.com/seldonframe/reelier) that puts that surface under test: its manifest command snapshots every tool name and JSON schema, diffs the snapshot in CI, and fails closed on drift,
with strict-client validation runnable against the snapshot in one place.
Coming from the org that built error tracking, the framing might resonate: it's observability for the contract the server exposes to agents.
Happy to open a PR so evaluating it costs one review. Feel free to close if it's not a fit, no follow-up from me.
Use Cases
No response
Example Interactions
No response