It would be nice if we could make a more granular expected-failures than at the scenario level.
E.g. stateless-server has ~25-30 different actual tests. As a result of a recent protocol merge we have to update the SDK in advance of the conformance tests: modelcontextprotocol/modelcontextprotocol#3002
to fix in the SDK we had to bring back an expected failure: modelcontextprotocol/typescript-sdk#2513
In reality this only invalidates 1 or 2 of the ~30 checks, but the granularity where you can mark "expected failures" is only scenario level.
It would be nice if we could mark expected failures as specific checks - something like
- stateless-server:header-is-valid-check
For example.
https://github.com/modelcontextprotocol/typescript-sdk/pull/2513/changes#diff-030af213028e5c06b2fd599c5604fa50d0ff4d281c26f77bdc124bc126fe8222R37
It would be nice if we could make a more granular expected-failures than at the scenario level.
E.g.
stateless-serverhas ~25-30 different actual tests. As a result of a recent protocol merge we have to update the SDK in advance of the conformance tests: modelcontextprotocol/modelcontextprotocol#3002to fix in the SDK we had to bring back an expected failure: modelcontextprotocol/typescript-sdk#2513
In reality this only invalidates 1 or 2 of the ~30 checks, but the granularity where you can mark "expected failures" is only scenario level.
It would be nice if we could mark expected failures as specific checks - something like
For example.
https://github.com/modelcontextprotocol/typescript-sdk/pull/2513/changes#diff-030af213028e5c06b2fd599c5604fa50d0ff4d281c26f77bdc124bc126fe8222R37