-
Notifications
You must be signed in to change notification settings - Fork 1.8k
fix: UriTemplate.match() handles optional, out-of-order, and encoded query parameters #1785
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
7775907
3634992
6fcd8c4
7e4fd95
fde6754
01bcc47
79d782e
a3cc896
7e25ee3
fb65293
2fc117c
37d2534
5e07261
8a9aefb
a911de3
585c5f7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| '@modelcontextprotocol/core': patch | ||
| --- | ||
|
|
||
| Fix `UriTemplate.match()` to correctly handle optional, out-of-order, and URL-encoded query parameters. Previously, query parameters had to appear in the exact order specified in the template and omitted parameters would cause match failures. Omitted query parameters are now absent from the result (rather than set to `''`), so callers can use `vars.param ?? defaultValue`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,14 @@ | |
| const MAX_TEMPLATE_EXPRESSIONS = 10_000; | ||
| const MAX_REGEX_LENGTH = 1_000_000; // 1MB | ||
|
|
||
| function safeDecode(s: string): string { | ||
| try { | ||
| return decodeURIComponent(s); | ||
| } catch { | ||
| return s; | ||
| } | ||
| } | ||
|
|
||
| export class UriTemplate { | ||
| /** | ||
| * Returns true if the given string contains any URI template expressions. | ||
|
|
@@ -97,7 +105,7 @@ | |
| return expr | ||
| .slice(operator.length) | ||
| .split(',') | ||
| .map(name => name.replace('*', '').trim()) | ||
| .map(name => name.replaceAll('*', '').trim()) | ||
| .filter(name => name.length > 0); | ||
| } | ||
|
|
||
|
|
@@ -254,12 +262,19 @@ | |
|
|
||
| match(uri: string): Variables | null { | ||
| UriTemplate.validateLength(uri, MAX_TEMPLATE_LENGTH, 'URI'); | ||
|
|
||
| // Build regex pattern for path (non-query) parts | ||
| let pattern = '^'; | ||
| const names: Array<{ name: string; exploded: boolean }> = []; | ||
| const queryParts: Array<{ name: string; exploded: boolean }> = []; | ||
|
|
||
| for (const part of this.parts) { | ||
| if (typeof part === 'string') { | ||
| pattern += this.escapeRegExp(part); | ||
| } else if (part.operator === '?' || part.operator === '&') { | ||
| for (const name of part.names) { | ||
| queryParts.push({ name, exploded: part.exploded }); | ||
| } | ||
| } else { | ||
| const patterns = this.partToRegExp(part); | ||
| for (const { pattern: partPattern, name } of patterns) { | ||
|
|
@@ -269,22 +284,63 @@ | |
| } | ||
| } | ||
|
|
||
| // Only strip the URI fragment when the template has no {#var} operator; | ||
| // otherwise the fragment is part of what the path regex must capture. | ||
| const hasHashOperator = this.parts.some(p => typeof p !== 'string' && p.operator === '#'); | ||
| let working = uri; | ||
| if (!hasHashOperator) { | ||
| const hashIndex = working.indexOf('#'); | ||
| if (hashIndex !== -1) working = working.slice(0, hashIndex); | ||
| } | ||
|
Check failure on line 294 in packages/core/src/shared/uriTemplate.ts
|
||
|
|
||
| // Only split path/query when the template actually has {?..}/{&..} | ||
| // operators. Otherwise match the path regex against the full URI so | ||
| // {+var} can capture across '?' as it did before query-param support. | ||
| let pathPart = working; | ||
| let queryPart = ''; | ||
| if (queryParts.length > 0) { | ||
| const queryIndex = working.indexOf('?'); | ||
| if (queryIndex !== -1) { | ||
| pathPart = working.slice(0, queryIndex); | ||
| queryPart = working.slice(queryIndex + 1); | ||
| } | ||
| } | ||
|
Check failure on line 307 in packages/core/src/shared/uriTemplate.ts
|
||
|
Comment on lines
+299
to
+307
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 The new Extended reasoning...What the bug is and how it manifests The new The specific code path that triggers it In Why existing code doesn't prevent it Before this PR, What the impact would be Round-trip fidelity breaks: How to fix it Only route Step-by-step proof
Comment on lines
+287
to
+307
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 When a template uses both {?param} and {#var} operators together (e.g., /path{?a}{#b}), match() always returns null for valid URIs like /path?a=1#section. The two new conditional preprocessing steps interact incorrectly: hasHashOperator=true prevents fragment stripping, but queryParts.length>0 then splits at '?', placing the fragment into queryPart ('a=1#section') and leaving pathPart='/path' which cannot satisfy the (.+) capture pattern that {#b} adds to the path regex. Additionally, even if matching succeeded, the fragment would leak into the query value for 'a'. Extended reasoning...What the bug is and how it manifests When a template contains both a {?...} query operator and a {#var} fragment operator (e.g., /path{?a}{#b}), the match() method unconditionally returns null for any valid matching URI. This is caused by two new conditional preprocessing steps introduced by this PR that were designed independently but fail when combined. The specific code path that triggers it For template /path{?a}{#b} and URI /path?a=1#section:
Result: null. Expected: {a: '1', b: '#section'}. Why existing code does not prevent it The hasHashOperator check was added to preserve the fragment for {#} capture; the queryParts.length > 0 check was added to split path from query for {?} parsing. Each works correctly in isolation. But when both operators appear in the same template, they interact destructively: fragment preservation keeps the fragment in working, but the query split then moves that fragment into queryPart rather than leaving it available for path regex capture by the {#} pattern. Impact Any URI template combining {?...} and {#...} operators always returns null from match(), even for perfectly valid matching URIs. Additionally, even if the path match were somehow forced to succeed, queryPart = a=1#section would assign value 1#section to variable a — meaning the fragment leaks into the query value. How to fix When both hasHashOperator and queryParts.length > 0 are true, the fragment must be separated before the ? split. One approach: always strip the fragment from working before splitting at ?, record the fragment separately, then pass it back to the path match when hasHashOperator is true (appending it to pathPart so the {#} pattern can capture it). Step-by-step proof
|
||
|
|
||
| pattern += '$'; | ||
| UriTemplate.validateLength(pattern, MAX_REGEX_LENGTH, 'Generated regex pattern'); | ||
| const regex = new RegExp(pattern); | ||
| const match = uri.match(regex); | ||
|
|
||
| const match = pathPart.match(regex); | ||
| if (!match) return null; | ||
|
|
||
| const result: Variables = {}; | ||
| for (const [i, name_] of names.entries()) { | ||
| const { name, exploded } = name_!; | ||
| const value = match[i + 1]!; | ||
| const cleanName = name.replace('*', ''); | ||
|
|
||
| for (const [i, { name, exploded }] of names.entries()) { | ||
| const value = match[i + 1]!; | ||
| const cleanName = name.replaceAll('*', ''); | ||
| result[cleanName] = exploded && value.includes(',') ? value.split(',') : value; | ||
| } | ||
|
Comment on lines
+317
to
321
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Path variable captures in match() are returned as raw percent-encoded strings (lines 317-321), while query parameters are decoded via safeDecode() (lines 329-330) — an asymmetry newly introduced by this PR. This breaks the round-trip fidelity goal the PR describes: expand({ q: 'hello world' }) for {+q} yields /hello%20world, but match('/hello%20world') returns { q: 'hello%20world' }, so a second expand() double-encodes to /hello%2520world. The fix is to apply safeDecode to path variable captures in the names loop at lines 317-321, mirroring the query param handling. Extended reasoning...What the bug isThis PR adds safeDecode() calls when parsing query parameters (lines 329-330) so that {?q} matching ?q=hello%20world returns { q: 'hello world' }. However, path variable captures at lines 317-321 are returned without any decoding — the raw regex capture group value is stored directly in result[cleanName]. This creates a new inconsistency between the two handling paths. The specific code pathIn match(), path variable captures go through the names loop with no decoding applied. Query params go through safeDecode() at lines 329-330 before being stored. The path variable loop has no equivalent decoding step. Why existing code does not prevent itBefore this PR, neither path variables nor query params were decoded — the behavior was symmetrically raw. This PR adds decoding only for query params, introducing a split where one code path decodes and the other does not. The path variable loop at lines 317-321 was not updated to match. Impact and round-trip failureThe PR description explicitly advertises URL-encoded values are decoded and round-trip fidelity (expand(match(uri)) === uri) as goals. These goals hold for query params but fail for path operators:
This also affects the simple {var} operator: template /users/{id} matching /users/john%20doe returns { id: 'john%20doe' }, while {?id} matching ?id=john%20doe returns { id: 'john doe' }. Addressing the refutationOne verifier argues this is pre-existing because path variable non-decoding predates the PR and lines 317-321 are unchanged. This is technically accurate, but the PR creates a new user-visible inconsistency: prior to this PR, both operator types behaved the same (both raw). After this PR, the behavior diverges. The PR's own stated goals imply users should expect decoding to work uniformly, making this worth flagging as part of this PR rather than as a separate issue. FixApply safeDecode to path variable values in the names loop at lines 317-321, similar to the query param handling at lines 329-330. This mirrors the query param decoding and satisfies the round-trip guarantee for all operator types.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Path-variable decoding is pre-existing behavior (base code never decoded either) and out of scope for this query-param fix. Will track applying safeDecode() to path captures separately. |
||
|
|
||
| if (queryParts.length > 0) { | ||
| const queryParams = new Map<string, string>(); | ||
| if (queryPart) { | ||
| for (const pair of queryPart.split('&')) { | ||
| const equalIndex = pair.indexOf('='); | ||
| if (equalIndex !== -1) { | ||
| const key = safeDecode(pair.slice(0, equalIndex)); | ||
| const value = safeDecode(pair.slice(equalIndex + 1)); | ||
| queryParams.set(key, value); | ||
claude[bot] marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| } | ||
| } | ||
| } | ||
|
|
||
| for (const { name, exploded } of queryParts) { | ||
| const cleanName = name.replaceAll('*', ''); | ||
| const value = queryParams.get(cleanName); | ||
| if (value === undefined) continue; | ||
| result[cleanName] = exploded && value.includes(',') ? value.split(',') : value; | ||
| } | ||
|
Check failure on line 341 in packages/core/src/shared/uriTemplate.ts
|
||
|
Comment on lines
+336
to
+341
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 The exploded query param handler decodes values via Extended reasoning...What the bug is and how it manifests In The specific code path
Why existing code does not prevent it Prior to this PR, neither path variable captures nor query parameters were decoded — both returned raw encoded strings. The comma-split check was always performed on the raw encoded string, where What the impact would be Any How to fix it Split on literal commas in the raw encoded value first, then decode each element individually: // Instead of:
const value = safeDecode(pair.slice(equalIndex + 1));
queryParams.set(key, value);
// ...later...
result[cleanName] = exploded && value.includes(',') ? value.split(',') : value;
// Do:
const rawValue = pair.slice(equalIndex + 1);
queryParams.set(key, rawValue); // store raw, not decoded
// ...later...
if (exploded && rawValue.includes(',')) {
result[cleanName] = rawValue.split(',').map(safeDecode);
} else {
result[cleanName] = safeDecode(rawValue);
}Step-by-step proof
|
||
| } | ||
|
|
||
| return result; | ||
| } | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔴 The fragment-stripping guard at line 289 only checks
p.operator === '#', but the{+var}(reserved expansion) operator also uses(.+)inpartToRegExpand is RFC 6570-defined to pass reserved characters including#— so'http://e.com{+rest}'.match('http://e.com/path#section')returns{ rest: '/path' }instead of{ rest: '/path#section' }. The fix is to extendhasHashOperatorto also include templates with+operators:this.parts.some(p => typeof p \!== 'string' && (p.operator === '#' || p.operator === '+')).Extended reasoning...
What the bug is
The
match()method introduced in this PR adds a pre-processing step that strips the URI fragment before regex matching (lines 287–294). The guard conditionhasHashOperatoris supposed to skip this stripping when a{#var}operator is present, because the fragment is part of what the path regex must capture. However, the{+var}(reserved expansion) operator also generates the pattern(.+)inpartToRegExp(viacase '+': case '#': pattern = '(.+)') and is RFC 6570-defined to pass reserved characters through unencoded — including#. The check only looks atp.operator === '#'and never considersp.operator === '+'.The specific code path
For template
'http://e.com{+rest}'and URI'http://e.com/path#section':hasHashOperator = this.parts.some(p => typeof p \!== 'string' && p.operator === '#')→ false (only+operator present)hashIndexfinds#, soworking = 'http://e.com/path'queryParts.length === 0(no?/&operators) → no query split;pathPart = 'http://e.com/path'^http://e\.com(.+)$is matched against'http://e.com/path'→ captures'/path'{ rest: '/path' }— fragment silently droppedWhy existing code doesn't prevent it
Both
+and#operators generate identical(.+)patterns inpartToRegExp. ThehasHashOperatorguard was added to protect#captures from fragment stripping, but the symmetric case for+was overlooked. The check is simply incomplete.Impact
This is a regression introduced by this PR. Before the PR,
match()applied the regex to the full URI string, so(.+)correctly captured'/path#section'. After the PR, the fragment is stripped first, silently breaking round-trip fidelity for{+var}templates. The round-trip guarantee advertised by the PR (expand(match(uri)) === uri) fails:expand({ rest: '/path#section' })yields'http://e.com/path#section', butmatch('http://e.com/path#section')returns{ rest: '/path' }, so a second expand produces'http://e.com/path'.How to fix
Extend
hasHashOperatorto also cover the+operator:Step-by-step proof
'http://e.com{+rest}', parsed parts:['http://e.com', { operator: '+', names: ['rest'], ... }]hasHashOperator=false(the part's operator is'+', not'#')hashIndex = uri.indexOf('#') = 20→working = 'http://e.com/path'queryParts.length = 0→pathPart = 'http://e.com/path'/^http:\/\/e\.com(.+)$/matches'http://e.com/path'→match[1] = '/path'result = { rest: '/path' }—'#section'is permanently lost