-
Notifications
You must be signed in to change notification settings - Fork 2.2k
Python: fix: preserve URL query parameters in DefaultHttpRequestHandler #7765
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
0edb412
9a8b705
6d3d312
d81257a
2c0eb5e
864932e
016397d
bd7e18e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,6 +24,7 @@ | |
| from collections.abc import Awaitable, Callable, Mapping | ||
| from dataclasses import dataclass, field | ||
| from typing import Any, Protocol, runtime_checkable | ||
| from urllib.parse import urlsplit, urlunsplit | ||
|
|
||
| import httpx | ||
|
|
||
|
|
@@ -173,12 +174,47 @@ async def send(self, info: HttpRequestInfo) -> HttpRequestResult: | |
| # callers (not just the YAML executor) get sensible defaults. | ||
| headers["Content-Type"] = info.body_content_type or "text/plain" | ||
|
|
||
| params: Mapping[str, str] | None = info.query_parameters or None | ||
| # Merge all three query sources into one ``params=`` list. Passing only | ||
| # ``query_parameters`` into ``params=`` (the pre-#7765 bug) drops URL-embedded | ||
| # params, and passing nothing (#7765 first iteration) lets the *client*' | ||
| # ``AsyncClient.params`` (via ``Client._merge_queryparams``) drop the URL's | ||
| # query. Collecting client params + URL params + query_parameters into one | ||
| # explicit list preserves every source. Precedence is client (lowest) < URL | ||
| # query < ``query_parameters`` (highest): on a duplicate key the later source' | ||
| # value replaces the earlier one (dict-of-lists assignment), matching the old | ||
| # ``_merge_queryparams.merge(params)`` per-request-wins behavior. urlsplit/ | ||
| # urlunsplit keeps the final query before any ``#fragment``. | ||
| url = info.url | ||
| raw = urlsplit(url) | ||
| merged: dict[str, list[str]] = {} | ||
| overridden: set[str] = set() | ||
|
|
||
| def _merge_layer(pairs: list[tuple[str, str]]) -> None: | ||
| seen: set[str] = set() | ||
| for key, value in pairs: | ||
| if key in overridden and key not in seen: | ||
| # First occurrence in this layer overrides the lower-precedence value. | ||
| merged[key] = [value] | ||
| else: | ||
| merged.setdefault(key, []).append(value) | ||
| seen.add(key) | ||
|
|
||
| _merge_layer(list(client.params.multi_items())) | ||
| overridden.update(merged) | ||
| _merge_layer(list(httpx.URL(url).params.multi_items())) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Following up on the earlier report: this commit preserves client defaults, but could we avoid round-tripping the URL's existing query through
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Both concerns confirmed against httpx 0.28.1 — and they're coupled, so wanted to put the options in front of you before writing code: What I verified
Fix shapes I can see
Tradeoffs
Which do you want? My lean is (c), falling back to (b1) if you'd rather keep it simple and accept the small behaviour drift for client-configured headers/cookies/auth. (Also flagging: when |
||
| overridden.update(merged) | ||
| _merge_layer([(key, value) for key, value in info.query_parameters.items() if key]) | ||
|
|
||
| params: list[tuple[str, Any]] = [(k, v) for k, values in merged.items() for v in values] | ||
| if params: | ||
| # urlunsplit with an empty query part strips the raw query so it is not | ||
| # double-applied; ``params=`` now carries all three sources. | ||
| url = urlunsplit((raw.scheme, raw.netloc, raw.path, "", raw.fragment)) | ||
|
|
||
| response = await client.request( | ||
| method=info.method, | ||
| url=info.url, | ||
| params=params, | ||
| url=url, | ||
|
manjunathshiva marked this conversation as resolved.
|
||
| params=params or None, | ||
| headers=headers or None, | ||
| content=content, | ||
| timeout=timeout, | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens when the same key is already present in the URL? At
_http_handler.py:195, the first explicit occurrence resets the list, so...?filter=region&filter=statusplusquery_parameters={"filter": "tenant"}sends onlyfilter=tenant. The issue and .NET implementation preserve the existing values and append the new pair. Could the client-default override happen separately, then treat the URL andquery_parametersas one request-level list?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depends on which fix shape you pick on the other thread (verbatim-URL reply directly above — the one about
?term=a%20b round-tripping through QueryParams). In (a) I can switch query_parameters to append-only to match .NET exactly (client params still override URL keys when they collide, per the old _merge_queryparams behaviour). In (b) / (c) the URL stays byte-identical and query_parameters lands as a separate appended delta anyway, so duplicate keys just work. Happy to proceed once the main choice lands.