Websockets non ascii headers

[sallib]

Summary

Version 16.1 introduced a change that blocks non-ASCII values in Headers (changelog: "Blocked non-ASCII values in Headers", July 9, 2026 — https://websockets.readthedocs.io/en/stable/project/changelog.html#id2). This breaks legitimate, previously-working setups where header values legitimately contain UTF-8 characters, with no opt-out and (as far as I can tell) no mention in the migration guide.

Cross-posting this per a maintainer's suggestion on streamlit/streamlit#16030: "This looks like an intended change by websockets so there might be a valid reason to exclude non-ASCII header. It would be good if you report this to the python-websockets/websockets repo as well to get more clarity on why this was done." I fully expect there's a good reason for this — mainly looking for clarity on what it is, and whether there's a recommended migration path for the use case below.

Use case that broke

A common reverse-proxy authentication pattern (e.g. Authelia forward-auth behind nginx/Traefik) injects headers such as Remote-Name or Remote-User containing a user's display name before forwarding the request to the backend application. These names routinely contain non-ASCII characters (accents, non-Latin scripts, etc.).

This pattern is used by, at minimum, Streamlit, which depends on websockets for its WebSocket support. After upgrading to websockets 16.1, the WebSocket handshake fails outright for any user whose proxy-injected header value isn't pure ASCII, with no clear error surfaced to the end user (connection just fails).

Question / request

1. Is this restriction motivated purely by the HTTP spec (RFC 7230 requires header field values to be token / quoted-string in printable US-ASCII, technically excluding raw UTF-8), or is there also a security rationale (e.g. request smuggling, header injection)? The changelog entry doesn't explain the reasoning, which makes it hard to know whether this is a hard security boundary or a strictness/spec-compliance choice.

2. If it's spec-compliance rather than a hard security requirement, would you consider:

• documenting this as a breaking change rather than filing it under "Improvements" in the changelog (it silently breaks existing deployments on a pip install -U, since most consumers depend on websockets with a >= constraint), and/or
• providing an opt-in/opt-out flag for strict ASCII validation

3. If this is intentional and non-negotiable for spec-compliance/security reasons, could the migration guide / docs be updated with guidance for the reverse-proxy + non-ASCII-header case (e.g. recommend percent-encoding or base64-encoding such headers), since this seems like it will affect a fair number of users running behind auth proxies?

Environment

websockets version: 16.1
Python version: 3.12
Observed via: Streamlit (streamlit/streamlit), which pins websockets>=12.0.0,<17 and therefore can resolve to 16.1

Additional context

Happy to provide a minimal reproduction (a WebSocket handshake with a header containing e.g. "Maël") if useful.

[aaugustin]

The primary reason is security hardening. Secondary reasons include the spec requiring ASCII, the lack of an agreed-upon standard for decoding (every browser decodes with a different codec), and websockets < 16.1 decoding with a mechanism that users are very unlikely to understand and use correctly (surrogate escapes), that was never documented, and that's just a bad "smart" idea in the first place.

In this brave new world ✨ AI ✨, I'm getting a bunch of emails that claim security vulnerabilities. They're mostly bullshit because LLMs are very good at crafting vulnerable code when prompted to come up with vulnerabilities, and such code is extremely unlikely to happen in real life. But it's hard to argue confidently that we can accept anything in headers and it will never cause any problem. It's also boring to have to check for the unlikely case when one of these reports would actually be grounded.

Since the HTTP spec requires headers to be ASCII, it was an easy decision.

You claim that "previously-working setups where header values legitimately contain UTF-8 characters". Do you have evidence of this claim? websockets accepted those headers, but the way it decoded them was definitely not UTF-8, so I don't think it was actually working, at least not if you cared about those headers. That's the problem with non-ASCII headers in general!

---

I'm fine to document it as a breaking change, but that doesn't really help you.

Per the "be strict in what you send, liberal in what you accept" principle:

• I recommend that you file a bug against the software that generates non compliant headers. I'd recommend UTF-8 + percent escaping.
• There's an argument to be made that websockets should live with non-ASCII headers, especially if your app doesn't do anything with them, but the question is "how". The set of requirements is:
  • Must never crash. Any incoming data must be a valid input.
  • Must be reversible. Since there's no standard, we'll always get the decoding wrong from some users, so they must be able to undo then decode again with the right codec.
  • Must be understandable by the general public.
  • Must not attract the attention of AI thrown at open-source repositories to bug maintainers with hallucinated security vulnerabilities.

Maybe I should just go for latin-1 decoding, which I do for meaningless data like the HTTP reason phrase. It will be wrong in many cases — if something is non-ASCII, it's more likely to be UTF-8 — but it ticks all the boxes while UTF-8 fails the first criterion. The previous implementation, ascii with surrogate escapes, failed the third criterion.

[aaugustin]

Worth noting in this comment — uvicorn relies on the fact that websockets uses surrogateescapes and undoes it... so I broke it.

[aaugustin]

Wait - I double-checked your claims, and they're wrong.

websockets still accepts non-ASCII header values when it receives a request (or a response, but in your context it's a server, so you're only receiving requests). So it's liberal in what it accepts. It decodes non-ASCII values with latin-1 per section 5.5 of RFC 9110:

Historically, HTTP allowed field content with text in the ISO-8859-1 charset.

It doesn't let you set a non-ASCII header value in the response, and I stand by that decision. My approach to compliance has always been to prevent users from breaking the protocol. I've made implementation mistakes sometimes, but I never deviated from this principle.

I believe you when you say that you're encountering a problem, but that problem is not: websockets rejects non-ASCII headers.

It might be: websockets changed the undocumented technique for dealing with invalid headers without documenting the change — because I don't want to commit to supporting invalid inputs or to handling them in a particular way — this won't be a public API. And uvicorn depended on the previously undocumented technique.

[aaugustin]

The actual issue is Kludex/uvicorn#3035.

[aaugustin]

Reopening to consider improving the changelog (tomorrow).

[sallib]

Thanks a lot for digging into this so thoroughly and for the detailed explanation, especially for catching that the actual regression is on the uvicorn side rather than in websockets' handling of non-ASCII headers itself. That clears things up considerably.

I'll follow up on Kludex/uvicorn#3035 instead. Thanks again for the quick and thorough response!

[aaugustin]

On the changelog --

The actual change in websockets 16.1 is to reject values containing newlines and other significant characters. Non-ASCII data is still allowed, unlike what I believed yesterday.

That being said, there are inconsistencies worth fixing in the handling of non-ASCII data in HTTP messages, and I'm going to do this now.

[aaugustin]

2503d10 cleans up everything. phew.