guard json schema ref retrieval against internal targets

[uwezkhan]

The JSON deserializer resolves a $ref the schema registry doesn't know about by handing the raw URI to httpx.get in _retrieve_via_httpx. The writer schema is selected by the schema id embedded in the consumed message, so a producer can register a schema whose $ref points at http://169.254.169.254/..., loopback, or an RFC1918 host, and the consumer fetches it during deserialization and parses the response as a schema.

Before, any scheme and any address were fetched. After, the helper requires http/https, resolves the host, and refuses private, loopback, link-local, reserved, multicast, or unspecified targets (including IPv4-mapped IPv6); public URLs still resolve as they did. The check lives in the retrieve callback because that is the single point every $ref lookup passes through, so the sync and async paths are both covered without each caller repeating it. Tradeoff: a schema legitimately served from an internal host is now rejected and has to be reachable at a public address or registered as a named reference.

[confluent-cla-assistant]

🎉 All Contributor License Agreements have been signed. Ready to merge.
✅ uwezkhan
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

[Robert Yokota (rayokota)]

/sem-approve

[Copilot]

Pull request overview

This PR hardens JSON Schema $ref retrieval during deserialization by adding an allowlist-style network guard in the shared external-retrieve callback, preventing SSRF-style fetches to internal/non-routable targets while preserving public URL retrieval.

Changes:

• Add _guard_external_uri() and IP classification helpers to block non-HTTP(S) schemes and non-public resolved targets before fetching.
• Update _retrieve_via_httpx() to apply the guard and explicitly disable redirect following.
• Add tests validating that internal/non-HTTP(S) targets are rejected without issuing an HTTP request, and that a public-resolving host is allowed.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
src/confluent_kafka/schema_registry/common/json_schema.py	Adds URI scheme/host validation and IP-range blocking before performing external $ref retrieval.
tests/schema_registry/test_json_schema_retrieve.py	Adds regression tests for allowed/blocked $ref retrieval behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[uwezkhan]

any update ?

[uwezkhan]

Rebased on master and force-pushed. The last commit had a malformed committer email, which is what kept the CLA check stuck; it passes now. The rebase also restored the urlsplit import that the earlier merge of master dropped, so the guard actually runs again. No other changes, flake8 and the retrieve tests pass locally.

[Robert Yokota (rayokota)]

/sem-approve

[uwezkhan]

The Semaphore failure was the style-format check: black wants the parametrize list in the new test file wrapped differently. Ran tools/style-format.sh --fix on it and pushed. Formatting only, no behavior change; flake8, mypy and the retrieve tests all pass locally. It will need another /sem-approve to rerun CI.