Skip to content

Add an HTTP header analyzer, cross-tool hand-offs, and repeated-header handling - #129

Merged
JE-Chen merged 10 commits into
mainfrom
dev
Jul 25, 2026
Merged

Add an HTTP header analyzer, cross-tool hand-offs, and repeated-header handling#129
JE-Chen merged 10 commits into
mainfrom
dev

Conversation

@JE-Chen

@JE-Chen JE-Chen commented Jul 25, 2026

Copy link
Copy Markdown
Member

What changed

Three self-contained changes to the developer-tools suite.

Combine repeated curl -H headers instead of overwriting

A second -H with the same name replaced the first, silently dropping a value curl would have sent. Repeated headers are now joined the way the receiver joins field lines (", ", or "; " for cookies), and header names are matched case-insensitively throughout — so an explicit -H beats a default implied by --json, -A or -e regardless of its casing, instead of producing a second header line.

New tool: HTTP Header Analyzer

Paste a request or response header block and get every field back plus what is worth knowing about it:

  • names sent more than once (the ones HTTP expects to repeat, such as Set-Cookie, are counted but not flagged)
  • Set-Cookie entries missing Secure / HttpOnly / SameSite
  • a wildcard CORS policy, and the wildcard-with-credentials combination browsers reject outright
  • an HSTS max-age too short to matter, and CSP unsafe-inline / unsafe-eval
  • product banners, deprecated headers, textual media types without a charset
  • for responses, the security headers that are absent

Findings carry a stable code that the UI translates, so the analysis stays language-neutral, and headers that carry credentials are reported by name only — their values are never copied into the report. The response inspector hands its raw text over in one click, which keeps repeated headers intact.

Cross-tool hand-offs

The shared open-tool-tab logic moves into tool_tabs.open_tool_tab (the response inspector already had a private copy for its status/JWT hand-offs). The cURL importer gained a button that opens the parsed URL in the URL parser/builder, handing over the full URL so the address matches what curl would send even though the parser keeps the query in params.

Verification

  • ruff check . — clean
  • pytest test/test_utils — 759 passed (75 new tests, including a hypothesis property that analyze_headers never crashes and never emits an unrenderable finding)
  • test_language_parity.py — English/Chinese keys and placeholders in parity

JE-Chen added 9 commits July 24, 2026 18:25
Browser- and hand-written curl commands routinely glue short flags together
(-XPOST, -fsSL) or attach a value directly (-d'a=1', -H'Accept: ...',
-uuser:pass). These previously parsed wrong: -XPOST left the method as GET,
-sX POST made the URL "POST", and attached values were dropped.

Expand short-flag clusters into canonical tokens before consuming them: a
value-taking flag ends the cluster and takes any glued remainder as its value,
while an unknown short flag leaves the token untouched. Short-flag sets are
derived from the existing tables, and -S/--show-error, -q/--disable plus a few
value-taking flags (-C/-z/-D/-K) are recognised so common clusters resolve.
Also drop a pre-existing unused import from the test module.
--max-time / -m was consumed and discarded, so a generated script silently
dropped the original timeout. Capture the (numeric-validated) value on
CurlRequest and emit timeout=<seconds> on the requests.request call, so the
generated requests and pytest output honour the command's timeout. Non-numeric
values are still consumed but ignored, keeping the URL safe.
Adding the timeout branch lifted _apply_value_flag's cognitive complexity to
the point where SonarQube S3776 would trip. Move the -d @file vs inline body
decision into a small helper so the dispatch stays a flat, low-complexity chain.
Browser "copy as cURL" keeps the query in the URL (e.g. ?q=test&page=2), so
params stayed empty and the query was opaque. Parse it into params (URL-decoding
values, keeping blanks) without overwriting explicit -G/-d pairs.

Add CurlRequest.full_url to rebuild the original address from the base URL plus
params, and drive the LoadDensity template by it so query params - from the URL
or from -G, which that template previously dropped - survive. The requests,
pytest and APITestka templates already emit params, so they gain the URL query
automatically.
-b/--cookie 'a=1; b=2' was stored as one opaque Cookie header string. Parse the
name=value pairs into CurlRequest.cookies and emit an idiomatic, editable
cookies={...} in the requests, pytest and APITestka output; a bare token with no
'=' is a cookie file curl reads, so it still falls back to a Cookie header.
LoadDensity drives by URL and ignores cookies, as it does headers and bodies.
A companion to the query-string and cURL tools: paste a URL to see its scheme,
host, port, path, query and fragment (plus credentials) as an editable JSON
object, then turn an edited object back into a URL.

- url_tools/url_convert.py: pure-logic parse_url/url_to_json and
  build_url/json_to_url, with IPv6 bracketing, out-of-range-port tolerance and a
  stable parse->build round trip
- UrlBuilderGUI wired as both a tab and a dock, reusing OutputActions
- UrlConvertException plus English and Traditional Chinese strings (parity kept)
- logic and widget tests
A second -H with the same name replaced the first, silently dropping a
value curl would have sent. Repeated headers are now joined the way the
receiver joins field lines (", ", or "; " for cookies), and header names
are matched case-insensitively throughout, so an explicit -H beats a
default implied by --json, -A or -e regardless of its casing.
Paste a request or response header block and get every field back plus
what is worth knowing about it: names sent more than once, Set-Cookie
entries missing Secure/HttpOnly/SameSite, a wildcard CORS policy (and
the wildcard-with-credentials combination browsers reject), a short HSTS
max-age, CSP unsafe-inline/unsafe-eval, product banners, deprecated
headers, and the response security headers that are absent. Headers that
carry credentials are reported by name only, never by value.

Findings carry a stable code that the UI translates, so the analysis
stays language-neutral. The response inspector can hand its raw text
over in one click, which keeps repeated headers intact, and the shared
open-tool-tab helper it used for the status/JWT hand-offs now lives in
tool_tabs so other tools can reuse it. The header-line pattern moves to
the new module and the response analyzer imports it.
The importer already knows the request URL, so a button now opens it in
the URL parser/builder, parsed into its parts and ready to edit. The
full URL is handed over, query string included, so the address matches
what curl would send even though the parser keeps the query in params.

The builder gained an initial_url argument for that hand-off, matching
how the JWT decoder and status reference are pre-filled.
@codacy-production

codacy-production Bot commented Jul 25, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 150 complexity · 2 duplication

Metric Results
Complexity 150
Duplication 2

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

The value joiner returned an interpolated f-string, which Codacy's
Semgrep rule for Flask views returning formatted strings matches. No
part of this module serves HTTP, so the finding is a false positive,
but joining on the separator states the intent more plainly anyway.
@sonarqubecloud

Copy link
Copy Markdown

@JE-Chen
JE-Chen merged commit 981b3b3 into main Jul 25, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant