[P1][Correctness] Replace regex-based cURL parsing with a quote-aware tokenizer

[baixiangcpp]

Summary

Curl to Code currently parses cURL commands with independent regular expressions. This silently truncates or drops common, valid cURL syntax while still producing generated code.

A representative JSON request is already misparsed:

curl https://api.example.com/users \
  -H 'Content-Type: application/json' \
  -d '{"name":"Ada"}'

The current dataMatch expression treats the first inner double quote as the end of the single-quoted argument, so parseCurl() records the body as only { instead of the complete JSON payload. The generated request then sends corrupted data without reporting a parse error.

Current behavior

src/features/tools/curl-to-code/logic.ts uses regular expressions that do not share shell quote state:

const dataMatch = rest.match(/(?:-d|--data|--data-raw|--data-binary)\s+['"]([^'"]*)['"]/i)

Because the character class excludes both quote characters, a single-quoted JSON string containing normal JSON double quotes is truncated.

Other common cases are also unsupported or lossy:

• --data-urlencode 'q=hello world' is ignored;
• repeated -d / --data flags are not combined;
• --request POST and --request=POST are not recognized;
• --header and unquoted -H X-Test:value forms are not recognized;
• escaped quotes/backslashes and shell line continuations are not parsed as one token stream;
• unsupported flags are silently ignored instead of being surfaced to the user.

Why this matters

The tool presents generated code as an equivalent request. Silent parser corruption is more dangerous than a visible unsupported-syntax error because users can copy code that targets the right URL but sends different headers, method, or body data.

JSON bodies are one of the most common cURL inputs, so this affects the primary workflow rather than an edge case.

Recommended implementation

Replace the regex-only parser with a small quote-aware tokenizer/parser that:

1. tokenizes shell-style single quotes, double quotes, backslash escapes, and line continuations;
2. normalizes supported aliases (-X/--request, -H/--header, -d/--data/--data-raw/--data-binary, --data-urlencode, --url);
3. preserves repeated flags in order and combines body fragments using cURL-compatible semantics;
4. returns structured warnings/errors for unsupported or ambiguous flags instead of silently dropping them;
5. keeps parsing separate from language-specific code generation.

A browser-compatible, well-maintained parser may be used if its supported cURL subset, bundle impact, and escaping semantics are documented and tested.

Acceptance criteria

• A single-quoted JSON body containing double quotes is parsed in full.
• Double-quoted arguments with escaped quotes/backslashes are parsed correctly.
• --request, --header, --url, and the supported data flag aliases are recognized.
• Repeated body/header flags are handled deterministically and covered by tests.
• --data-urlencode is either represented faithfully or rejected with a clear unsupported-syntax message.
• Unsupported flags produce a visible warning/error; they are not silently discarded.
• Parser tests cover JSON, form data, spaces, escaped characters, multiline commands, and malformed quoting.
• Existing code-injection regression tests continue to pass.

Relevant files

• src/features/tools/curl-to-code/logic.ts
• src/features/tools/curl-to-code/page.tsx
• tests/unit/curl-to-code-codegen.test.ts

[baixiangcpp]

Resolved in merged PR #335 by commit 782db5b: quote-aware cURL tokenization and unsafe shell expansion rejection. The final PR CI build (Node 20) and Cloudflare Pages checks passed.