feat: add generate-spec command#2937
Conversation
🦋 Changeset detectedLatest commit: 2d41e65 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
2c94a80 to
6b62b28
Compare
sobanieca-redocly
changed the title
generate-spec commandgenerate-spec command
Performance Benchmark (Lower is Faster)
|
|
@cursor review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
@cursor review |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
2 issues from previous reviews remain unresolved.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 956bf00. Configure here.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 3a11b20. Configure here.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit d7ee556. Configure here.
|
@cursor review |
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 90a2052. Configure here.
|
@cursor review |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 22674ba. Configure here.
| error instanceof Error ? error.message : String(error) | ||
| }\n` | ||
| ); | ||
| } |
There was a problem hiding this comment.
Final lint drops refinements
Medium Severity
When --with-ai runs, refineSpecWithAi can accept and merge some operations, then fail on the final whole-document lint. The handler catches that error and writes the original baseline YAML, so successful per-operation refinements are discarded instead of being emitted.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 896fd97. Configure here.
Co-authored-by: Jacek Łękawa <164185257+JLekawa@users.noreply.github.com>
| // cuid ("cjld2cjxh0000qzrmn831i7rn") and cuid2 ("tz4a98xxat96iws9zmbrgj3a") as | ||
| // generated by Prisma: 24-25 lowercase alphanumerics with at least one digit. | ||
| const CUID_RE = /^[a-z](?=[a-z0-9]*\d)[a-z0-9]{23,24}$/; | ||
| export const HTTP_METHODS = new Set([ |
There was a problem hiding this comment.
Can we use already defined constants OPENAPI3_METHOD_NAMES from core instead of redefining them?
Same for drift and proxy commands.
Its already redefined multiple times in
packages/cli/src/commands/drift/openapi/loader.tspackages/cli/src/commands/proxy/server.ts
| ## Usage | ||
|
|
||
| ```bash | ||
| redocly generate-spec <traffic> |
There was a problem hiding this comment.
should we call it traffic-log or traffic-record to make it more clear?
There was a problem hiding this comment.
The description already says "traffic log file or folder", I would keep it short.
| | `--ai-provider` | `claude`, `codex`, or `cursor` (default `claude`). | | ||
| | `--ai-model` | Model passed to the selected provider. | | ||
| | `--ai-concurrency` | Number of operations refined in parallel (default `4`). | | ||
| | `-o, --output` | Write the result to a file instead of stdout. | |
There was a problem hiding this comment.
Should we also mentioned in README the limitation of producing only OAS v3.1.0 description?
| return /^[0-9a-f]{16,}$/i.test(segment); | ||
| } | ||
|
|
||
| export function singularize(value: string): string { |
There was a problem hiding this comment.
We already have pluralize in our core. Could you please check if you can reuse it instead of implementing singularize ?
There was a problem hiding this comment.
Thanks. We need singularize, but in this case I've just added an export in core with singularize. Hope it's fine.
| return finalizeVariants(foldExcessObjectVariants(variants)); | ||
| } | ||
|
|
||
| function sniffJson(bodyText: string | undefined): unknown { |
There was a problem hiding this comment.
Can you just reuse isJson here?
We can move it to core from respect if it feet your needs
export function isJSON(str: string): boolean {
try {
JSON.parse(str);
return true;
} catch {
return false;
}
}
| } | ||
|
|
||
| return { | ||
| openapi: '3.1.0', |
| const trafficPath = normalizeFsPath(argv.traffic); | ||
| const trafficFormat = argv['traffic-format']; | ||
|
|
||
| const baseline = applyValueInference( |
There was a problem hiding this comment.
should we lint generated spec without ai ?
| document.paths[path][method] = fragment.operation; | ||
| if (Object.keys(fragment.components).length > 0) { | ||
| document.components ??= { schemas: {} }; | ||
| Object.assign(document.components.schemas, fragment.components); |
There was a problem hiding this comment.
will it override already existed component ?
|
|
||
| import type { TrafficSample } from '../samples.js'; | ||
|
|
||
| const SYSTEM_INSTRUCTIONS = `You are an expert API designer. You are given one operation from an OpenAPI 3.1 description that was inferred mechanically from recorded HTTP traffic, the component schemas it references, and a sample of the real recorded HTTP exchanges for that operation. The inferred description is only a hypothesis: types are coarse, every observed property is marked required, and descriptions are missing. Alternative body shapes observed in the traffic appear as "oneOf" variants or type unions. Common string formats (uuid, date-time, date, email, uri, ipv4) and small closed enums are detected conservatively from the observed values. |
| - Factor structure shared between variants or schemas with "allOf": extract the common base into components/schemas and compose each variant as "allOf" of the base plus its specific properties. | ||
| - Keep the $ref references to the component schemas you were given. You may refine the definition of a referenced component when the samples justify it, and you may add new components, but never rename an existing component and never reuse a reserved component name listed in the input. | ||
| - Add concise, useful "summary" and "description" fields to the operation, its parameters and important schema properties. | ||
| - Add example values drawn from the real samples where helpful. |
| }); | ||
| }); | ||
|
|
||
| describe('generate-spec - AI refinement', () => { |
There was a problem hiding this comment.
can we test the prompt somehow? to avoid regression while editing
| Refine the operation into the most accurate OpenAPI 3.1 definition you can justify from the evidence, while staying conservative: | ||
| - Keep the path template and the HTTP method exactly as given. Do not rename path parameters, do not move parameters to the path item level, and keep every response status code. The result is programmatically checked and rejected otherwise. | ||
| - Keep the operationId unless it is clearly wrong. | ||
| - Narrow property types where the samples clearly support it (formats such as date-time, uuid, email, uri; integer vs number; enums when a field only ever takes a small fixed set of values). Verify the formats and enums already detected against the samples; correct or extend them where justified. |
There was a problem hiding this comment.
Verify the formats and enums already detected against the samples
will it remove what was found in the samples but was not passed to ai?
|
|
||
| Refine the operation into the most accurate OpenAPI 3.1 definition you can justify from the evidence, while staying conservative: | ||
| - Keep the path template and the HTTP method exactly as given. Do not rename path parameters, do not move parameters to the path item level, and keep every response status code. The result is programmatically checked and rejected otherwise. | ||
| - Keep the operationId unless it is clearly wrong. |
There was a problem hiding this comment.
Keep the operationId unless it is clearly wrong
can it remove operationId ?
| import { normalizeServerPrefix, resolvePathForServer } from '../drift/utils/server.js'; | ||
| import { HTTP_METHODS, templatizePath } from './generator.js'; | ||
|
|
||
| export interface TrafficSample { |
There was a problem hiding this comment.
can we add another parameters (e.g. headers) ?
| - Relax "required" for properties that are plausibly optional (e.g. absent in some samples, or clearly nullable). Express nullable values as type unions such as ["string", "null"]. | ||
| - When a request or response body has alternative shapes — the baseline shows "oneOf" variants, or the samples show clearly different payloads — model them explicitly with "oneOf". Name each variant in components/schemas, and add a "discriminator" when a property (such as "type" or "kind") selects the variant. | ||
| - Factor structure shared between variants or schemas with "allOf": extract the common base into components/schemas and compose each variant as "allOf" of the base plus its specific properties. | ||
| - Keep the $ref references to the component schemas you were given. You may refine the definition of a referenced component when the samples justify it, and you may add new components, but never rename an existing component and never reuse a reserved component name listed in the input. |
There was a problem hiding this comment.
can concurrency break this instruction?
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 2d41e65. Configure here.
| }; | ||
|
|
||
| const workers = Math.max(1, Math.min(options.concurrency ?? 1, operations.length)); | ||
| await Promise.all(Array.from({ length: workers }, worker)); |
There was a problem hiding this comment.
Parallel workers throw unhandled rejections
Medium Severity
When --ai-concurrency is above 1 and the provider CLI is missing, one worker rethrows CliNotFoundError while others may still be inside refineOperation. Promise.all settles on the first rejection, so additional worker rejections can surface as unhandled promise rejections, and in-flight workers keep calling the provider until their current attempt finishes.
Reviewed by Cursor Bugbot for commit 2d41e65. Configure here.
4 participants
What/Why/How?
Added new
generate-speccommand which will infer OAD basing on traffic data. To improve accuracy, there is option to use AI agents to improve results.Reference
Testing
Screenshots (optional)
Check yourself
Security
Note
Medium Risk
Large new surface area: optional
--with-aisends traffic samples to external CLIs (data-exfiltration concern if misused), plus substantial inference/refinement logic that could produce misleading specs if users treat output as authoritative without their own lint/governance.Overview
Introduces the experimental
generate-speccommand: it reads the same traffic formats asdrift(HAR, Kong, Nginx/Apache JSON, NDJSON), builds an OpenAPI 3.1 baseline deterministically, then optionally refines it with--with-ai.The baseline pipeline templatizes identifier-like path segments (using new
singularizein@redocly/openapi-core), merges request/response schemas across observations (includingoneOf/ nullable unions), extracts repeated shapes intocomponents/schemas, and applies conservative format/enum hints from observed strings.--serverscopes which exchanges count and setsservers.With
--with-ai, each operation is refined in parallel (--ai-concurrency, default 4) by spawning isolatedclaude,codex, orcursorCLIs; traffic samples are capped and sent in prompts. Accepted refinements must keep path, method, observed status codes, and pass lint with the built-inspecruleset—otherwise the baseline is kept and the command can fall back entirely if refinement fails.Wiring includes yargs registration, docs/sidebars, CODEOWNERS, unit tests, and e2e tests with stub AI binaries.
Reviewed by Cursor Bugbot for commit 2d41e65. Bugbot is set up for automated code reviews on this repo. Configure here.