[Automation] Generate SDK based on TypeSpec 0.45.4 DEV (typespec-azure PR 4905)#49843
Closed
azure-sdk-automation[bot] wants to merge 27 commits into
Closed
[Automation] Generate SDK based on TypeSpec 0.45.4 DEV (typespec-azure PR 4905)#49843azure-sdk-automation[bot] wants to merge 27 commits into
azure-sdk-automation[bot] wants to merge 27 commits into
Conversation
Ports the post-publish SDK generation automation from Azure/autorest.java to this repo, adapted for the typespec-java emitter's new home in Azure/typespec-azure (packages/typespec-java). - eng/pipelines/post-publish-sdk.yaml: regenerates all TypeSpec-based SDKs and opens an automated draft PR. Uses the published @azure-tools/typespec-java npm package by default; when the TypeSpecJavaPRId parameter is set, builds the emitter dev package from that typespec-azure PR (turbo builds workspace deps, then Build-TypeSpec.ps1 builds and packs). - eng/pipelines/scripts/sync_sdk.py: regeneration script. - eng/pipelines/scripts/patches/: SDK patches applied after generation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the single PR-id parameter with a DevPackage boolean (default false) plus a PRId string that defaults to the sentinel 'none' so the run panel never requires input for the default (published-package) path. PRId is validated at runtime only when DevPackage is true. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
DevPackage=false uses the published npm package. DevPackage=true builds the emitter from source: from the typespec-azure PR in PRId, or from main when PRId is left at the 'none' sentinel. Drops the previous PRId-required error. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The dev build failed because the 'core' submodule (microsoft/typespec) was not checked out: tspd and the http-client-java generator required by Build-Generator.ps1 live there. Initialize submodules after settling on the ref, only when DevPackage=true. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
tsp-client generate-lock-file failed with 'Unsupported URL Type catalog:' because New-EmitterPackageJson.ps1 was fed the typespec-azure monorepo source package.json, whose deps use the catalog:/workspace: protocols. Extract the packed tarball's package.json instead (pnpm pack resolves those protocols to concrete versions) and use it as the basis for emitter-package.json. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The default (DevPackage=false) route now pins eng/emitter-package.json to the published emitter given by TypeSpecJavaVersion, then regenerates - matching the post-publish-sdk intent. sync_sdk.py gains --emitter-version: it npm-packs the published package, seeds emitter-package.json from the tarball's resolved package.json, and regenerates the lock. DevPackage=true still builds from source (PRId or main). The typespec-azure clone/build steps are gated to the dev route, and the default route fails fast if no version is given. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The presence of a published version fully determines the route, so DevPackage is redundant. TypeSpecJavaVersion set = published route; empty = build emitter from source (PRId, or main). sync_sdk.py drops --dev-package and infers the route from --emitter-version. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Azure DevOps string parameters cannot be left truly empty in the run UI, so default TypeSpecJavaVersion to the 'none' sentinel (matching PRId). Pipeline conditions compare against 'none'; sync_sdk.py normalizes 'none' to empty and treats it as the dev (build-from-source) route. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Once the published tarball is downloaded via npm pack, its package.json has resolved dependency versions, so tsp-client generate-config-files can consume it directly to produce both emitter-package.json and its lock file - replacing the separate New-EmitterPackageJson.ps1 + generate-lock-file calls. The dev route still uses those two steps to inject the local .tgz path between them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
pnpm pack resolves workspace:^ dependencies to caret ranges (e.g. ^0.69.1) in the packed package.json. New-EmitterPackageJson.ps1 pins peer deps from those devDependencies values, so carets leaked into emitter-package.json (^0.69.0 instead of the exact 0.69.2 that autorest.java produced). Strip the leading caret/tilde from the extracted package.json so peer deps are pinned to exact versions. Simple ranges like '>=x <y', 'workspace:', 'catalog:' are left as-is. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This reverts commit fc1b881.
Regenerating emitter-package.json from the emitter tarball pinned the peer/dev dependency versions from that tarball, which could be stale (e.g. an older typespec-client-generator-core than the one being released). Instead, only overwrite the @azure-tools/typespec-java dependency (published version or dev .tgz path) in the existing emitter-package.json and regenerate the lock file. Other dependency versions are managed elsewhere and updated before the pipeline runs. Removes the now-unused tarball extraction / config-file generation helpers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Dev route now seeds emitter-package.json from the built dev package via New-EmitterPackageJson.ps1 (instead of generate-config-files, which resolves against a published version that does not exist for a dev build) and points the emitter dependency at the local tarball. Published route uses tsp-client generate-config-files against the resolved published package.json. Both routes then inject the designated TypeSpec libraries that the emitter does not declare (openai-typespec, typespec-liftr-base, typespec-azure-portal-core), pinned to their latest published version via 'npm view <pkg>@latest version', and regenerate the lock file. This avoids adding those libraries as emitter dependencies in Azure/typespec-azure (PR 4866). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
Refine emitter-package.json generation per Azure/typespec-azure PR 4866 review: Published route: seed with 'tsp-client generate-config-files --use-npm-pinning' against the published emitter manifest (obtained via 'npm view --json', no tgz). Dev route: still inspect the built dev .tgz and seed via New-EmitterPackageJson.ps1 (generate-config-files cannot resolve an unpublished dev build), then point the emitter dependency at the local tarball. Both routes then: (1) resolve every @azure-tools/* and @typespec/* devDependency to its latest published exact version via 'npm view <pkg>@latest version' (--use-npm-pinning alone still yields caret ranges), and (2) add the designated libraries the emitter does not declare (openai-typespec, typespec-liftr-base, typespec-azure-portal-core) pinned to the exact version from the azure-rest-api-specs package.json (fetched from raw.githubusercontent main), falling back to npm latest if the specs repo has not updated yet. Finally regenerate the lock file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
The dev package tarball's package.json has fully resolved dependency versions (no
workspace:/catalog: protocols), so generate-config-files can seed emitter-package.json
for the dev route too - removing the New-EmitterPackageJson.ps1 special case.
generate-config-files always runs a lock step ('npm install') that resolves the emitter
dependency, which fails for an unpublished dev build. The dev route works around this by
passing an --overrides file that points @azure-tools/typespec-java at the local dev
tarball, so npm installs the emitter from the tarball instead of the registry.
--use-npm-pinning is omitted for the dev route (it would 'npm view' the unpublished
emitter); the published route keeps it. Both routes still resolve TypeSpec deps to the
latest published versions and add the designated libs afterward.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
Add @typespec/openapi3 as a designated library (it is carried in eng/emitter-package.json but is not a typespec-java emitter dependency), sourced from the specs repo package.json alongside openai-typespec and typespec-liftr-base. Version typespec-azure-portal-core from the latest published npm version instead, to track the newest release. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
They belong to the same release group as the TypeSpec/Azure libraries the emitter depends on, which are already pinned to latest; keeping them on latest avoids peer conflicts from a lagging specs-repo pin. openai-typespec and typespec-liftr-base are released independently, so they stay sourced from the specs repo. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
…d libs These two are currently peer dependencies of typespec-java, so generate-config-files pins them (openai-typespec to the exact peer 1.21.0). Re-versioning them as designated libs from the specs repo would violate the emitter's peer constraint and break the lock 'npm install' with ERESOLVE. Skip them for now (kept in DESIGNATED_LIBRARIES_SKIPPED so they stay excluded from resolve-to-latest) and pin emitter-package.json to compatible versions (openai-typespec 1.21.0, typespec-liftr-base 0.14.0). TODO: move them back into DESIGNATED_LIBRARIES_FROM_SPECS once removed from typespec-java's peerDependencies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
tsp-client generate-config-files MERGES into an existing eng/emitter-package.json and only overwrites emitter-peer entries. A designated library carried over from a previous run (e.g. @typespec/openapi3, which is not an emitter peer) kept its stale version and conflicted with the freshly pinned peers, breaking the internal lock-generation 'npm install' with ERESOLVE - the failure seen in the sync pipeline for 0.45.6. Remove the designated libraries from the existing emitter-package.json before generating, so the merge starts from only the emitter's own peers; they are added back afterward (from npm latest or the specs repo). Skipped designated libs (still emitter peers) are re-added by generate-config-files from the emitter's peerDependencies. This preserves the file's existing key order (cleaner diffs) versus deleting the whole file. Verified locally end-to-end (published route, 0.45.6): remove-designated, clean seed, internal lock, resolve-to-latest, designated libs, and final lock all succeed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: abf20313-1bb4-4ae3-9764-f0562785cf32
github-actions
Bot
added
DevCenter
Document Intelligence
Event Grid
Load Testing
Mgmt
github-project-automation
Bot
moved this from Untriaged
to Done
in Azure SDK for Search
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[Automation] Generate SDK based on TypeSpec 0.45.4 DEV (typespec-azure PR 4905)