Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/front-matter.md
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,10 @@ supply-chain: # optional internal supply-chain mirror (see docs
name: myacr.azurecr.io/mirror # registry host or base path (artifact names kept under it)
service-connection: acr-conn # REQUIRED when registry is set (ACR has no System.AccessToken path)
service-connection: shared-conn # optional feed/registry fallback; never applies to pipeline-artifact
packages: # optional shared package feed for runtimes.python/node/dotnet
feed: my-proj/my-feed # feed name or project/feed; scalar `packages: my-feed` shorthand also works
# organization: myorg # optional; defaults to the org inferred from the git remote
# python: true # per-ecosystem opt-out (node:/dotnet: likewise); each defaults to true
# ado-aw-debug: # debug-only knobs; see docs/ado-aw-debug.md
# skip-integrity: false # omit generated pipeline integrity verification
parameters: # optional ADO runtime parameters (surfaced in UI when queuing a run)
Expand Down
18 changes: 18 additions & 0 deletions docs/runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,12 @@ runtimes:
| `feed-url` | string | Internal PyPI feed URL. Injects `PIP_INDEX_URL` and `UV_DEFAULT_INDEX` env vars into the agent environment. |
| `config` | string | Path to a pip/uv config file. Accepted with a warning — the file will not be available inside the AWF agent environment until proxy-auth support lands. |

> **Shared feed:** `feed-url` is optional. When
> [`supply-chain.packages`](supply-chain.md#shared-package-feed-for-the-runtimes-packages)
> declares one feed identity, this runtime derives its endpoint from it
> automatically. An explicit `feed-url` here overrides the shared feed; a
> `config` file suppresses it.

When enabled, the compiler:
- Contributes a `UsePythonVersion@0` task to `Declarations::agent_prepare_steps` (runs before AWF)
- If `feed-url` is set, also injects `PipAuthenticate@1` to authenticate the ADO build service identity for internal feeds
Expand Down Expand Up @@ -93,6 +99,12 @@ runtimes:
| `feed-url` | string | Internal npm registry URL. Injects `NPM_CONFIG_REGISTRY` env var into the agent environment. |
| `config` | string | Path to an .npmrc config file. Accepted with a warning — the file will not be available inside the AWF agent environment until proxy-auth support lands. |

> **Shared feed:** `feed-url` is optional. When
> [`supply-chain.packages`](supply-chain.md#shared-package-feed-for-the-runtimes-packages)
> declares one feed identity, this runtime derives its endpoint from it
> automatically. An explicit `feed-url` here overrides the shared feed; a
> `config` file suppresses it.

When enabled, the compiler:
- Contributes a `UseNode@1` task to `Declarations::agent_prepare_steps` (runs before AWF)
- If `feed-url` or `config` is set, also injects `npmAuthenticate@0` (and an ensure-`.npmrc` step) to authenticate the ADO build service identity for internal feeds
Expand Down Expand Up @@ -137,6 +149,12 @@ runtimes:
| `feed-url` | string | Internal NuGet feed URL (typically the v3 `index.json` of an Azure Artifacts feed). When set, the compiler creates a minimal `nuget.config` if none exists and runs `NuGetAuthenticate@1`. |
| `config` | string | Path to a checked-in `nuget.config` in the repo. When set, the compiler runs `NuGetAuthenticate@1` (which auto-discovers `nuget.config` files in the workspace). Mutually exclusive with `feed-url`. |

> **Shared feed:** `feed-url` is optional. When
> [`supply-chain.packages`](supply-chain.md#shared-package-feed-for-the-runtimes-packages)
> declares one feed identity, this runtime derives its endpoint from it
> automatically. An explicit `feed-url` here overrides the shared feed; a
> `config` file suppresses it.

**`global.json` precedence.** A `global.json` file in the repo is the canonical
way to pin the .NET SDK. The compiler enforces a single source of truth:

Expand Down
108 changes: 108 additions & 0 deletions docs/supply-chain.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,16 @@ supply-chain:
name: myacr.azurecr.io/mirror # registry host or base path
service-connection: acr-conn # required when registry is set
service-connection: shared-conn # optional shared fallback for both targets
packages: # shared package feed for the language runtimes
feed: my-proj/my-feed # feed name or "project/feed"
```

| Field | Type | Required | Purpose |
|-------|------|----------|---------|
| `feed` | scalar **or** `{ name, service-connection }` | optional | Enables the binary mirror (#1–#3). A bare string is shorthand for `{ name: <string> }`. |
| `pipeline-artifact` | `{ project, definition-id, run-id, artifact }` | optional | Uses one exact producer run as the complete binary source (#1–#3). Mutually exclusive with `feed`. |
| `registry` | scalar **or** `{ name, service-connection }` | optional | Enables the image mirror (#4), independently of either binary source. |
| `packages` | scalar **or** `{ feed, organization, project, python, node, dotnet }` | optional | One shared package-feed identity that the `python`, `node`, and `dotnet` runtimes each restore packages from. Independent of every other key. |
| `service-connection` | string | optional | Shared fallback connection for `feed` and `registry`; it does not apply to `pipeline-artifact`. |

`feed` and `pipeline-artifact` are mutually exclusive. `registry` is
Expand Down Expand Up @@ -225,6 +228,97 @@ passes both `--image-tag <awf-version>` and `--image-registry <registry>`
directly to the AWF invocation so `--skip-pull` resolves every pre-pulled image
(including `api-proxy`) under the mirror name instead of GHCR.

## Shared package feed for the runtimes (`packages`)

`feed` / `pipeline-artifact` / `registry` all mirror **ado-aw's own**
artifacts. `packages` is a different concern: it is the feed that the language
runtimes restore **your project's** packages from. Without it, every runtime
needs its own `feed-url:`:

```yaml
runtimes:
python:
feed-url: "https://pkgs.dev.azure.com/myorg/my-proj/_packaging/my-feed/pypi/simple/"
node:
feed-url: "https://pkgs.dev.azure.com/myorg/my-proj/_packaging/my-feed/npm/registry/"
dotnet:
feed-url: "https://pkgs.dev.azure.com/myorg/my-proj/_packaging/my-feed/nuget/v3/index.json"
```

An Azure Artifacts feed is multi-protocol, so those three URLs are one feed.
`supply-chain.packages` declares that feed **identity** once and lets the
compiler derive each endpoint:

```yaml
runtimes:
python: true
node: true
dotnet: true
supply-chain:
packages: my-proj/my-feed
```

`packages` is independent of `feed`, `pipeline-artifact`, and `registry` — set
it alone or alongside any of them.

### Fields

| Field | Type | Required | Purpose |
|-------|------|----------|---------|
| `feed` | string | **yes** | Feed name (org-scoped feed) or `project/feed` (project-scoped feed). |
| `organization` | string | optional | ADO organization. Defaults to the org inferred from the repository's git remote. |
| `project` | string | optional | ADO project. Mutually exclusive with the `project/feed` form of `feed`. |
| `python` / `node` / `dotnet` | bool | optional | Per-ecosystem opt-out; each defaults to `true`. |

A bare scalar is sugar for `{ feed: <string> }`:

```yaml
supply-chain:
packages: my-feed # same as { feed: my-feed }
```

### Derived endpoints

For organization `ORG`, optional project `PROJ`, and feed `FEED`:

| Runtime | Derived URL | Applied as |
|---------|-------------|------------|
| `python` | `https://pkgs.dev.azure.com/ORG/PROJ/_packaging/FEED/pypi/simple/` | `PIP_INDEX_URL` + `UV_DEFAULT_INDEX` env vars, plus `PipAuthenticate@1` |
| `node` | `https://pkgs.dev.azure.com/ORG/PROJ/_packaging/FEED/npm/registry/` | `NPM_CONFIG_REGISTRY` env var, plus an ensure-`.npmrc` step and `npmAuthenticate@0` |
| `dotnet` | `https://pkgs.dev.azure.com/ORG/PROJ/_packaging/FEED/nuget/v3/index.json` | generated `nuget.config` package source, plus `NuGetAuthenticate@1` |

The `/PROJ` segment is omitted for an org-scoped feed (a bare `feed:` with no
`project:`). The organization and project are embedded verbatim, so both must
match `[A-Za-z0-9._-]`; anything requiring URL escaping is rejected at compile
time rather than silently producing a malformed URL. Compilation also fails
with an actionable message when no organization can be resolved — set
`organization:` explicitly when compiling outside an Azure DevOps clone.

### Precedence

For each runtime, the effective package source resolves as:

1. `runtimes.<runtime>.feed-url` — an explicit per-runtime URL always wins.
2. `runtimes.<runtime>.config` — the checked-in config file owns the package
source, so `supply-chain.packages` is **not** applied. A compile warning is
emitted when both are set.
3. `supply-chain.packages` (when the ecosystem has not opted out).
4. The ecosystem's public default (PyPI / npmjs / nuget.org).

### Authentication

No service connection is involved. The runtimes authenticate with the standard
`PipAuthenticate@1` / `npmAuthenticate@0` / `NuGetAuthenticate@1` tasks under
the pipeline's build identity, exactly as they do for a per-runtime
`feed-url:`. Grant that identity the **Feed Reader** role on the feed. This
matches the same-organization feed story described under
[Authentication](#authentication) above; cross-organization package feeds are
not supported by this key — use a per-runtime `feed-url:` plus your own
authentication step.

`pkgs.dev.azure.com` is already in the agent's default AWF allowlist, so no
`network:` change is needed for the agent to restore packages from the feed.

## Examples

Mirror everything, two different connections:
Expand Down Expand Up @@ -280,6 +374,20 @@ supply-chain:
service-connection: acr-conn
```

One shared package feed for all three runtimes, with npm left on the public
registry:

```yaml
runtimes:
python: true
node: true
dotnet: true
supply-chain:
packages:
feed: my-proj/my-feed
node: false
```

## Network isolation note

The mirror fetches (`NuGetAuthenticate@1`, `DownloadPackage@1`,
Expand Down
40 changes: 39 additions & 1 deletion src/compile/extensions/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ use std::collections::BTreeMap;
use std::fmt;
use std::str::FromStr;

use super::types::FrontMatter;
use super::types::{FrontMatter, PackageEcosystem};

// ──────────────────────────────────────────────────────────────────────
// MCPG types (used by both the trait and standalone compiler)
Expand Down Expand Up @@ -175,6 +175,44 @@ impl<'a> CompileContext<'a> {
})
}

/// Resolve the shared `supply-chain.packages` feed endpoint for
/// `ecosystem`, if one is configured and the ecosystem opts in.
///
/// Returns `Ok(None)` when no `supply-chain.packages` block is present or
/// the ecosystem is switched off; returns `Err` when a feed is configured
/// but no organization can be determined (see
/// [`PackageFeedConfig::url_for`](crate::compile::types::PackageFeedConfig::url_for)).
pub fn package_feed_url(&self, ecosystem: PackageEcosystem) -> Result<Option<String>> {
let Some(packages) = self
.front_matter
.supply_chain()
.and_then(|sc| sc.packages.as_ref())
else {
return Ok(None);
};
if !packages.applies_to(ecosystem) {
return Ok(None);
}
let url = packages.url_for(ecosystem, self.ado_org())?;
crate::validate::validate_feed_url(&url, "supply-chain.packages")?;
Ok(Some(url))
}

/// Cheap presence check: is a `supply-chain.packages` block configured
/// that opts `ecosystem` in?
///
/// Unlike [`Self::package_feed_url`] this performs no organization
/// resolution and no URL validation, so it never fails. Use it in
/// diagnostic-only paths (e.g. deciding whether to warn that a
/// runtime-local `config` takes precedence) where the resolved URL is
/// never consumed.
pub fn has_package_feed(&self, ecosystem: PackageEcosystem) -> bool {
self.front_matter
.supply_chain()
.and_then(|sc| sc.packages.as_ref())
.is_some_and(|packages| packages.applies_to(ecosystem))
}

fn ado_context_override() -> Result<Option<AdoContext>> {
let Some(value) = std::env::var_os(COMPILE_REMOTE_URL_ENV) else {
return Ok(None);
Expand Down
Loading