feat(repo): add 'azldev repo query' for inspecting and managing RPM repositories by liunan-ms · Pull Request #213 · microsoft/azure-linux-dev-tools

liunan-ms · 2026-05-28T03:14:57Z

This PR adds a new repo command group with a query subcommand that wraps dnf, auto-discovers the RPM sub-repos it should enable, probes them for reachability, then exec's into dnf with the surviving slots wired up via --repofrompath / --enablerepo. --no-debuginfo / --no-srpms are available for dropping sub-repos by their declared kind.

Two selection modes:

URL mode (azldev repo query --repo-prefix URL [--template] …)
Each URL is expanded against an rpm-repo-set-template (default --template) into one sub-repo per template row, fanned out per --arch where the row's subpath contains $basearch http://, https://, and file:// prefixes are all accepted, so a locally synthesized repo tree (e.g. one produced by a build) can be queried directly without serving it over HTTP
Project-config mode (azldev repo query --version VER [--use-case rpm-build|image-build] ...)
Resolves the inputs list of the default distro's VER version from [distros..versions..inputs]. GPG keys, per-repo arch allowlists, and metalink-only rejection come from [resources.rpm-repo-sets.] / [resources.rpm-repos.]. --use-case defaults to rpm-build. Mutually exclusive with --repo-prefix / --template.

Example output:

URL mode

2. Project-config mode using the distro version inputs in [azurelinux.repos.distro.toml](https://github.com/microsoft/azurelinux/pull/17597)

reubeno

Thanks for putting this together! A few questions -- happy to chat synchronously to go over some of it.

reubeno · 2026-05-28T17:57:50Z

+
+// ReadRootConfig returns the bytes of the embedded root default configuration
+// file (defaults.toml). Callers that need to inspect a single value from the
+// defaults without paying for the full project-config loader can decode the


Is this just a performance optimization? I don't know that we want to set the precedent for directly inspecting the configs without going through validation, include loading, group inheritance, etc.

Removed this helper. Refactored the code to use project-config with full validation.

reubeno · 2026-05-28T17:59:15Z

+func OnAppInit(app *azldev.App) {
+	cmd := &cobra.Command{
+		Use:   "repo",
+		Short: "Inspect and synthesize Azure Linux RPM repositories",


This is relatively specifc (i.e., "inspect and synthesize"). I think the command tree could be more broadly described as inspecting and managing RPM repositories. They need not even be Azure Linux, as we could use it to query, say, Fedora too.

reubeno · 2026-05-28T18:01:08Z

@@ -0,0 +1,329 @@
+// Copyright (c) Microsoft Corporation.


Meta-question: for querying, is there a reason we can't just wrap dnf? The value is in being able to wire up all the required repo flags, gpg options, etc. -- but there doesn't seem like there's much value in doing the heavy lifting of parsing and inspecting rpm repo metadata.

The current dnf wrapper script is something I use often -- and it lets me directly use all dnf commands (e.g., repoclosure, repoquery, list, etc.) as well as all their standard options.

Refactored the code to make repo query command a dnf wrapper.

reubeno · 2026-05-28T18:04:59Z

+	cmd.Flags().BoolVar(&options.Synthesize, "synthesize", false,
+		"reserved for a future release")
+
+	if err := cmd.MarkFlagRequired("repo-prefix"); err != nil {


Why would this be required if the user wants to point to an RPM set?

"--repo-prefix" now is not a required flag, repeatable repo prefixes are allowed for URL mode.

reubeno · 2026-05-28T18:05:43Z

+		"comma-separated arches to expand $basearch over")
+	cmd.Flags().StringVar(&options.OutputDir, "output-dir", "",
+		"with --output json, write per-slot packages.json files under <dir>/<template>/<subrepo>/[<arch>/]")
+	cmd.Flags().BoolVar(&options.Synthesize, "synthesize", false,


Let's leave out future options for latest PRs?

synthesize support is removed from the current PR and will be added in a separate PR.

reubeno · 2026-05-28T18:07:54Z

+
+	cmd.Flags().StringArrayVar(&options.RepoPrefixes, "repo-prefix", nil,
+		"layout prefix; expanded under the chosen --template; may be repeated")
+	cmd.Flags().StringVar(&options.Template, "template", repolayout.DefaultTemplateName,


Is it possible to have reasonable defaults that leverage the default RPM inputs in the currently selected distro? My thought is that we'll have distro "versions" for "koji" vs. "dev" vs. "prod" -- and distro version selector may allow simplifying the querying?

Yes, this's added into the new --version mode in this PR. It pulls the repo list straight from [distros..versions..inputs] of the project's selected distro, only version is needed.

… probing Adds a new 'repo' command group with a 'query' subcommand that auto-discovers Azure Linux RPM sub-repos under one or more --repo-prefix URLs and exec's dnf against the reachable slots. Includes the shared repolayout package (template expansion, prefix normalization, dedup) used by the resolver. - bounded-parallel HEAD/Stat probes via parmap.Map + IOBoundConcurrency - aggregates all probe failures before aborting; per-prefix kept/dropped/failed logging - passes --disablerepo=* --refresh to dnf; one --repofrompath/--enablerepo pair per kept slot - regenerated CLI docs

Resolve the dnf repo set from the default distro's [distros.<d>.versions.<VER>.inputs] list instead of requiring --repo-prefix. --use-case selects rpm-build (default) or image-build inputs. Per-repo arch allowlists, GPG keys, and metalink-only rejection are honored; source repos with no $basearch are deduped across the per-arch fan-out. --repo-prefix/--template stay mutually exclusive with --version. Adds InputRepo.RepoID / InputRepo.GPGKey and a SubstituteBasearch helper in repolayout so the version-mode resolver can carry canonical ids and forward gpgkey/gpgcheck setopts to dnf. Includes internal unit tests for the new resolver and regenerated CLI docs.

reubeno · 2026-06-02T14:59:10Z

+
+// DefaultTemplateName is the name of the built-in standard Azure Linux layout
+// template defined in `defaultconfigs/content/defaults.toml`.
+const DefaultTemplateName = "azl-standard"


Since this is defined by defaultconfigs, could we define this const in the defaultconfigs package too? That would leave them close together.

reubeno · 2026-06-02T15:00:12Z

+	Arch string
+	// URL is the fully-resolved repo base URL.
+	URL string
+	// PrefixIndex is the 1-based position of the originating --repo-prefix among


Why are these prefix indices/counts needed? It seems like a bit of knowledge from above leaking down into this layer.

reubeno · 2026-06-02T15:01:12Z

+					SubrepoName:  sub.Name,
+					Kind:         kind,
+					Arch:         arch,
+					URL:          base + "/" + strings.ReplaceAll(sub.Subpath, basearchPlaceholder, arch),


Can we use URL.JoinPath instead here?

reubeno · 2026-06-02T15:01:20Z

+			TemplateName: templateName,
+			SubrepoName:  sub.Name,
+			Kind:         kind,
+			URL:          base + "/" + sub.Subpath,


Same as above.

reubeno · 2026-06-02T15:02:07Z

+}
+
+// DedupInputRepos drops duplicate entries by URL while preserving order.
+func DedupInputRepos(repos []InputRepo) []InputRepo {


Does the lo package offer any helpers that can do this for us? Perhaps lo.Uniq or perhaps they have a disjoint set data structure we could use?

reubeno · 2026-06-02T15:05:23Z

+	workerEnv, cancel := env.WithCancel()
+	defer cancel()
+
+	results := probeAll(workerEnv, env.IOBoundConcurrency(), repos)


Why do we need to probe the repos? dnf will end up doing the same anyhow. Let's see if we can simplify the code that we need to maintain here.

reubeno · 2026-06-02T15:05:37Z

+// surviving slots wired up via --repofrompath / --enablerepo. It does not
+// return on success — control is transferred to dnf via [syscall.Exec].
+func RunQuery(env *azldev.Env, options *QueryOptions, dnfArgs []string) error {
+	if !env.CommandInSearchPath(DnfBinary) {


There's a prereqs package in azldev that we should try to use here.

reubeno · 2026-06-02T15:08:51Z

+	}
+
+	// Hand control to dnf — does not return on success.
+	if err := syscall.Exec(dnfPath, argv, os.Environ()); err != nil {


Why exec? Can we use the externalcmd package and retain control after it exits?

reubeno · 2026-06-02T15:18:37Z

+
+	argv := buildDNFArgv(kept, dnfArgs)
+
+	dnfPath, err := exec.LookPath(DnfBinary)


Is this required? Shouldn't execution do the right searching through paths.

reubeno · 2026-06-02T15:19:43Z

+	cfg := env.Config()
+	version := options.Version
+
+	distroName := cfg.Project.DefaultDistro.Name


Could you file a follow-up GitHub issue for us to allow overriding the "default distro" on the azldev command line. Right now I think it's only configurable via TOML files.

Copilot AI review requested due to automatic review settings May 28, 2026 03:14

liunan-ms force-pushed the liunan/repoquery branch from b12a4e7 to c392546 Compare May 28, 2026 16:52

reubeno reviewed May 28, 2026

View reviewed changes

liunan-ms force-pushed the liunan/repoquery branch from c392546 to 4885e25 Compare June 1, 2026 19:29

liunan-ms force-pushed the liunan/repoquery branch from 4885e25 to ce01d33 Compare June 1, 2026 20:46

liunan-ms changed the title ~~feat(repo): add 'azldev repo query' for standard AZL repo layouts~~ feat(repo): add 'azldev repo query' for inspecting and managing RPM repositories Jun 1, 2026

reubeno reviewed Jun 2, 2026

View reviewed changes


		argv := buildDNFArgv(kept, dnfArgs)

		dnfPath, err := exec.LookPath(DnfBinary)

Conversation

liunan-ms commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

reubeno left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

liunan-ms commented May 28, 2026 •

edited

Loading