test: improve unit test coverage across core packages#7297
Open
jongio wants to merge 14 commits intoAzure:mainfrom
Open
test: improve unit test coverage across core packages#7297jongio wants to merge 14 commits intoAzure:mainfrom
jongio wants to merge 14 commits intoAzure:mainfrom
Conversation
jongio
requested review from
JeffreyCA,
hemarina,
rajeshkamal5050,
tg-msft,
vhvb1989,
wbreza and
weikanglim
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
Adds unit tests to previously low-/no-coverage core packages to improve confidence in foundational utilities and telemetry environment detection logic (part of #7295).
Changes:
- Introduces new test suites for
pkg/syncmap,pkg/contracts,internal/tracing/resource,pkg/ux, andpkg/azureutil. - Exercises JSON (un)marshaling contracts and enum constants in
pkg/contracts. - Adds behavioral tests around UX primitives (printer/canvas/visuals) and CI/host environment detection.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| cli/azd/pkg/ux/ux_additional_test.go | Adds additional UX tests for console width, pointer helper, visuals, printer, and canvas behaviors. |
| cli/azd/pkg/syncmap/syncmap_test.go | Adds comprehensive tests for the typed sync.Map wrapper including concurrency. |
| cli/azd/pkg/contracts/contracts_test.go | Adds broad JSON/RFC3339Time coverage for core contract types and enums. |
| cli/azd/pkg/azureutil/resource_group_test.go | Adds tests for ResourceNotFoundError formatting and errors.As behavior. |
| cli/azd/internal/tracing/resource/resource_test.go | Adds tests for CI/provider detection, MAC validation, host detection, and basic New() construction. |
Member
|
We rely in Functional tests to get the real coverage. For example, you mentioned that 
With this PR, it is increased: 
Looking at the totals, before this PR: and with this PR: There is a 0.2% increase, which is good, but maybe not super significant. Some of the things we've mentioned in the past (please correct me if I am wrong @weikanglim ) is that real functional data is easier to fix and maintain than a heavy load of unit tests. This could be totally different now with the use of AI, but in the past we were having a very single change coming with a lot of unit tests changes - This used to make a developer to expend an hour in a fix and a day in updating mocks and unit tests 😃. As you can see, the unit tests we are adding here are not increasing too much the coverage, so consider looking at the Azure DevOps coverage reports for targeting the next coverage improvements. |
vhvb1989
approved these changes
Mar 26, 2026
jongio
force-pushed
the
jongio/improve-test-coverage
branch
from
377e4c1 to
d4eafa6
Compare
Member
Author
|
Thanks for this context @vhvb1989 — you're absolutely right and this is really helpful. I pulled the CI coverage artifacts (build 6065857 from main) and merged unit + integration coverage locally using Packages I targeted in this PR — actual combined coverage:
So 4 of 5 packages I targeted were already well-covered by functional tests. My initial analysis only ran Real gaps (combined < 5%):
Updated approach: I'm going to retarget this PR to focus on the real gaps (packages that are low even with integration test coverage), and I'll use the Azure DevOps coverage artifacts as the baseline going forward rather than unit-only analysis. Question: do you want me to keep the existing unit tests in this PR (they still add value for test isolation even if combined coverage was okay) or strip them out and focus purely on the real gaps? |
5 tasks
Collaborator
Azure Dev CLI Install InstructionsInstall scriptsMacOS/Linux
bash: pwsh: WindowsPowerShell install MSI install Standalone Binary
MSI
Documentationlearn.microsoft.com documentationtitle: Azure Developer CLI reference
|
Add comprehensive tests for the type-safe sync.Map wrapper including: - Store/Load for various key and value types - LoadOrStore for new and existing keys - LoadAndDelete for existing and missing keys - Delete including no-op on missing keys - Range with full iteration, early stop, and empty map - Concurrent read/write safety verification - Struct value types and integer keys
Add tests for ResourceNotFoundError: - Error message formatting with inner error - Error message with nil inner error - errors.As type assertion verification - Table-driven tests for different inner error types
Add comprehensive tests for CI environment detection: - All CI providers: GitHub Actions, Azure Pipelines, Travis, CircleCI, etc. - Case-insensitive boolean env var matching - Bool rule precedence over set rules - IsRunningOnCI integration test - isValidMacAddress with valid and invalid MAC addresses - execEnvForHosts for Codespaces detection - New() resource creation smoke test
Add tests for pure functions and core components: - ConsoleWidth with COLUMNS env var and fallbacks - Ptr generic helper function - Render/NewVisualElement creation and rendering - Printer: Fprintf, Fprintln, Size tracking, CursorPosition, ClearCanvas - Canvas: Run with single/multiple visuals, error propagation, Clear - CanvasManager.CanUpdate behavior - ErrCancelled sentinel error
- Wrap long t.Fatalf lines to stay under 125 char limit (lll) - Replace JWT-like test token with plain string to avoid gosec G101 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add unit tests targeting packages with genuinely low combined (unit + integration) CI coverage, identified by downloading and merging CI coverage artifacts from Azure DevOps: - pkg/keyvault (1% combined): ParseAzureKeyVaultSecret, IsValidSecretName, NewAzureKeyVaultSecret, IsAzureKeyVaultSecret - pkg/devcentersdk (0% combined): NewResourceId, NewResourceGroupId, NewApiVersionPolicy, constants and types - pkg/azdo (2.9% combined): createBuildDefinitionVariable, getDefinitionVariables for Bicep/Terraform/secrets/nil-creds Also adds eng/scripts/Get-CICoverageReport.ps1 for downloading and analyzing CI coverage artifacts from Azure DevOps pipelines. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…gin, kustomize, github Add comprehensive unit tests for 6 packages with <50% combined coverage: - pkg/ux (2% -> ~61%): formatting, helpers, render, confirm/prompt, select/multiselect - pkg/cloud (25% -> ~90%): all cloud configs, NewCloud, ParseCloudConfig - pkg/httputil (15% -> ~70%): ReadRawResponse, RetryAfter, TlsEnabledTransport - pkg/kubelogin (5% -> ~80%): construction, CheckInstalled, ConvertKubeConfig args - pkg/kustomize (43% -> ~80%): construction, CheckInstalled, WithCwd - pkg/github (36% -> ~90%): GetSlugForRemote edge cases, non-GitHub hosts Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ls, input, middleware, watch Add 23 test files across 11 packages targeting combined coverage gaps: - pkg/pipeline (21% -> ~55%): config parsing, provider selection, credential options - pkg/project (60% -> ~72%): service targets, resources, artifacts, framework parsing - pkg/prompt (28% -> ~55%): format helpers, subscription display, demo mode - pkg/ai (26% -> ~50%): scope construction, model helpers, mapper registry - pkg/azure (43% -> ~75%): resource ID builders, ARM template types - pkg/tools/git (44% -> ~70%): command arg builders, output parsers - pkg/tools/github (47% -> ~75%): CLI output parsers, regex patterns, constants - pkg/tools/maven (32% -> ~65%): command builders, POM parsing, version regex - pkg/input (58% -> ~75%): asker logic, console helpers, FS suggestions - cmd/middleware (40% -> ~60%): child actions, debug middleware, options - pkg/watch (49% -> ~80%): FileChange formatting, sorting, change types Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…cserver Add 13 test files across 6 package groups: - internal/cmd/add (5% -> ~40%): validators, resource types, service detection, diff formatting - internal/cmd/show (38% -> ~60%): showType mapping, container selection, resource meta - internal/agent (38% -> ~65%): consent checker, consent types, copilot features, logging utils - internal/mcp (39% -> ~55%): proxy handlers, MCP config types, tool registration - internal/vsrpc (27% -> ~45%): models serialization, progress messages, stream close - internal/grpcserver (26% -> ~50%): AI error mapping, proto conversion, subscription validation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…l, cmdsubst, convert, custommaps Add 14 test files across 8 packages: - pkg/azapi (44% -> ~60%): deployment outputs, provisioning states, login error detection - pkg/auth (52% -> ~65%): token claims, login scopes, memory cache, auth errors - pkg/infra (57% -> ~70%): resource ID utils, deployment manager, scope factories - pkg/infra/provisioning: parameter types, output parsing, state merge, options validation - pkg/containerapps (43% -> ~60%): API version policy, ingress config - pkg/tools/kubectl (50% -> ~80%): CLI methods, resource parsing, model serialization - pkg/cmdsubst (43% -> 100%): full coverage of command substitution and eval - pkg/convert (58% -> ~98%): all conversion helpers edge cases - pkg/custommaps (56% -> ~87%): ordered map operations, JSON parsing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add 6 test files across 3 package groups: - extensions/internal (48% -> ~65%): ToPascalCase, checksums, zip, UserFriendlyError - extensions/internal/models: SafeDashId, LoadExtension/LoadRegistry schema parsing - pkg/azdext (55% -> ~70%): framework/service/event envelopes, error types, version - pkg/azsdk/storage (37% -> ~60%): BlobSdkClient construction, account config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Run 'go fix ./...' to satisfy CI go-fix check: - Use new() instead of &v for pointer construction - Add //go:fix inline directives to helper functions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
jongio
force-pushed
the
jongio/improve-test-coverage
branch
from
a72e209 to
577080c
Compare
go fix inlined strPtr, ptrTo, kindPtr, locKindPtr to new() calls but left the now-unreferenced helper functions behind. golangci-lint unused check catches these dead functions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Comprehensive unit test coverage improvement across 36+ packages in the Azure Developer CLI. This PR adds 66 new test files with ~19,200 lines of test code, targeting packages with the lowest combined (unit + integration) coverage.
Coverage Impact
Packages Reaching 100% Unit Coverage
pkg/syncmappkg/contractspkg/cmdsubstpkg/kubeloginpkg/kustomizeinternal/agent/logginginternal/agent/tools/mcpMajor Coverage Gains (>30 percentage points)
pkg/cloudpkg/httputilpkg/uxpkg/cmdsubstpkg/azurepkg/tools/mavenpkg/convertpkg/custommapspkg/tools/kubectlpkg/containerappsSignificant Coverage Gains (10-30 percentage points)
internal/agent/securitypkg/watchpkg/tools/gitextensions/internal/modelsextensions/internalpkg/infrainternal/mcpinternal/cmd/addinternal/mcp/toolsModerate Coverage Gains (5-10 percentage points)
pkg/projectinternal/agent/copilotinternal/agent/consentpkg/aipkg/pipelinepkg/tools/githubinternal/grpcserverpkg/authpkg/azapiinternal/vsrpcpkg/azdextFiles Added (66 new test files)
Wave 1: Critical Coverage Gaps
pkg/ux/formatting_test.go— VisibleLength, CountLineBreaks, regex patternspkg/ux/helpers_test.go— DurationAsText, WritePart, boolean helperspkg/ux/render_test.go— TaskList + Spinner renderingpkg/ux/confirm_prompt_test.go— Confirm + Prompt componentspkg/ux/select_test.go— Select + MultiSelect componentspkg/cloud/cloud_test.go— Cloud configs, URL builderspkg/httputil/util_coverage_test.go— HTTP retry, request helperspkg/kubelogin/cli_test.go— CLI arg buildingpkg/kustomize/cli_coverage_test.go— Kustomize CLIpkg/github/remote_coverage_test.go— GetSlugForRemoteWave 2: Pipeline, Project, AI, Tools
pkg/pipeline/pipeline_helpers_test.go— Config parsing, provider selection, credentialspkg/project/service_target_kind_test.go— Service target kindspkg/project/resources_test.go— Resource typespkg/project/artifact_filter_test.go— Artifact filteringpkg/project/framework_service_language_test.go— Language parsingpkg/project/framework_service_noop_test.go— NoOp frameworkpkg/project/service_target_springapp_test.go— Spring app deprecationpkg/project/service_logic_test.go— Service config, UTF8 BOM strippingpkg/prompt/format_helpers_test.go— Subscription formattingpkg/ai/scope_test.go— NewScope constructorpkg/ai/model_helpers_test.go— Model version, SKU conversionpkg/ai/mapper_registry_test.go— Proto round-tripspkg/azure/resource_ids_additional_test.go— Resource ID builderspkg/azure/arm_template_test.go— ARM template typespkg/tools/git/git_additional_test.go— Git CLI methodspkg/tools/github/github_additional_test.go— GitHub CLI parserspkg/tools/maven/maven_additional_test.go— Maven CLIpkg/input/asker_test.go— Asker logicpkg/input/console_helpers_test.go— Console helperspkg/input/console_fs_test.go— FS suggestionscmd/middleware/middleware_helpers_test.go— Middleware wiringcmd/middleware/debug_test.go— Debug middlewarepkg/watch/watch_helpers_test.go— FileChange formattingWave 3: Commands, Agent, MCP, VSRPC
internal/cmd/add/add_unit_test.go— Validators, resource types, service detectioninternal/cmd/show/show_unit_test.go— Type mapping, container selectioninternal/agent/consent/checker_test.go— Consent checkerinternal/agent/consent/types_test.go— Consent typesinternal/agent/copilot/feature_test.go— Copilot featuresinternal/agent/copilot/helpers_test.go— Helper functionsinternal/agent/logging/util_test.go— TruncateStringinternal/mcp/proxy_handlers_test.go— Proxy handlersinternal/mcp/types_test.go— MCP config typesinternal/mcp/tools/tools_registration_test.go— Tool registrationinternal/vsrpc/models_test.go— Model serializationinternal/vsrpc/stream_test.go— Stream closeinternal/grpcserver/ai_errors_test.go— AI error mappingWave 4: Azure API, Auth, Infra, Utilities
pkg/azapi/deployments_unit_test.go— Deployment outputs, provisioning statespkg/auth/claims_test.go— Token claims, login scopes, cachepkg/infra/util_test.go— Resource ID utilspkg/infra/deployment_manager_test.go— Deployment managerpkg/infra/provisioning/deployment_additional_test.go— Parameter typespkg/infra/provisioning/provisioning_test.go— Provider parsingpkg/infra/provisioning/options_test.go— Options validationpkg/containerapps/container_app_additional_test.go— API version policypkg/tools/kubectl/kubectl_additional_test.go— Kubectl CLIpkg/tools/kubectl/util_additional_test.go— Resource parsingpkg/tools/kubectl/models_additional_test.go— Model serializationpkg/cmdsubst/cmdsubst_additional_test.go— Command substitutionpkg/convert/util_additional_test.go— Conversion helperspkg/custommaps/with_order_map_additional_test.go— Ordered mapsWave 5: Extensions
extensions/.../internal/helpers_coverage_test.go— ToPascalCase, checksumsextensions/.../internal/user_friendly_error_test.go— Error typesextensions/.../internal/models/extension_schema_coverage_test.go— Schema parsingpkg/azdext/envelope_coverage_test.go— Service envelopespkg/azdext/run_coverage_test.go— Error types, versionpkg/azsdk/storage/storage_blob_client_coverage_test.go— Blob clientPrior Commits (before waves)
pkg/syncmap/syncmap_test.go— Map operations + concurrencypkg/contracts/contracts_test.go— Contract types + RFC3339Timepkg/azureutil/resource_group_test.go— ResourceNotFoundErrorinternal/tracing/resource/resource_test.go— CI env detectionpkg/ux/ux_additional_test.go— ConsoleWidth, Printer, Canvaspkg/keyvault/keyvault_test.go— Parse/validate AKVS referencespkg/devcentersdk/devcentersdk_test.go— Resource ID parsingpkg/azdo/azdo_additional_test.go— Pipeline variable assemblyeng/scripts/Get-CICoverageReport.ps1— CI coverage download toolTesting
All tests pass locally:
Resolves #7295