Skip to content

Commit f298605

Browse files
committed
docs: 📄 update acceptance testing guidelines
- Add section on getting started with `AcceptanceTest` flag - Clarify timeout behavior and add disable option (-1) - Update environment variables table (remove ORG_URL, clarify others) - Add minimal test scaffold example with all required fields - Update step-by-step instructions to use new poll utility - Document new file writing helpers with temp directory usage - Improve examples and clarify when to use acceptance tests
1 parent 3335c55 commit f298605

1 file changed

Lines changed: 33 additions & 5 deletions

File tree

‎TESTING.md‎

Lines changed: 33 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,15 @@ Use this pattern as a seed and add more mocks only as the code under test demand
8080

8181
Acceptance tests (`*_acc_test.go`) run against a live Azure DevOps organization using the harness under `internal/test`. They are opt-in and should only be used when unit tests and mocks cannot provide enough confidence.
8282

83+
### Getting started (`AcceptanceTest`)
84+
85+
`inttest.Test` executes a `TestCase` using a `TestContext` created by the harness. The `TestCase.AcceptanceTest` boolean controls which context is used:
86+
87+
- `AcceptanceTest: true` (recommended for all `*_acc_test.go`): uses `inttest.NewAccTestContext(t)` which reads the required `AZDO_ACC_*` environment variables and creates real SDK clients for the configured organization.
88+
- `AcceptanceTest: false`: uses `inttest.NewTestContext(t)` which generates random placeholder values for `Org`, `PAT`, and `Project`. This is only useful for hermetic unit tests that want a lightweight `util.CmdContext` implementation; it is not suitable for live Azure DevOps operations.
89+
90+
For acceptance tests, always set `AcceptanceTest: true`. Otherwise the harness will still be gated by `AZDO_ACC_TEST=1`, but the generated placeholder org/token values will typically cause confusing failures once your steps attempt real API calls.
91+
8392
### When to add an acceptance test
8493

8594
- You need to verify a workflow/command (e.g., modifying security permissions) against real data.
@@ -92,20 +101,39 @@ Acceptance tests (`*_acc_test.go`) run against a live Azure DevOps organization
92101
| ------------------ | ------------------------------------------------------------------------------------------------------ |
93102
| `AZDO_ACC_TEST=1` | Enables acceptance tests. Without it, `inttest.Test` skips all steps. |
94103
| `AZDO_ACC_ORG` | Organization name used for the session. |
95-
| `AZDO_ACC_ORG_URL` | Optional explicit organization URL; defaults to `https://dev.azure.com/<org>`. |
96104
| `AZDO_ACC_PAT` | Personal Access Token with the scopes required by the test steps. |
97105
| `AZDO_ACC_PROJECT` | Project name used by acceptance tests that operate on project-scoped resources. |
98-
| `AZDO_ACC_TIMEOUT` | Optional override for the default 60 s timeout. Accepts Go durations (`45s`, `2m`) or integer seconds. |
106+
| `AZDO_ACC_TIMEOUT` | Optional override for the default 240 s timeout. Accepts Go durations (`45s`, `2m`) or integer seconds. Use `-1` to disable timeouts. |
99107

100108
### Step-by-step skeleton
101109

102110
1. Place the test in the command package with the `_acc_test.go` suffix.
103-
2. Wrap your steps in `inttest.Test(t, inttest.TestCase{ Steps: []inttest.Step{ ... } })`.
111+
2. Wrap your steps in `inttest.Test(t, inttest.TestCase{ AcceptanceTest: true, Steps: []inttest.Step{ ... } })`.
104112
3. **PreRun**: create or seed live resources (groups, repositories, permissions) using `ctx.ClientFactory()`.
105113
4. **Run**: construct the command options and call the command’s `run...` helper directly (e.g., `return runCommand(ctx, opts)`).
106-
5. **Verify**: use `inttest.Poll` to wait for eventual consistency and assert desired state.
114+
5. **Verify**: use `internal/util/poll.go` (`pollutil.Poll(ctx.Context(), ...)`) to wait for eventual consistency and assert desired state.
107115
6. **PostRun**: delete or revert all resources you created; aggregate cleanup errors with `errors.Join`.
108116

117+
Minimal scaffold:
118+
119+
```go
120+
inttest.Test(t, inttest.TestCase{
121+
AcceptanceTest: true,
122+
PreCheck: func() error {
123+
// Validate required inputs (e.g., ensure AZDO_ACC_PROJECT is set when needed).
124+
return nil
125+
},
126+
Steps: []inttest.Step{
127+
{
128+
PreRun: func(ctx inttest.TestContext) error { return nil },
129+
Run: func(ctx inttest.TestContext) error { return nil },
130+
Verify: func(ctx inttest.TestContext) error { return nil },
131+
PostRun: func(ctx inttest.TestContext) error { return nil },
132+
},
133+
},
134+
})
135+
```
136+
109137
Example shell to execute a single acceptance test:
110138
```bash
111139
AZDO_ACC_TEST=1 \
@@ -119,7 +147,7 @@ Acceptance tests are not run in CI; execute them manually before publishing feat
119147

120148
- `inttest.TestContext` now exposes `Project()` alongside `Org`, `OrgUrl`, and `PAT`. Set `AZDO_ACC_PROJECT` when a test needs to target a specific project and fail fast in `PreRun` if it is missing.
121149
- Use `TestContext.SetValue(key, value)`/`Value(key)` to propagate data across `PreRun`, `Run`, `Verify`, and `PostRun` without relying on package-level variables. Keys can be simple strings or typed aliases; mimic `context.Context` usage.
122-
- The helper `inttest.WriteTestFile(path, contents)` creates or truncates files with `0600` permissions and ensures parent directories exist, which is useful for acceptance tests that need temporary credentials or certificates.
150+
- The helpers `inttest.WriteTestFile(t, contents)` and `inttest.WriteTestFileWithName(t, filename, contents)` create files with `0600` permissions under `t.TempDir()`, which is useful for acceptance tests that need temporary credentials or certificates.
123151

124152
### Updating Mocks
125153

0 commit comments

Comments
 (0)