diff --git a/docs/wiki/API-Endpoints.md b/docs/wiki/API-Endpoints.md new file mode 100644 index 0000000..3d460a1 --- /dev/null +++ b/docs/wiki/API-Endpoints.md @@ -0,0 +1,313 @@ +# API Endpoints + +The DataStack Platform API powers an internal employee management platform used by the HR Portal, Mobile App, Payroll Service, and Analytics Dashboard. + +All endpoints are versioned under `/v1` to support graceful deprecation. + +--- + +## Base URL + +``` +http://localhost:8000 +``` + +--- + +## Interactive Documentation + +When the server is running, interactive docs are available at: + +| URL | Format | +|-----|--------| +| `http://localhost:8000/docs` | Swagger UI (interactive) | +| `http://localhost:8000/redoc` | ReDoc (read-only) | +| `http://localhost:8000/openapi.json` | Raw OpenAPI spec (JSON) | + +--- + +## Endpoints Overview + +| Method | Path | Summary | Tag | +|--------|------|---------|-----| +| GET | `/health` | Health check | ops | +| GET | `/v1/employees` | List employees | employees | +| POST | `/v1/employees` | Onboard a new employee | employees | +| GET | `/v1/employees/{employee_id}` | Get an employee by ID | employees | +| PATCH | `/v1/employees/{employee_id}` | Update an employee | employees | +| GET | `/v1/audit` | View API call mismatch audit log | ops | +| DELETE | `/v1/audit` | Clear the audit log | ops | + +--- + +## Health Check + +### `GET /health` + +Returns the current health status of the service. + +**Response (200):** +```json +{ + "status": "ok", + "version": "1.0.0" +} +``` + +**Example:** +```bash +curl -s http://localhost:8000/health | python -m json.tool +``` + +--- + +## Employee Endpoints + +### `GET /v1/employees` -- List Employees + +List all employees, optionally filtered by department or status. Used by the HR Portal for the employee directory and by the Analytics Dashboard for headcount reporting. + +**Query Parameters:** + +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `department` | string | No | Filter by department (e.g., "Engineering", "Sales", "HR") | +| `status` | string | No | Filter by status: `active`, `on_leave`, `terminated` | + +**Response (200):** +```json +{ + "total": 5, + "employees": [ + { + "id": 1, + "first_name": "Sarah", + "last_name": "Chen", + "email": "sarah.chen@datastack.com", + "department": "Engineering", + "title": "Senior Backend Engineer", + "role": "member", + "status": "active", + "hired_at": "2022-03-15T00:00:00Z", + "bio": null + } + ] +} +``` + +**Examples:** +```bash +# List all employees +curl -s http://localhost:8000/v1/employees | python -m json.tool + +# Filter by department +curl -s "http://localhost:8000/v1/employees?department=Engineering" | python -m json.tool + +# Filter by status +curl -s "http://localhost:8000/v1/employees?status=on_leave" | python -m json.tool +``` + +--- + +### `POST /v1/employees` -- Onboard a New Employee + +Create a new employee record during onboarding. Called by the HR Portal when a new hire accepts their offer. + +**Request Body:** + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `first_name` | string | Yes | Employee first name | +| `last_name` | string | Yes | Employee last name | +| `email` | string | Yes | Corporate email address | +| `department` | string | Yes | Department (e.g., Engineering, Sales, HR) | +| `title` | string | Yes | Job title | +| `role` | string | No | Access role: `member` (default), `manager`, `admin` | + +**Response (201):** +```json +{ + "id": 6, + "first_name": "Alex", + "last_name": "Rivera", + "email": "alex.rivera@datastack.com", + "department": "Product", + "title": "Product Manager", + "role": "member", + "status": "active", + "hired_at": "2026-02-24T02:00:00+00:00", + "bio": null +} +``` + +**Example:** +```bash +curl -s -X POST http://localhost:8000/v1/employees \ + -H "Content-Type: application/json" \ + -d '{ + "first_name": "Alex", + "last_name": "Rivera", + "email": "alex.rivera@datastack.com", + "department": "Product", + "title": "Product Manager" + }' | python -m json.tool +``` + +**Error (422):** Returned when required fields are missing or invalid. + +--- + +### `GET /v1/employees/{employee_id}` -- Get Employee by ID + +Retrieve a single employee by their unique ID. Used by the Mobile App for the profile screen and by the Payroll Service to look up employee details. + +**Path Parameters:** + +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `employee_id` | integer | Yes | Unique employee identifier | + +**Response (200):** +```json +{ + "id": 1, + "first_name": "Sarah", + "last_name": "Chen", + "email": "sarah.chen@datastack.com", + "department": "Engineering", + "title": "Senior Backend Engineer", + "role": "member", + "status": "active", + "hired_at": "2022-03-15T00:00:00Z", + "bio": null +} +``` + +**Error (404):** Returned when the employee ID does not exist. + +**Example:** +```bash +curl -s http://localhost:8000/v1/employees/1 | python -m json.tool +``` + +--- + +### `PATCH /v1/employees/{employee_id}` -- Update Employee + +Update an employee's fields (partial update). Used by the HR Portal to change titles, departments, or status (e.g., when someone goes on leave or gets promoted). + +**Path Parameters:** + +| Name | Type | Required | Description | +|------|------|----------|-------------| +| `employee_id` | integer | Yes | Unique employee identifier | + +**Request Body (partial -- only include fields to update):** + +| Field | Type | Description | +|-------|------|-------------| +| `first_name` | string | Employee first name | +| `last_name` | string | Employee last name | +| `email` | string | Corporate email address | +| `department` | string | Department | +| `title` | string | Job title | +| `role` | string | Access role: `member`, `manager`, `admin` | +| `status` | string | Employment status: `active`, `on_leave`, `terminated` | + +**Response (200):** Returns the full updated employee object. + +**Error (404):** Returned when the employee ID does not exist. + +**Examples:** +```bash +# Promote an employee +curl -s -X PATCH http://localhost:8000/v1/employees/3 \ + -H "Content-Type: application/json" \ + -d '{"title": "Senior Account Executive", "role": "manager"}' | python -m json.tool + +# Put employee on leave +curl -s -X PATCH http://localhost:8000/v1/employees/1 \ + -H "Content-Type: application/json" \ + -d '{"status": "on_leave"}' | python -m json.tool +``` + +--- + +## Audit Endpoints + +### `GET /v1/audit` -- View Audit Log + +Return all API call mismatches recorded by the middleware. Used by the platform team to identify clients using stale field names or calling nonexistent endpoints. + +**Response (200):** +```json +{ + "total_mismatches": 2, + "entries": [ + { + "timestamp": 1708000000.0, + "method": "POST", + "path": "/v1/employees", + "issue_type": "unknown_field", + "details": "Client sent field `username` but the schema only expects: ...", + "client_ip": "127.0.0.1" + } + ] +} +``` + +**Example:** +```bash +curl -s http://localhost:8000/v1/audit | python -m json.tool +``` + +--- + +### `DELETE /v1/audit` -- Clear Audit Log + +Clear all recorded mismatches. + +**Response (200):** +```json +{ + "status": "cleared" +} +``` + +**Example:** +```bash +curl -s -X DELETE http://localhost:8000/v1/audit | python -m json.tool +``` + +--- + +## Seed Data + +The API starts with 5 pre-loaded employees: + +| ID | Name | Department | Title | Role | Status | +|----|------|------------|-------|------|--------| +| 1 | Sarah Chen | Engineering | Senior Backend Engineer | member | active | +| 2 | Marcus Johnson | Engineering | Engineering Manager | manager | active | +| 3 | Priya Patel | Sales | Account Executive | member | active | +| 4 | James Wright | HR | HR Business Partner | admin | active | +| 5 | Lisa Kim | Engineering | Frontend Engineer | member | on_leave | + +The in-memory store resets every time the server restarts. + +--- + +## Authentication + +No authentication is currently required. JWT auth is planned for a future release. + +--- + +## Common Error Codes + +| Code | Meaning | +|------|---------| +| 200 | Success | +| 201 | Created (new employee onboarded) | +| 404 | Resource not found (bad employee ID, nonexistent endpoint) | +| 422 | Validation error (missing required fields, wrong types) | diff --git a/docs/wiki/Architecture-Overview.md b/docs/wiki/Architecture-Overview.md new file mode 100644 index 0000000..0d38a73 --- /dev/null +++ b/docs/wiki/Architecture-Overview.md @@ -0,0 +1,129 @@ +# Architecture Overview + +Docs Drift Guard implements a **three-layer defense** to keep API code, documentation, and client integrations in sync. Each layer catches a different type of mismatch and feeds results into Slack for team visibility. + +--- + +## The Three Layers + +``` + +-----------------------+ + | API Source Code | + | (app/main.py) | + +-----------+-----------+ + | + generates live OpenAPI spec + | + +-----------v-----------+ + +-------> OpenAPI Spec <-------+ + | | (/openapi.json) | | + | +-----------+-----------+ | + | | | + Layer 1: Drift Layer 2: Runtime Layer 3: Contract + Detection Call Guard Tests + | | | + compares to validates every replays stale + stored baseline incoming request client scenarios + | | | + v v v + +------+------+ +-------+-------+ +-------+-------+ + | drift_guard/ | | app/middleware | | tests/ | + | guard.py | | | | test_contract | + | check cmd | | APICallGuard | | | + +--------------+ | Middleware | +-------+-------+ + | +-------+-------+ | + | | | + +--------+-----------+--------------------+ + | + v + Slack Notifications + + Audit Reports + + Generated Docs +``` + +--- + +## Layer 1: Code-vs-Docs Drift Detection + +**Module:** `drift_guard/guard.py` (check command) + `drift_guard/drift_detect.py` + +**Problem:** A developer changes the API code (adds a field, removes an endpoint) but forgets to update the documentation. + +**How it works:** + +1. A baseline OpenAPI spec is saved to `artifacts/openapi_baseline.json` +2. When `check` is run, it fetches the live OpenAPI spec from the running server +3. The two specs are compared field-by-field and endpoint-by-endpoint +4. Changes are classified by severity (GREEN / YELLOW / RED) +5. `docs/api_reference.md` is auto-regenerated from the live spec +6. A Slack notification is sent for YELLOW and RED changes + +**See:** [Drift Detection](Drift-Detection.md) for full details. + +--- + +## Layer 2: Runtime API Call Guard + +**Module:** `app/middleware.py` + +**Problem:** Clients built from stale documentation send requests with wrong field names or call endpoints that no longer exist. + +**How it works:** + +1. A FastAPI middleware intercepts every incoming request +2. It checks the request path against the OpenAPI spec +3. For POST/PUT/PATCH requests, it compares request body fields against the schema +4. Mismatches are logged to an in-memory audit log (never blocking the request) +5. An `X-API-Mismatch: true` header is added to responses when issues are found +6. The audit log is queryable via `GET /v1/audit` + +**See:** [Runtime Call Guard](Runtime-Call-Guard.md) for full details. + +--- + +## Layer 3: Contract Tests + +**Module:** `tests/test_contract.py` + +**Problem:** Over time, multiple clients accumulate mismatches against the API. You need a systematic way to find and report all of them. + +**How it works:** + +1. A pytest suite makes real HTTP calls against the running server +2. Tests cover correct calls, stale field names, and nonexistent endpoints +3. Each test validates response codes, field names, and types +4. The audit log is checked to confirm the middleware caught all issues +5. Results can be combined with the `audit` command for a full report + +**See:** [Contract Tests](Contract-Tests.md) for full details. + +--- + +## Data Flow + +### Artifacts Generated + +| Artifact | Generated By | Description | +|----------|-------------|-------------| +| `artifacts/openapi_baseline.json` | `baseline` / `check` commands | Stored OpenAPI snapshot for comparison | +| `artifacts/drift_summary.md` | `check` command | Human-readable change summary with severity | +| `artifacts/audit_report.md` | `audit` command | Runtime mismatch report | +| `docs/api_reference.md` | `baseline` / `check` commands | Auto-generated Markdown API reference | + +### Notification Flow + +1. **Drift detected** (YELLOW/RED) --> Slack notification with change summary, impact, docs status, and action items +2. **Runtime mismatches found** --> Slack notification with mismatch details and context +3. **GREEN drift** --> No Slack notification (only local output to avoid noise) + +--- + +## Key Design Decisions + +1. **Observe-only middleware:** The Runtime Call Guard never blocks requests. It logs issues and adds a response header. This makes it safe to deploy in production in observe mode before enforcing. + +2. **Severity-based automation:** GREEN changes auto-update docs. YELLOW/RED changes require human review. This balances automation with safety. + +3. **Single source of truth:** The live OpenAPI spec (generated by FastAPI from code) is always the source of truth. Documentation is derived from it, never the other way around. + +4. **In-memory audit log:** The audit log resets on server restart. This is intentional for a demo but could be backed by a database in production. diff --git a/docs/wiki/CI-CD-Integration.md b/docs/wiki/CI-CD-Integration.md new file mode 100644 index 0000000..bae54c0 --- /dev/null +++ b/docs/wiki/CI-CD-Integration.md @@ -0,0 +1,168 @@ +# CI/CD Integration + +Docs Drift Guard includes a GitHub Actions workflow that automatically checks for API drift, runs contract tests, and generates audit reports on every push and pull request. + +--- + +## Overview + +The workflow file is at `.github/workflows/drift-guard.yml`. It runs on: +- Every **push** to `main` +- Every **pull request** targeting `main` + +--- + +## What the CI Pipeline Does + +The pipeline runs three phases in sequence: + +### Phase 1: Code-vs-Docs Drift Detection + +1. Starts the API server (`uvicorn app.main:app`) +2. Waits for it to be healthy (`curl /health`) +3. If no baseline exists, saves one (`drift_guard baseline`) +4. Runs `drift_guard check` to compare the live spec against the baseline +5. Captures the exit code and severity level + +**Severity handling in CI:** + +| Severity | CI Behavior | +|----------|-------------| +| GREEN (exit 10) | Auto-updates baseline. Adds a GitHub notice. **Passes.** | +| YELLOW (exit 11) | Adds a GitHub warning. **Fails** -- requires PR review. | +| RED (exit 2) | Adds a GitHub error. **Fails** -- breaking change, must be resolved. | +| None (exit 0) | No drift. **Passes.** | + +### Phase 2: Contract Tests + +1. Runs `pytest tests/test_contract.py -v --tb=short` +2. Captures the exit code +3. If tests fail, adds a GitHub warning + +### Phase 3: Audit Summary + +1. Runs `drift_guard audit` to fetch the runtime mismatch log +2. Captures the exit code + +### Final Verdict + +The pipeline evaluates all three phases together: + +| Condition | Result | +|-----------|--------| +| YELLOW or RED drift detected | **FAIL** | +| Contract tests failed | **FAIL** | +| Everything else | **PASS** | + +On failure, the pipeline outputs instructions: +``` +To fix: review the drift summary, run 'python -m drift_guard.guard baseline', +then commit the updated artifacts/openapi_baseline.json and docs/api_reference.md +``` + +--- + +## Artifacts + +The pipeline uploads these files as build artifacts (viewable in the GitHub Actions UI): + +| Artifact | Description | +|----------|-------------| +| `artifacts/drift_summary.md` | Change summary with severity classification | +| `artifacts/audit_report.md` | Runtime mismatch report | + +These are uploaded even if some steps fail (`if: always()`), so you can always review what happened. + +--- + +## Enabling Slack in CI + +To receive Slack notifications from CI runs: + +1. Go to your GitHub repository settings +2. Navigate to **Settings > Secrets and variables > Actions** +3. Add a new secret: `SLACK_WEBHOOK_URL` with your Slack webhook URL + +The drift guard commands will automatically use this secret to send notifications. + +--- + +## How the Failing Check Looks on a PR + +When drift is detected on a pull request: + +1. The PR will show a **failing check** with a red X +2. The check summary will include: + - The severity level (GREEN/YELLOW/RED) + - A message explaining what changed + - Instructions on how to fix it +3. Build artifacts (drift summary, audit report) are downloadable from the Actions tab + +**To fix a failing PR:** +1. Review `artifacts/drift_summary.md` (download from the Actions artifact) +2. Run `python -m drift_guard.guard baseline` locally +3. Commit the updated `artifacts/openapi_baseline.json` and `docs/api_reference.md` +4. Push to the PR branch + +--- + +## Workflow Configuration Reference + +```yaml +name: Drift Guard + +on: + push: + branches: [main] + pull_request: + branches: [main] + +jobs: + drift-check: + name: Detect API drift & contract violations + runs-on: ubuntu-latest + steps: + - Checkout code + - Set up Python 3.12 + - Install dependencies (pip install -r requirements.txt) + - Start API server (uvicorn, wait for health check) + - Save baseline (if none exists) + - Check for API drift (capture severity) + - Run contract tests + - Fetch audit log + - Upload artifacts (drift_summary.md, audit_report.md) + - Final result (pass/fail based on all phases) +``` + +--- + +## Customizing the Workflow + +### Changing the Python version + +Update the `python-version` in the workflow: +```yaml +- name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" # Change this +``` + +### Running on different branches + +Update the trigger branches: +```yaml +on: + push: + branches: [main, develop] + pull_request: + branches: [main, develop] +``` + +### Adding additional test suites + +Add a new step after the contract tests step: +```yaml +- name: Run additional tests + run: pytest tests/test_integration.py -v +``` diff --git a/docs/wiki/CLI-Reference.md b/docs/wiki/CLI-Reference.md new file mode 100644 index 0000000..a8bc624 --- /dev/null +++ b/docs/wiki/CLI-Reference.md @@ -0,0 +1,155 @@ +# CLI Reference + +The Docs Drift Guard CLI is the main interface for detecting drift, managing baselines, and generating audit reports. + +--- + +## Usage + +```bash +python -m drift_guard.guard [--url URL] +``` + +The CLI must be run from the repository root directory (where `artifacts/` and `docs/` are located). + +--- + +## Commands + +### `baseline` + +Save the current OpenAPI spec as the baseline and regenerate the Markdown API reference. + +```bash +python -m drift_guard.guard baseline [--url URL] +``` + +**What it does:** +1. Fetches the live OpenAPI spec from the running server +2. Saves it to `artifacts/openapi_baseline.json` with metadata (timestamp + machine name) +3. Generates `docs/api_reference.md` from the spec + +**Default URL:** `http://localhost:8000/openapi.json` + +**Exit codes:** +| Code | Meaning | +|------|---------| +| 0 | Success | +| 1 | Runtime error (server down, network issue) | + +**Example:** +```bash +# Using default URL +python -m drift_guard.guard baseline + +# Using a custom URL +python -m drift_guard.guard baseline --url http://staging:8000/openapi.json +``` + +--- + +### `check` + +Compare the live OpenAPI spec to the saved baseline, classify changes by severity, regenerate docs, and notify Slack. + +```bash +python -m drift_guard.guard check [--url URL] +``` + +**What it does:** +1. Loads the baseline from `artifacts/openapi_baseline.json` +2. Fetches the live OpenAPI spec from the running server +3. Compares the two specs and classifies each change by severity +4. Regenerates `docs/api_reference.md` (Tier 1 -- always safe) +5. Writes a change summary to `artifacts/drift_summary.md` +6. Updates the baseline with the current spec +7. Sends a Slack notification for YELLOW and RED changes +8. Exits with a severity-specific code + +**Default URL:** `http://localhost:8000/openapi.json` + +**Exit codes:** +| Code | Meaning | +|------|---------| +| 0 | No drift detected | +| 1 | Runtime error (server down, missing baseline) | +| 2 | RED -- Breaking drift detected | +| 10 | GREEN -- Non-breaking drift detected | +| 11 | YELLOW -- Potentially breaking drift detected | + +**Example:** +```bash +python -m drift_guard.guard check +``` + +**Output includes:** +- Severity badge and label +- One-line summary (e.g., "1 breaking, 2 non-breaking") +- Bullet list of all changes grouped by severity +- Paths of regenerated files + +--- + +### `audit` + +Fetch the runtime mismatch audit log from the server, generate a report, and notify Slack. + +```bash +python -m drift_guard.guard audit [--url URL] +``` + +**What it does:** +1. Fetches the audit log from `GET /v1/audit` +2. Summarizes all mismatches +3. Identifies repeated mismatch patterns ("Runtime Drift Signals") +4. Writes a report to `artifacts/audit_report.md` +5. Appends a "Runtime Mismatches" section to `docs/api_reference.md` +6. Sends a Slack notification with the summary +7. Exits with code 3 if mismatches were found + +**Default URL:** `http://localhost:8000/v1/audit` + +**Exit codes:** +| Code | Meaning | +|------|---------| +| 0 | No runtime mismatches recorded | +| 1 | Runtime error (server down) | +| 3 | Runtime mismatches found | + +**Example:** +```bash +python -m drift_guard.guard audit +``` + +--- + +## The `--url` Flag + +All commands accept an optional `--url` flag to override the default server URL: + +```bash +# Point to a different server +python -m drift_guard.guard check --url http://staging-server:8000/openapi.json + +# Point audit to a different server +python -m drift_guard.guard audit --url http://staging-server:8000/v1/audit +``` + +**Default URLs:** +- `baseline` and `check`: `http://localhost:8000/openapi.json` +- `audit`: `http://localhost:8000/v1/audit` + +--- + +## Exit Code Summary + +| Code | Meaning | Command(s) | +|------|---------|------------| +| 0 | Success / no drift / no mismatches | All | +| 1 | Runtime error (server down, missing baseline, etc.) | All | +| 2 | RED -- Breaking drift detected | `check` | +| 3 | Runtime mismatches found | `audit` | +| 10 | GREEN -- Non-breaking drift detected | `check` | +| 11 | YELLOW -- Potentially breaking drift detected | `check` | + +These exit codes are used by CI pipelines and pre-commit hooks to determine whether to pass, warn, or fail. diff --git a/docs/wiki/Contract-Tests.md b/docs/wiki/Contract-Tests.md new file mode 100644 index 0000000..2ee8be7 --- /dev/null +++ b/docs/wiki/Contract-Tests.md @@ -0,0 +1,165 @@ +# Contract Tests + +Layer 3 of the Docs Drift Guard system uses a pytest suite to systematically verify that the API behaves as documented and to identify all client-vs-code mismatches. + +--- + +## Overview + +The contract tests (`tests/test_contract.py`) make **real HTTP calls** against the running API server. They verify: + +- Response status codes match expectations +- Response bodies contain the expected fields +- Field types are correct +- Stale client patterns are properly caught +- The middleware audit log records all mismatches + +--- + +## Running the Tests + +### Prerequisites + +The API server must be running: + +```bash +uvicorn app.main:app --reload +``` + +### Run All Contract Tests + +```bash +pytest tests/test_contract.py -v +``` + +### Run the Full Cleanup Script + +The cleanup script runs contract tests AND generates an audit report: + +```bash +bash scripts/demo_cleanup.sh +``` + +This script: +1. Runs `pytest tests/test_contract.py -v --tb=short` (allows failures) +2. Runs `python -m drift_guard.guard audit` (summarizes runtime mismatches) + +--- + +## Test Suites + +### `TestHealthContract` + +Verifies the health check endpoint: + +| Test | What It Checks | +|------|---------------| +| `test_health_returns_status_and_version` | `GET /health` returns 200 with `status` and `version` fields | + +### `TestGetEmployeeContract` + +Verifies `GET /v1/employees/{id}`: + +| Test | What It Checks | +|------|---------------| +| `test_get_employee_returns_correct_fields` | Response contains all expected fields (`id`, `first_name`, `last_name`, `email`, `department`, `title`, `role`, `status`, `hired_at`) | +| `test_get_employee_does_not_return_legacy_name_field` | Response does NOT contain a legacy `name` field (it should be `first_name` + `last_name`) | +| `test_get_nonexistent_employee_returns_404` | Requesting a nonexistent ID returns 404 | +| `test_get_employee_field_types` | All fields have the correct types (`id` is int, strings are str, etc.) | + +### `TestListEmployeesContract` + +Verifies `GET /v1/employees` (list): + +| Test | What It Checks | +|------|---------------| +| `test_list_returns_total_and_employees` | Response contains `total` (int) and `employees` (list) with at least 1 entry | +| `test_filter_by_department` | `?department=Engineering` filter works correctly | +| `test_filter_by_status` | `?status=on_leave` filter works correctly | + +### `TestCreateEmployeeContract` + +Verifies `POST /v1/employees`: + +| Test | What It Checks | +|------|---------------| +| `test_onboard_employee_with_correct_fields` | Creating with valid fields returns 201 with correct data | +| `test_onboard_with_stale_single_name_field` | Using legacy `name` field instead of `first_name`/`last_name` returns 422 | +| `test_onboard_missing_required_fields` | Sending only `email` without other required fields returns 422 | + +### `TestUpdateEmployeeContract` + +Verifies `PATCH /v1/employees/{id}`: + +| Test | What It Checks | +|------|---------------| +| `test_promote_employee` | Updating `title` and `role` returns 200 with updated values | +| `test_put_employee_on_leave` | Updating `status` to `on_leave` returns 200 | +| `test_update_nonexistent_employee` | Patching a nonexistent ID returns 404 | + +### `TestAPITruthLoop` + +The **Full API Truth Loop** test (Tier 0) -- proves the complete chain from code to OpenAPI to runtime: + +| Test | What It Checks | +|------|---------------| +| `test_create_employee_contract` | POST call validates status, schema, field types, no mismatch header, and cross-checks the live OpenAPI spec | + +This test: +1. Creates an employee with correct fields +2. Verifies the response has exactly the expected fields (no more, no less) +3. Checks all field types +4. Confirms no `X-API-Mismatch` header was added +5. Fetches `/openapi.json` and verifies the endpoint is defined + +### `TestAuditEndpoint` + +Verifies the audit log endpoint: + +| Test | What It Checks | +|------|---------------| +| `test_audit_endpoint_exists` | `GET /v1/audit` returns 200 with `total_mismatches` and `entries` | +| `test_bad_call_appears_in_audit` | After sending a bad POST, the audit log has recorded the mismatch | + +### `TestStaleClientScenarios` + +Simulates real-world stale client calls: + +| Test | What It Checks | +|------|---------------| +| `test_old_users_endpoint` | Old mobile app calling `/v1/users/1` gets 404 or 405 | +| `test_old_teams_endpoint` | Analytics dashboard calling `/v1/teams` gets 404 or 405 | + +--- + +## Writing New Contract Tests + +To add a new contract test: + +1. Add your test class or method to `tests/test_contract.py` +2. Use the helper functions `_get()`, `_post()`, `_patch()` for HTTP calls +3. Use `_has_mismatch_header()` to check if the middleware flagged the request +4. All tests target `http://localhost:8000` (the `BASE_URL` constant) + +Example: + +```python +class TestMyNewEndpoint: + def test_new_feature(self) -> None: + resp = _get("/v1/my-endpoint") + assert resp.status_code == 200 + data = resp.json() + assert "expected_field" in data +``` + +--- + +## Relationship to Other Layers + +The contract tests complement the other guard layers: + +- **Layer 1 (Drift Detection)** catches schema-level changes at build time +- **Layer 2 (Runtime Guard)** catches bad calls one-at-a-time in production +- **Layer 3 (Contract Tests)** systematically replays all known scenarios and proves the complete truth loop + +Running all three together gives you full confidence that code, docs, and clients are aligned. diff --git a/docs/wiki/Demo-Walkthrough.md b/docs/wiki/Demo-Walkthrough.md new file mode 100644 index 0000000..25e4d96 --- /dev/null +++ b/docs/wiki/Demo-Walkthrough.md @@ -0,0 +1,169 @@ +# Demo Walkthrough + +This is a step-by-step guide to running the full Docs Drift Guard demo, suitable for presentations, Loom recordings, or team onboarding. + +--- + +## Prerequisites + +1. Repository cloned and dependencies installed (see [Getting Started](Getting-Started.md)) +2. Virtual environment activated: `source .venv/bin/activate` +3. API server running in a separate terminal: `uvicorn app.main:app --reload` + +--- + +## Part 1: Show the Stale Manual Docs + +Start by showing the **hand-written documentation** that is intentionally out of date: + +```bash +cat docs/api_manual.md +``` + +**Key things to point out:** +- The manual says the endpoint is `/v1/users/{user_id}` but the API uses `/v1/employees/{employee_id}` +- The manual says the field is `username` but the API uses `first_name` and `last_name` +- The manual is missing many endpoints and fields that actually exist +- The manual was "last updated June 2025 by @sarah" + +This is the problem: **manual docs get stale**, and nobody notices until clients break. + +--- + +## Part 2: Baseline + Drift Detection + +### Save the Baseline + +```bash +python -m drift_guard.guard baseline +``` + +This captures the current API state. Show the generated files: +- `artifacts/openapi_baseline.json` -- The stored snapshot +- `docs/api_reference.md` -- The auto-generated docs (always accurate!) + +### Introduce Drift + +```bash +bash scripts/demo_drift.sh +``` + +This modifies `app/main.py` to: +- Add a `role` field to the User model +- Add a new `GET /v1/users` endpoint (list all users) + +If the server is running with `--reload`, it will pick up the changes automatically. + +### Detect the Drift + +```bash +python -m drift_guard.guard check +``` + +**What to show:** +- The severity classification (each change gets a GREEN/YELLOW/RED badge) +- The change summary (added endpoints, schema changes) +- The regenerated `docs/api_reference.md` +- The drift summary in `artifacts/drift_summary.md` +- The Slack notification (or the printed payload if no webhook is set) + +--- + +## Part 3: Bad Client Calls + Audit + +### Simulate Stale Client Calls + +```bash +bash scripts/demo_bad_client.sh +``` + +This sends API calls using the wrong field names from the stale manual docs: +1. `GET /health` -- Correct call (baseline) +2. `GET /v1/users/1` -- Correct call +3. `POST /v1/users` with `username` instead of `name` -- **Caught by middleware** +4. `POST /v1/users` with `username` and `department` -- **Multiple unknown fields** +5. `GET /v1/teams` -- **Nonexistent endpoint** + +**What to show:** +- The HTTP response codes +- The `X-API-Mismatch: true` header on bad responses + +### View the Audit Log + +```bash +curl -s http://localhost:8000/v1/audit | python -m json.tool +``` + +**What to show:** +- Each mismatch entry with method, path, issue type, and details +- The total mismatch count + +### Generate Audit Report + Slack Summary + +```bash +python -m drift_guard.guard audit +``` + +**What to show:** +- The mismatch summary +- Runtime Drift Signals (repeated patterns) +- The generated `artifacts/audit_report.md` +- The Slack notification + +--- + +## Part 4: Contract Tests + +### Run the Contract Test Suite + +```bash +pytest tests/test_contract.py -v +``` + +**What to show:** +- Tests that pass (correct API calls) +- Tests that fail (stale field names, nonexistent endpoints) +- The clear pass/fail output showing exactly what's broken + +### Or Run the Full Cleanup Script + +```bash +bash scripts/demo_cleanup.sh +``` + +This runs both the contract tests AND the audit report in sequence. + +--- + +## Full Flow (Copy-Paste Script) + +```bash +# Setup +source .venv/bin/activate +uvicorn app.main:app --reload # separate terminal + +# Part 1: Show the stale manual docs +cat docs/api_manual.md # note: says "username", API uses "first_name" + "last_name" + +# Part 2: Baseline + drift detection +python -m drift_guard.guard baseline +bash scripts/demo_drift.sh +python -m drift_guard.guard check # drift detected! + +# Part 3: Bad client calls + audit +bash scripts/demo_bad_client.sh # middleware catches mismatches +python -m drift_guard.guard audit # Slack summary + +# Part 4: Contract tests +pytest tests/test_contract.py -v # shows exactly what's broken +``` + +--- + +## Demo Tips + +1. **Use split terminals** -- Have the server running in one terminal and demo commands in another +2. **Show the Swagger UI** -- Open `http://localhost:8000/docs` in a browser to show the live API alongside the stale manual docs +3. **Compare docs side-by-side** -- Show `docs/api_manual.md` (stale) next to `docs/api_reference.md` (auto-generated, accurate) +4. **Highlight the problem** -- Emphasize that the manual docs say `username` but the API expects `first_name` and `last_name` +5. **Show the Slack output** -- Even without a real webhook, the printed payload shows what the notification would look like diff --git a/docs/wiki/Drift-Detection.md b/docs/wiki/Drift-Detection.md new file mode 100644 index 0000000..c27d9a4 --- /dev/null +++ b/docs/wiki/Drift-Detection.md @@ -0,0 +1,154 @@ +# Code-vs-Docs Drift Detection + +Layer 1 of the Docs Drift Guard system detects when a developer changes the API code but doesn't update the documentation. + +--- + +## How It Works + +### Step 1: Save a Baseline + +The `baseline` command fetches the current OpenAPI spec from the running server and saves it as a snapshot: + +```bash +python -m drift_guard.guard baseline +``` + +This creates/updates: +- `artifacts/openapi_baseline.json` -- The stored OpenAPI snapshot (with metadata) +- `docs/api_reference.md` -- Auto-generated Markdown API reference + +The baseline includes metadata about when and where it was saved: + +```json +{ + "_meta": { + "saved_at": "2026-02-24 02:41:49 UTC", + "saved_by_machine": "my-laptop.local" + }, + "spec": { ... } +} +``` + +### Step 2: Make Code Changes + +A developer modifies `app/main.py` -- adding fields, removing endpoints, changing schemas, etc. + +### Step 3: Detect the Drift + +The `check` command compares the live OpenAPI spec to the saved baseline: + +```bash +python -m drift_guard.guard check +``` + +This will: +1. Load the saved baseline from `artifacts/openapi_baseline.json` +2. Fetch the live OpenAPI spec from the running server +3. Compare the two specs using `drift_guard/drift_detect.py` +4. Classify each change by severity (GREEN / YELLOW / RED) +5. Regenerate `docs/api_reference.md` from the live spec +6. Write a change summary to `artifacts/drift_summary.md` +7. Update the baseline with the current spec +8. Send a Slack notification (for YELLOW and RED only) +9. Exit with a severity-specific exit code + +--- + +## What Gets Compared + +The drift detection engine (`drift_detect.py`) compares: + +### Endpoint-Level Changes +- **Added endpoints** -- New `METHOD /path` pairs not in the baseline +- **Removed endpoints** -- `METHOD /path` pairs that existed in the baseline but are gone + +### Schema-Level Changes +For every schema defined in `components/schemas`: +- **Added fields** -- New properties in the schema +- **Removed fields** -- Properties that were in the baseline but are gone +- **Type changes** -- A field's type changed (e.g., `string` to `integer`) +- **Required flag toggles** -- A field changed from optional to required or vice versa +- **Enum changes** -- Enum values added, removed, or the enum constraint itself added/removed + +--- + +## Severity Classification + +Each change is classified into one of three severity levels. See [Severity Classification](Severity-Classification.md) for the full reference. + +| Severity | Examples | Exit Code | +|----------|----------|-----------| +| GREEN (non-breaking) | New optional field, new endpoint, description change | 10 | +| YELLOW (potentially breaking) | Type change, field renamed, required flag toggled, enum modified | 11 | +| RED (breaking) | Required field added, field removed, endpoint removed | 2 | + +The overall severity of a drift check is the **maximum** severity across all detected changes. + +--- + +## Generated Artifacts + +### `artifacts/drift_summary.md` + +A human-readable change summary, for example: + +```markdown +# Drift Summary + +**Severity:** RED Breaking +**Detected:** 2026-02-24 02:45:00 UTC +**Machine:** my-laptop.local + +### RED Breaking (1) +- Removed endpoint: `GET /v1/users` + +### GREEN Non-breaking (1) +- New endpoint: `GET /v1/employees` + +--- + +`docs/api_reference.md` has been regenerated (Tier 1). + +## Proposed Narrative Updates (Human Review Required) +... +``` + +For YELLOW and RED changes, the summary includes a "Proposed Narrative Updates" section that flags changes needing human review of conceptual documentation, migration guides, or client SDK examples. + +### `docs/api_reference.md` + +Auto-generated Markdown API reference with: +- Endpoints overview table +- Detailed section per endpoint (parameters, request body, response schemas, example curl) +- Auto-generated footer with timestamp and machine name + +--- + +## Demo: Introducing Drift + +The `scripts/demo_drift.sh` script deliberately introduces API changes to demonstrate drift detection: + +```bash +# Save the current API as baseline +python -m drift_guard.guard baseline + +# Introduce code changes (adds 'role' field + GET /v1/users endpoint) +bash scripts/demo_drift.sh + +# Detect the drift +python -m drift_guard.guard check +# --> "Added endpoint: GET /v1/users" +# --> "Schema change: User gained field role" +# --> Regenerates docs/api_reference.md +# --> Posts to Slack (if SLACK_WEBHOOK_URL is set) +# --> Exits with code 2 +``` + +--- + +## Integration Points + +- **Pre-commit hook:** Runs drift detection before every commit that touches `app/` files. See [Pre-Commit Hooks](Pre-Commit-Hooks.md). +- **CI/CD:** The GitHub Action runs drift detection on every push and PR. See [CI/CD Integration](CI-CD-Integration.md). +- **Slack:** Notifications are sent for YELLOW and RED drift. See [Slack Notifications](Slack-Notifications.md). diff --git a/docs/wiki/Getting-Started.md b/docs/wiki/Getting-Started.md new file mode 100644 index 0000000..84e6c2e --- /dev/null +++ b/docs/wiki/Getting-Started.md @@ -0,0 +1,113 @@ +# Getting Started + +This guide walks you through setting up the Docs Drift Guard demo from scratch. + +--- + +## Prerequisites + +- **Python 3.11+** installed on your system +- **Git** for cloning the repository +- A terminal / shell environment + +--- + +## Installation + +### Option A: One-Command Bootstrap + +```bash +git clone && cd docs-drift-guard-demo +bash scripts/bootstrap.sh +``` + +The bootstrap script will: +1. Create a Python virtual environment (`.venv`) +2. Activate the virtual environment +3. Install all dependencies from `requirements.txt` + +### Option B: Manual Setup + +```bash +# 1. Clone the repository +git clone && cd docs-drift-guard-demo + +# 2. Create and activate a virtual environment +python3 -m venv .venv +source .venv/bin/activate + +# 3. Install dependencies +pip install -r requirements.txt +``` + +--- + +## Dependencies + +The project requires the following Python packages (defined in `requirements.txt`): + +| Package | Version | Purpose | +|---------|---------|---------| +| `fastapi` | >= 0.104.0 | Web framework for the API | +| `uvicorn` | >= 0.24.0 | ASGI server to run FastAPI | +| `requests` | >= 2.31.0 | HTTP client for Slack webhooks and API calls | +| `rich` | >= 13.7.0 | Rich text CLI output | +| `pytest` | >= 7.4.0 | Test runner for contract tests | +| `ruff` | >= 0.1.0 | Python linter | + +--- + +## Starting the API Server + +Start the FastAPI server with hot-reload enabled: + +```bash +source .venv/bin/activate +uvicorn app.main:app --reload +``` + +The server will start on `http://localhost:8000`. You can verify it's running: + +```bash +curl http://localhost:8000/health +# → {"status":"ok","version":"1.0.0"} +``` + +### Useful URLs + +| URL | Description | +|-----|-------------| +| `http://localhost:8000/health` | Health check endpoint | +| `http://localhost:8000/docs` | Interactive Swagger UI | +| `http://localhost:8000/redoc` | ReDoc API documentation | +| `http://localhost:8000/openapi.json` | Raw OpenAPI spec (JSON) | + +--- + +## Verifying the Setup + +Once the server is running, try these commands to verify everything works: + +```bash +# List all employees +curl -s http://localhost:8000/v1/employees | python -m json.tool + +# Get a specific employee +curl -s http://localhost:8000/v1/employees/1 | python -m json.tool + +# Save the OpenAPI baseline +python -m drift_guard.guard baseline + +# Run contract tests +pytest tests/test_contract.py -v +``` + +If all of the above succeed, your setup is complete and ready for the demo. + +--- + +## Next Steps + +- Read the [Architecture Overview](Architecture-Overview.md) to understand how the three layers work +- Follow the [Demo Walkthrough](Demo-Walkthrough.md) for a guided tour +- Check the [CLI Reference](CLI-Reference.md) for all available commands diff --git a/docs/wiki/Home.md b/docs/wiki/Home.md new file mode 100644 index 0000000..e9488c2 --- /dev/null +++ b/docs/wiki/Home.md @@ -0,0 +1,57 @@ +# Docs Drift Guard -- Documentation Wiki + +Welcome to the **Docs Drift Guard** documentation wiki. This project is a 3-layer automation that keeps your API, its documentation, and its consumers in sync -- and alerts your team on Slack when they aren't. + +--- + +## What Is Docs Drift Guard? + +Docs Drift Guard is a developer toolkit built for the **DataStack Platform API** that solves a common enterprise problem: API code changes faster than its documentation. This leads to stale docs, confused clients, and broken integrations. + +The system provides three complementary guard layers: + +| Layer | What It Catches | How | +|-------|----------------|-----| +| **Code-vs-Docs Drift** | Developer changes the API but forgets to update the docs | Compares live OpenAPI spec to a stored baseline | +| **Runtime Call Guard** | Clients call the API with wrong field names or hit nonexistent endpoints | FastAPI middleware validates every request against the OpenAPI spec | +| **Contract Tests** | Accumulated mismatches from clients built against stale docs | pytest suite replays "old client" calls and reports what breaks | + +All three feed into **Slack notifications** so the team knows what to fix. + +--- + +## Wiki Pages + +| Page | Description | +|------|-------------| +| [Getting Started](Getting-Started.md) | Installation, setup, and running the API server | +| [Architecture Overview](Architecture-Overview.md) | How the three guard layers work together | +| [Code-vs-Docs Drift Detection](Drift-Detection.md) | Layer 1: Detecting when API code diverges from docs | +| [Runtime Call Guard](Runtime-Call-Guard.md) | Layer 2: Middleware that catches bad client calls in real time | +| [Contract Tests](Contract-Tests.md) | Layer 3: Test suite that replays stale client scenarios | +| [CLI Reference](CLI-Reference.md) | Full reference for `python -m drift_guard.guard` commands | +| [API Endpoints](API-Endpoints.md) | DataStack Platform API endpoint documentation | +| [Slack Notifications](Slack-Notifications.md) | Configuring and understanding Slack alerts | +| [CI/CD Integration](CI-CD-Integration.md) | GitHub Actions workflow and automation | +| [Pre-Commit Hooks](Pre-Commit-Hooks.md) | Catching drift before code is pushed | +| [Demo Walkthrough](Demo-Walkthrough.md) | Step-by-step guide to running the full demo | +| [Project Structure](Project-Structure.md) | File layout and module responsibilities | +| [Severity Classification](Severity-Classification.md) | How drift changes are classified (GREEN / YELLOW / RED) | +| [Troubleshooting](Troubleshooting.md) | Common issues and how to resolve them | + +--- + +## Tech Stack + +- **Python 3.11+** +- **FastAPI + Uvicorn** -- API framework and ASGI server +- **Requests** -- HTTP client for Slack webhooks and OpenAPI fetching +- **Rich** -- Beautiful CLI output with colors and formatting +- **Pytest** -- Contract test runner +- **Ruff** -- Fast Python linter + +--- + +## License + +MIT -- use freely for demos, talks, and internal tooling. diff --git a/docs/wiki/Pre-Commit-Hooks.md b/docs/wiki/Pre-Commit-Hooks.md new file mode 100644 index 0000000..7ab12ad --- /dev/null +++ b/docs/wiki/Pre-Commit-Hooks.md @@ -0,0 +1,137 @@ +# Pre-Commit Hooks + +Catch API drift before code is even pushed to the remote repository. The pre-commit hook checks whether staged changes to `app/` files introduce API drift and takes action based on severity. + +--- + +## Installation + +### Option A: Using the `pre-commit` Framework (Recommended) + +The project includes a `.pre-commit-config.yaml` that configures two hooks: + +1. **Drift Guard** -- Checks for API drift when `app/` files are staged +2. **Ruff** -- Lints Python code with auto-fix + +```bash +pip install pre-commit +pre-commit install +``` + +That's it! The hooks will run automatically on every `git commit`. + +### Option B: Manual Git Hook + +Copy the hook script directly into your `.git/hooks/` directory: + +```bash +cp scripts/pre-commit-drift-check.sh .git/hooks/pre-commit +chmod +x .git/hooks/pre-commit +``` + +--- + +## How the Hook Works + +### Trigger Condition + +The hook **only runs** when you stage changes to files under the `app/` directory. Changes to tests, docs, scripts, or other files will not trigger it. + +### Execution Flow + +1. Checks if any files under `app/` are staged (`git diff --cached --name-only -- 'app/'`) +2. If no app files are staged, exits immediately (no check needed) +3. Verifies that a baseline exists at `artifacts/openapi_baseline.json` +4. Starts a temporary server on port 18199 +5. Runs `drift_guard check` against the temporary server +6. Takes action based on the exit code (severity) + +### Severity-Based Behavior + +| Severity | Exit Code | Hook Behavior | +|----------|-----------|---------------| +| **No drift** | 0 | Commit proceeds normally | +| **GREEN** (non-breaking) | 10 | Auto-updates baseline + docs, stages the files, commit proceeds | +| **YELLOW** (potentially breaking) | 11 | Auto-updates baseline + docs, stages the files, commit proceeds | +| **RED** (breaking) | 2 | **Blocks the commit** with instructions | +| **Error** | 1 | Warning printed, commit proceeds (fail-open) | + +### When the Commit Is Blocked (RED) + +If breaking drift is detected, the hook blocks the commit and displays: + +``` +[drift-guard] BREAKING API drift detected! Commit blocked. + + This change removes endpoints, removes fields, or adds new + required fields -- it will break existing clients. + + To proceed: + 1. Review the drift summary in artifacts/drift_summary.md + 2. Update baseline: python -m drift_guard.guard baseline + 3. Stage files: git add artifacts/openapi_baseline.json docs/api_reference.md + 4. Commit again +``` + +### When Drift Auto-Updates (GREEN / YELLOW) + +For non-breaking and potentially breaking changes, the hook: +1. Runs `drift_guard baseline` to regenerate the baseline and docs +2. Stages the updated files (`git add artifacts/openapi_baseline.json docs/api_reference.md`) +3. Allows the commit to proceed + +This means your commit will automatically include the updated documentation. + +--- + +## `.pre-commit-config.yaml` Reference + +```yaml +repos: + - repo: local + hooks: + - id: drift-guard + name: Drift Guard - API drift check + entry: bash scripts/pre-commit-drift-check.sh + language: system + files: ^app/ + pass_filenames: false + stages: [pre-commit] + + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.8.6 + hooks: + - id: ruff + args: [--fix] +``` + +--- + +## Skipping the Hook + +To bypass the pre-commit hook for a single commit (not recommended): + +```bash +git commit --no-verify +``` + +--- + +## Prerequisites + +For the hook to work, you need: + +1. **A saved baseline** at `artifacts/openapi_baseline.json` (run `python -m drift_guard.guard baseline` first) +2. **Dependencies installed** (`pip install -r requirements.txt`) +3. **Port 18199 available** (the hook starts a temporary server on this port) + +--- + +## Troubleshooting + +| Issue | Solution | +|-------|----------| +| "No baseline found" | Run `python -m drift_guard.guard baseline` first | +| "Failed to start temporary server" | Check that port 18199 is free and dependencies are installed | +| Hook doesn't run | Verify the hook is installed: `ls -la .git/hooks/pre-commit` or `pre-commit install` | +| Hook runs on non-app changes | The hook only triggers on `app/` files. This is by design. | diff --git a/docs/wiki/Project-Structure.md b/docs/wiki/Project-Structure.md new file mode 100644 index 0000000..3959f38 --- /dev/null +++ b/docs/wiki/Project-Structure.md @@ -0,0 +1,116 @@ +# Project Structure + +A complete guide to the file layout and module responsibilities in the Docs Drift Guard repository. + +--- + +## Directory Tree + +``` +docs-drift-guard-demo/ + app/ + main.py # FastAPI application + audit endpoint + middleware.py # API Call Guard middleware + audit_log.py # In-memory audit log + docs/ + api_reference.md # Auto-generated (by drift_guard) + api_manual.md # Hand-written - intentionally stale + wiki/ # Documentation wiki (you are here) + drift_guard/ + guard.py # CLI entrypoint (baseline, check, audit) + openapi_to_md.py # OpenAPI JSON -> Markdown converter + drift_detect.py # Compares two OpenAPI snapshots + slack_notify.py # Slack Incoming Webhook integration + tests/ + test_contract.py # Contract test suite + artifacts/ + openapi_baseline.json # Stored OpenAPI snapshot + drift_summary.md # Generated change summary + audit_report.md # Generated audit report + scripts/ + bootstrap.sh # One-command setup + demo_drift.sh # Introduces code drift + demo_bad_client.sh # Simulates stale-doc client calls + demo_cleanup.sh # Runs contract tests + audit + pre-commit-drift-check.sh # Git pre-commit hook + .github/workflows/ + drift-guard.yml # GitHub Action CI gate + .pre-commit-config.yaml # pre-commit framework config + requirements.txt + README.md +``` + +--- + +## Module Details + +### `app/` -- The FastAPI Application + +This is the DataStack Platform API -- the "source of truth" for the API contract. + +| File | Responsibility | +|------|---------------| +| **`main.py`** | FastAPI app definition, Pydantic models (`Employee`, `EmployeeCreate`, `EmployeeList`, `HealthResponse`), all endpoint handlers, in-memory database with seed data | +| **`middleware.py`** | `APICallGuardMiddleware` -- intercepts every request, validates against OpenAPI spec, records mismatches to audit log, adds `X-API-Mismatch` header | +| **`audit_log.py`** | `AuditLog` class (singleton) -- in-memory storage for mismatch entries, with methods for recording, reading, clearing, summarizing, and exporting | + +**Key models in `main.py`:** +- `EmployeeCreate` -- Request body for `POST /v1/employees` (6 fields: `first_name`, `last_name`, `email`, `department`, `title`, `role`) +- `Employee` -- Full employee representation (10 fields, adds `id`, `status`, `hired_at`, `bio`) +- `EmployeeList` -- Paginated list wrapper (`total` + `employees` array) +- `HealthResponse` -- Health check response (`status` + `version`) + +### `drift_guard/` -- The CLI and Detection Engine + +| File | Responsibility | +|------|---------------| +| **`guard.py`** | CLI entrypoint with `baseline`, `check`, and `audit` commands. Orchestrates fetching specs, running detection, generating artifacts, and sending notifications | +| **`drift_detect.py`** | `detect_drift()` function -- compares two OpenAPI specs and returns a `DriftReport` with severity-classified `DriftEvent` objects | +| **`openapi_to_md.py`** | `openapi_to_markdown()` function -- converts an OpenAPI JSON dict into a deterministic Markdown API reference document | +| **`slack_notify.py`** | Slack integration -- builds Block Kit payloads, infers severity for formatting, posts to webhook (or prints if not configured) | + +**Key classes in `drift_detect.py`:** +- `Severity` -- IntEnum with `GREEN` (0), `YELLOW` (1), `RED` (2) and badge/label properties +- `DriftEvent` -- A single classified change with severity, category, description, impacted endpoints, and explanation +- `DriftReport` -- Container for all drift events with helper methods for filtering, formatting, and summarizing + +### `tests/` -- Contract Test Suite + +| File | Responsibility | +|------|---------------| +| **`test_contract.py`** | pytest suite with 8 test classes covering health, employees CRUD, audit log, truth loop validation, and stale client scenarios | + +### `scripts/` -- Utility Scripts + +| Script | Purpose | +|--------|---------| +| **`bootstrap.sh`** | One-command setup: creates venv, installs dependencies | +| **`demo_drift.sh`** | Replaces `app/main.py` with a modified version to introduce deliberate drift | +| **`demo_bad_client.sh`** | Sends API calls with wrong field names to demonstrate the runtime guard | +| **`demo_cleanup.sh`** | Runs contract tests + audit report in sequence | +| **`pre-commit-drift-check.sh`** | Git pre-commit hook that detects drift before committing | + +### `artifacts/` -- Generated Files + +| File | Generated By | Purpose | +|------|-------------|---------| +| **`openapi_baseline.json`** | `baseline` / `check` commands | Stored OpenAPI snapshot with metadata | +| **`drift_summary.md`** | `check` command | Human-readable change summary | +| **`audit_report.md`** | `audit` command | Runtime mismatch report | + +### `docs/` -- Documentation + +| File | Type | Purpose | +|------|------|---------| +| **`api_reference.md`** | Auto-generated | Markdown API reference generated from the live OpenAPI spec | +| **`api_manual.md`** | Hand-written | Intentionally stale documentation to demonstrate the problem | +| **`wiki/`** | Hand-written | This documentation wiki | + +### Configuration Files + +| File | Purpose | +|------|---------| +| **`.github/workflows/drift-guard.yml`** | GitHub Actions CI workflow | +| **`.pre-commit-config.yaml`** | pre-commit framework configuration | +| **`requirements.txt`** | Python dependencies | +| **`.gitignore`** | Git ignore patterns | diff --git a/docs/wiki/Runtime-Call-Guard.md b/docs/wiki/Runtime-Call-Guard.md new file mode 100644 index 0000000..2d9d8e2 --- /dev/null +++ b/docs/wiki/Runtime-Call-Guard.md @@ -0,0 +1,170 @@ +# Runtime Call Guard + +Layer 2 of the Docs Drift Guard system catches clients that call the API using incorrect field names or hit nonexistent endpoints -- in real time. + +--- + +## How It Works + +The Runtime Call Guard is implemented as a **FastAPI middleware** (`app/middleware.py`) that intercepts every incoming HTTP request and validates it against the live OpenAPI spec. + +### What It Checks + +1. **Unknown endpoints:** Does the request path match any endpoint defined in the OpenAPI spec? +2. **Unknown methods:** Is the HTTP method (GET, POST, etc.) defined for this path? +3. **Unknown fields:** For POST/PUT/PATCH requests, does the JSON body contain fields not defined in the schema? + +### What It Does NOT Do + +- **It never blocks requests.** The middleware is observe-only. All requests proceed to the handler regardless of mismatches. +- **It does not validate field types.** Only field names are checked (FastAPI's own validation handles types). +- **It does not check response bodies.** Only incoming requests are validated. + +--- + +## Mismatch Recording + +When a mismatch is found, the middleware: + +1. **Records it** in the in-memory audit log (`app/audit_log.py`) +2. **Adds a response header** `X-API-Mismatch: true` so the caller can detect the issue programmatically + +Each audit entry contains: + +| Field | Description | +|-------|-------------| +| `timestamp` | Unix timestamp of the mismatch | +| `method` | HTTP method (GET, POST, etc.) | +| `path` | Request path | +| `issue_type` | `unknown_endpoint`, `unknown_field`, or `type_mismatch` | +| `details` | Human-readable description of what went wrong | +| `client_ip` | IP address of the calling client | + +--- + +## Audit Log Endpoints + +### `GET /v1/audit` -- View the Audit Log + +Returns all recorded mismatches: + +```bash +curl -s http://localhost:8000/v1/audit | python -m json.tool +``` + +Response: +```json +{ + "total_mismatches": 2, + "entries": [ + { + "timestamp": 1708000000.0, + "method": "POST", + "path": "/v1/employees", + "issue_type": "unknown_field", + "details": "Client sent field `username` but the schema only expects: department, email, first_name, last_name, role, title", + "client_ip": "127.0.0.1" + }, + { + "timestamp": 1708000001.0, + "method": "GET", + "path": "/v1/teams", + "issue_type": "unknown_endpoint", + "details": "No matching endpoint in OpenAPI spec for GET /v1/teams", + "client_ip": "127.0.0.1" + } + ] +} +``` + +### `DELETE /v1/audit` -- Clear the Audit Log + +Clears all recorded mismatches: + +```bash +curl -s -X DELETE http://localhost:8000/v1/audit | python -m json.tool +# --> {"status": "cleared"} +``` + +--- + +## Skipped Paths + +The middleware skips validation for these paths to avoid noise: + +- `/openapi.json` -- The OpenAPI spec endpoint itself +- `/docs` and `/docs/*` -- Swagger UI +- `/redoc` -- ReDoc documentation +- `/v1/audit` -- The audit log endpoint (to avoid self-referential logging) + +--- + +## Demo: Simulating Bad Client Calls + +The `scripts/demo_bad_client.sh` script simulates a client built from stale documentation: + +```bash +# Start the server first +uvicorn app.main:app --reload + +# Run the bad client simulation +bash scripts/demo_bad_client.sh +``` + +The script sends these calls: + +1. `GET /health` -- Correct call (no mismatch) +2. `GET /v1/users/1` -- Correct call +3. `POST /v1/users` with `username` instead of `name` -- **Unknown field detected** +4. `POST /v1/users` with `username` and `department` -- **Multiple unknown fields detected** +5. `GET /v1/teams` -- **Unknown endpoint detected** + +After running the bad client, view the audit log: + +```bash +curl -s http://localhost:8000/v1/audit | python -m json.tool +``` + +Then generate a Slack summary: + +```bash +python -m drift_guard.guard audit +``` + +--- + +## Implementation Details + +### Path Matching + +The middleware converts OpenAPI path templates (e.g., `/v1/employees/{employee_id}`) into regex patterns to match actual request paths (e.g., `/v1/employees/42`). + +### Field Validation + +For request bodies, the middleware: +1. Reads the raw request body bytes +2. Parses as JSON +3. Looks up the expected schema from the OpenAPI spec (resolving `$ref` pointers) +4. Compares the actual field names against the expected field names +5. Any fields in the request that aren't in the schema are flagged as `unknown_field` + +### In-Memory Storage + +The audit log is stored in memory as a singleton `AuditLog` instance. It resets when the server restarts. The log supports: +- `record()` -- Add a new entry +- `entries` -- Read all entries (returns a copy) +- `clear()` -- Remove all entries +- `summary()` -- Generate a Markdown summary +- `export_json()` -- Write the full log to a JSON file + +--- + +## Production Considerations + +The middleware is designed for **observe-only mode**. In a production deployment, you might: + +1. **Start in observe mode** (current behavior) to understand what clients are doing wrong +2. **Add enforcement mode** that returns 400 errors for unknown fields +3. **Persist the audit log** to a database instead of in-memory storage +4. **Add sampling** to reduce overhead on high-traffic APIs +5. **Add per-client tracking** using API keys or auth tokens diff --git a/docs/wiki/Severity-Classification.md b/docs/wiki/Severity-Classification.md new file mode 100644 index 0000000..0ef6add --- /dev/null +++ b/docs/wiki/Severity-Classification.md @@ -0,0 +1,158 @@ +# Severity Classification + +Docs Drift Guard classifies every detected change into one of three severity levels. This classification drives automation behavior, notification decisions, and CI gate logic. + +--- + +## Severity Levels + +### GREEN -- Non-Breaking + +**Badge:** Green circle +**Exit code:** 10 +**Meaning:** Changes that are safe for existing clients. No action required. + +**Examples:** +| Change Type | Example | +|-------------|---------| +| New endpoint added | `GET /v1/reports` added to the API | +| New optional field added | `Employee` gained optional field `bio` (with default) | +| Required field made optional | `Employee.role` changed from required to optional | +| Enum constraint removed | `status` field no longer restricted to specific values | +| New enum values added | `status` now also allows `"sabbatical"` | + +**Automation behavior:** +- `docs/api_reference.md` is regenerated (Tier 1) +- Baseline is updated +- **No Slack notification** (to avoid noise) +- Pre-commit hook: auto-updates and stages files, commit proceeds +- CI: auto-updates baseline, adds a GitHub notice, **passes** + +--- + +### YELLOW -- Potentially Breaking + +**Badge:** Yellow circle +**Exit code:** 11 +**Meaning:** Changes that may break some clients depending on usage patterns. Review recommended. + +**Examples:** +| Change Type | Example | +|-------------|---------| +| Field type changed | `Employee.id` type changed from `string` to `integer` | +| Optional field made required | `Employee.role` changed from optional to required | +| Enum values removed | `status` no longer allows `"pending"` | +| Enum constraint added | `role` field now restricted to `["member", "manager", "admin"]` | + +**Automation behavior:** +- `docs/api_reference.md` is regenerated (Tier 1) +- `artifacts/drift_summary.md` includes a "Proposed Narrative Updates" section for human review +- Baseline is updated +- **Slack notification sent** with "Review recommended before merge" +- Pre-commit hook: auto-updates and stages files, commit proceeds +- CI: adds a GitHub warning, **fails** -- requires PR review + +--- + +### RED -- Breaking + +**Badge:** Red circle +**Exit code:** 2 +**Meaning:** Changes that will break existing clients. Immediate review required. + +**Examples:** +| Change Type | Example | +|-------------|---------| +| Endpoint removed | `DELETE /v1/users/{id}` no longer exists | +| Required field added | `Employee` now requires `social_security_number` (new required field) | +| Field removed | `Employee` lost field `username` | + +**Automation behavior:** +- `docs/api_reference.md` is regenerated (Tier 1) +- `artifacts/drift_summary.md` includes a "Proposed Narrative Updates" section for human review +- Baseline is updated +- **Slack notification sent** with "Review required before merge" +- Pre-commit hook: **blocks the commit** with instructions +- CI: adds a GitHub error, **fails** -- must be resolved + +--- + +## How Severity Is Determined + +The maximum severity across all detected changes determines the overall severity of the drift check. + +For example, if a single commit adds a new optional field (GREEN) and removes an endpoint (RED), the overall severity is **RED**. + +### Detection Rules + +| Change | Severity | Rationale | +|--------|----------|-----------| +| New endpoint | GREEN | Additive -- existing clients unaffected | +| Removed endpoint | RED | Clients calling it will get 404 | +| New optional field | GREEN | Has a default -- existing clients unaffected | +| New required field | RED | Clients not sending it will get 422 | +| Removed field | RED | Clients relying on it will break | +| Type change | YELLOW | Depends on client usage | +| Required --> Optional | GREEN | Relaxes the contract | +| Optional --> Required | YELLOW | May break clients not sending the field | +| Enum values added | GREEN | More values accepted | +| Enum values removed | YELLOW | Clients using removed values will fail | +| Enum constraint added | YELLOW | Restricts previously open field | +| Enum constraint removed | GREEN | Relaxes the contract | + +--- + +## Severity in Different Contexts + +### CLI Output + +The `check` command displays severity with colored output: +- GREEN changes shown in green text +- YELLOW changes shown in yellow text +- RED changes shown in red text + +Each change gets an individual severity badge, and the overall severity is shown at the top. + +### Drift Summary (`artifacts/drift_summary.md`) + +Changes are grouped by severity: +```markdown +### RED Breaking (1) +- Removed endpoint: `DELETE /v1/users/{id}` + +### GREEN Non-breaking (2) +- New endpoint: `GET /v1/reports` +- `Employee` gained optional field `bio` +``` + +### Slack Notifications + +Slack messages use emoji badges and severity-specific messaging: +- GREEN: No notification sent +- YELLOW: "Potentially Breaking" header, "Review recommended" action +- RED: "Breaking" header, "Review required" action, additional review links + +### CI Pipeline + +The GitHub Actions workflow maps severity to CI outcomes: +- GREEN: GitHub notice, auto-updates baseline, **passes** +- YELLOW: GitHub warning, **fails** with review requirement +- RED: GitHub error, **fails** with escalation + +### Pre-Commit Hook + +The hook maps severity to commit behavior: +- GREEN/YELLOW: Auto-updates docs + baseline, stages files, **allows commit** +- RED: **Blocks commit** with instructions to review and manually update + +--- + +## Why Three Levels? + +The three-level system balances automation with safety: + +1. **GREEN** enables full automation for safe changes (no human in the loop needed) +2. **YELLOW** provides a warning that encourages review without being alarmist +3. **RED** enforces a hard stop that prevents breaking changes from being deployed without review + +This avoids the two extremes of "auto-merge everything" (dangerous) and "require manual review for every change" (slow and annoying). diff --git a/docs/wiki/Slack-Notifications.md b/docs/wiki/Slack-Notifications.md new file mode 100644 index 0000000..c18d399 --- /dev/null +++ b/docs/wiki/Slack-Notifications.md @@ -0,0 +1,146 @@ +# Slack Notifications + +Docs Drift Guard sends notifications to Slack when it detects API drift or runtime mismatches, so your team always knows what to fix. + +--- + +## Setup + +### 1. Create a Slack Incoming Webhook + +1. Go to [Slack API: Incoming Webhooks](https://api.slack.com/messaging/webhooks) +2. Create a new webhook for your desired channel +3. Copy the webhook URL + +### 2. Set the Environment Variable + +```bash +export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T.../B.../xxx" +``` + +### 3. For CI (GitHub Actions) + +Add `SLACK_WEBHOOK_URL` as a repository secret: + +1. Go to your repo on GitHub +2. Navigate to **Settings > Secrets and variables > Actions** +3. Click **New repository secret** +4. Name: `SLACK_WEBHOOK_URL` +5. Value: Your Slack webhook URL + +--- + +## What Happens Without the Webhook + +If `SLACK_WEBHOOK_URL` is not set, the tool **prints the JSON payload** that would have been sent to the console. This lets you verify the message format without actually sending anything. + +--- + +## Notification Types + +### 1. Drift Notification + +Sent by the `check` command when YELLOW or RED drift is detected. GREEN drift does NOT trigger Slack notifications (to avoid noise). + +**Message structure (Slack Block Kit):** + +| Section | Content | +|---------|---------| +| **Header** | Severity emoji + "Docs Drift Guard \| {Severity Label}" | +| **Change Summary** | List of added/removed endpoints and schema changes | +| **Impact** | What this means for existing clients | +| **Docs Status** | Which docs were regenerated or drafted | +| **Files Changed** | List of modified files | +| **Action** | What the reader needs to do (if anything) | +| **Review Links** | Links to updated docs on GitHub, drift summary, and Devin | + +**Severity-specific behavior:** + +| Severity | Slack? | Action Text | Review Links | +|----------|--------|-------------|--------------| +| GREEN | No | -- | -- | +| YELLOW | Yes | "Review recommended before merge." | Updated docs, Devin | +| RED | Yes | "Review required before merge." | Updated docs, drift summary, Devin | + +### 2. Audit Notification + +Sent by the `audit` command when runtime mismatches are found. + +**Message structure (Slack Block Kit):** + +| Section | Content | +|---------|---------| +| **Header** | "Runtime Audit \| API call mismatches detected" | +| **Summary** | Bullet list of all mismatches | +| **Context** | Explanation that these indicate stale client calls | + +--- + +## Message Examples + +### Drift Notification (RED) + +``` +Header: RED Docs Drift Guard | Breaking +Change Summary: + - Removed endpoint: DELETE /v1/users + - `User` lost field `username` +Impact: + - Existing clients may fail. Removed endpoints or fields will cause errors. +Docs Status: + - docs/api_reference.md has been regenerated from the live OpenAPI spec. +Files Changed: + - docs/api_reference.md + - artifacts/openapi_baseline.json +Action: + - Review required before merge. +Links: + Review updated docs on GitHub | View drift summary | Open Devin to investigate +``` + +### Audit Notification + +``` +Header: Runtime Audit | API call mismatches detected +Summary: + - POST /v1/employees (unknown_field): Client sent field `username` + but the schema only expects: department, email, first_name, last_name, role, title + - GET /v1/teams (unknown_endpoint): No matching endpoint in OpenAPI spec +Context: + These mismatches indicate clients calling the API with field names or + endpoints that don't match the current spec. +``` + +--- + +## Severity Inference (for Slack formatting) + +The Slack notification module (`drift_guard/slack_notify.py`) infers severity purely for formatting purposes: + +| Condition | Inferred Severity | +|-----------|------------------| +| Removed endpoint or lost field | **Breaking** (RED) | +| Non-additive schema changes (type change, rename, etc.) | **Potentially Breaking** (YELLOW) | +| Only added endpoints or gained fields | **Non-Breaking** (GREEN) | + +This is separate from the detection-level severity classification in `drift_detect.py` -- it's used only to style the Slack message. + +--- + +## Customization + +The Slack payload builders are in `drift_guard/slack_notify.py`. Key functions: + +| Function | Purpose | +|----------|---------| +| `build_drift_message()` | Builds a structured `DriftMessage` from drift data | +| `send_drift_notification()` | Sends the drift message to Slack | +| `send_audit_notification()` | Sends an audit summary to Slack | +| `_build_drift_payload()` | Constructs the Slack Block Kit JSON for drift | +| `_build_audit_payload()` | Constructs the Slack Block Kit JSON for audits | +| `_post_or_print()` | Posts to webhook or prints if not configured | + +The review links point to: +- `https://github.com/justinemach/DevinTestRepo/blob/main/docs/api_reference.md` +- `https://github.com/justinemach/DevinTestRepo/blob/main/artifacts/drift_summary.md` +- `https://app.devin.ai` diff --git a/docs/wiki/Troubleshooting.md b/docs/wiki/Troubleshooting.md new file mode 100644 index 0000000..e497f90 --- /dev/null +++ b/docs/wiki/Troubleshooting.md @@ -0,0 +1,233 @@ +# Troubleshooting + +Common issues and how to resolve them when using Docs Drift Guard. + +--- + +## Server Issues + +### "Could not connect to http://localhost:8000" + +**Cause:** The API server is not running. + +**Fix:** +```bash +source .venv/bin/activate +uvicorn app.main:app --reload +``` + +Then verify it's up: +```bash +curl http://localhost:8000/health +``` + +### Server crashes on startup + +**Cause:** Missing dependencies or Python version mismatch. + +**Fix:** +```bash +# Ensure you're using Python 3.11+ +python3 --version + +# Reinstall dependencies +pip install -r requirements.txt +``` + +--- + +## Drift Guard Issues + +### "No baseline found at artifacts/openapi_baseline.json" + +**Cause:** You haven't saved a baseline yet, or the file was deleted. + +**Fix:** +```bash +# Make sure the server is running first +python -m drift_guard.guard baseline +``` + +### Drift check shows no changes but code was modified + +**Cause:** The server might not have reloaded yet, or the baseline was already updated. + +**Fix:** +1. If using `--reload`, wait a moment for the server to pick up changes +2. Check that the baseline is from before your changes: + ```bash + python -m json.tool artifacts/openapi_baseline.json | head -10 + ``` + Look at the `_meta.saved_at` timestamp +3. If the baseline is current, re-save it from before your changes and try again + +### Exit code 10 or 11 instead of 2 + +**Cause:** The severity classification might not match your expectation. + +**Explanation:** +- Exit 10 = GREEN (non-breaking) -- e.g., added optional field or new endpoint +- Exit 11 = YELLOW (potentially breaking) -- e.g., type change or required flag toggle +- Exit 2 = RED (breaking) -- e.g., removed endpoint or removed field + +See [Severity Classification](Severity-Classification.md) for the full rules. + +--- + +## Pre-Commit Hook Issues + +### Hook doesn't run on commit + +**Possible causes:** +1. Hook not installed +2. No `app/` files staged + +**Fix:** +```bash +# If using pre-commit framework +pre-commit install + +# Or if using manual hook +cp scripts/pre-commit-drift-check.sh .git/hooks/pre-commit +chmod +x .git/hooks/pre-commit + +# Verify the hook is installed +ls -la .git/hooks/pre-commit +``` + +### "Failed to start temporary server" + +**Cause:** Port 18199 is in use or dependencies are missing. + +**Fix:** +```bash +# Check if port is in use +lsof -i :18199 + +# Kill any process using it +kill $(lsof -t -i :18199) + +# Ensure dependencies are installed +pip install -r requirements.txt +``` + +### Hook blocks commit unexpectedly + +**Cause:** RED (breaking) drift was detected. + +**Fix:** +1. Review the drift summary: `cat artifacts/drift_summary.md` +2. If the breaking change is intentional, update the baseline: + ```bash + python -m drift_guard.guard baseline + git add artifacts/openapi_baseline.json docs/api_reference.md + git commit + ``` +3. As a last resort (not recommended): `git commit --no-verify` + +--- + +## Contract Test Issues + +### Tests fail with "Connection refused" + +**Cause:** The API server is not running. + +**Fix:** +```bash +uvicorn app.main:app --reload +# Wait for it to start, then +pytest tests/test_contract.py -v +``` + +### Some tests fail after running `demo_drift.sh` + +**Cause:** The demo drift script replaces `app/main.py` with a different version that has different endpoints and schemas. + +**Fix:** This is expected behavior during the demo. The contract tests are designed to show what breaks when the API changes. To restore the original code: +```bash +git checkout app/main.py +``` + +### Tests pass locally but fail in CI + +**Cause:** The CI server might have a slightly different state (e.g., different seed data from earlier test runs). + +**Fix:** The in-memory database resets on server restart, so CI should get a clean state. Check the CI logs for details on which specific assertions failed. + +--- + +## Slack Issues + +### "SLACK_WEBHOOK_URL not set -- printing payload instead" + +**Cause:** The `SLACK_WEBHOOK_URL` environment variable is not set. + +**Fix:** +```bash +export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T.../B.../xxx" +``` + +For CI, add it as a GitHub repository secret (see [Slack Notifications](Slack-Notifications.md)). + +### "Slack webhook returned 4xx" + +**Cause:** The webhook URL is invalid, expired, or the Slack app was removed. + +**Fix:** +1. Verify the webhook URL is correct +2. Check that the Slack app/integration is still active +3. Create a new webhook if needed + +### No Slack notification for GREEN drift + +**Cause:** This is by design. GREEN (non-breaking) changes do not trigger Slack notifications to avoid noise. + +**Fix:** If you want notifications for all changes, modify `drift_guard/guard.py` in the `cmd_check()` function to remove the severity check before calling `send_drift_notification()`. + +--- + +## CI/CD Issues + +### CI fails with "BREAKING API drift" + +**Cause:** Your code changes introduce breaking API changes (removed endpoints, removed fields, or new required fields). + +**Fix:** +1. Download the drift summary artifact from the GitHub Actions run +2. Review the changes listed +3. If intentional, update the baseline locally: + ```bash + python -m drift_guard.guard baseline + git add artifacts/openapi_baseline.json docs/api_reference.md + git commit -m "Update API baseline after intentional breaking change" + git push + ``` + +### CI fails with "Contract tests failed" + +**Cause:** The contract tests found mismatches between expected and actual API behavior. + +**Fix:** +1. Check the CI logs for which specific tests failed +2. Run the tests locally to reproduce: `pytest tests/test_contract.py -v` +3. Fix the API code or update the tests as appropriate + +### CI passes locally but fails on GitHub + +**Cause:** Possible differences in the committed baseline vs. local state. + +**Fix:** +1. Ensure your committed `artifacts/openapi_baseline.json` matches your current API +2. Run `python -m drift_guard.guard baseline` and commit the result +3. Ensure `docs/api_reference.md` is up to date + +--- + +## General Tips + +1. **Always run the server before using drift guard commands** -- The CLI needs to fetch the live OpenAPI spec +2. **Commit baseline changes along with code changes** -- Keep them in sync +3. **Use `--reload` during development** -- The server automatically picks up code changes +4. **Check the artifacts directory** -- Generated reports are always saved there for review +5. **Read the CLI output carefully** -- The Rich-formatted output includes severity badges, file paths, and next-step instructions