feat: add secret and exposure scanning for PRs

[hossain-rayhan]

Add secret & exposure scanning for PRs

What & why

This PR adds a defense-in-depth secret-scanning setup so accidental leaks (passwords, private keys/certs, cloud keys, DocumentDB connection strings) and accidental public exposure never make it into main. There was previously no secret scanning in the repo — only govulncheck (Go CVEs).

Coverage spans three layers:

1. Deterministic CI gate (blocks the PR) — the enforced check.
2. Local pre-commit hook — catches leaks before they're even committed.
3. AI reviewer — Copilot code review as a broad, contextual second pass.

Changes

File	Purpose
.github/workflows/secret-scan.yml	PR gate: gitleaks config self-test + gitleaks scan (diff-scoped, uploads SARIF to Security tab) + public LoadBalancer exposure check
.gitleaks.toml	Ruleset: passwords, private keys/certs, generic + DocumentDB connection strings, Azure App Insights keys, Azure subscription IDs; with a placeholder/test allowlist
.pre-commit-config.yaml	Local hook (gitleaks + detect-private-key + detect-aws-credentials)
scripts/security/test-secret-scan.sh	Self-test that proves every rule fires and placeholders are ignored
.github/copilot-review-instructions.md	New Security & secret-leak review section instructing Copilot review to flag the same categories 🔴 Critical

Detection coverage

• Hardcoded passwords / secrets / tokens / API keys
• Private keys & certificates (-----BEGIN … PRIVATE KEY-----, .pem/.key/.pfx/.p12)
• Credentialed connection strings (mongodb://, postgres://, redis://, …)
• DocumentDB connection strings — gateway port :10260, mongodb+srv://, *.mongocluster.cosmos.azure.com (the +srv form is missed by the generic rule)
• Azure Application Insights instrumentation keys & connection strings
• Azure subscription IDs (subscriptionId, AZURE_/ARM_SUBSCRIPTION_ID, /subscriptions/<guid>/…)
• Accidental public exposure — Service type: LoadBalancer without an internal-LB annotation

Design notes

• Uses the free MIT-licensed gitleaks CLI, not gitleaks/gitleaks-action, which requires a paid GITLEAKS_LICENSE secret for GitHub organizations (this repo is under the documentdb org).
• Diff-scoped scanning — the gate only scans commits a PR/push introduces, so pre-existing findings in git history don't block unrelated changes.
• Placeholder allowlist keeps docs/tests/playground examples (changeme, ${VAR}, user:pass, all-zeros GUID) from false-flagging. The self-test asserts this.

How to test locally

# One-off full scan
brew install gitleaks            # or: go install github.com/gitleaks/gitleaks/v8@latest
gitleaks detect --config .gitleaks.toml --redact -v

# Scan only staged changes (what a commit would include)
gitleaks protect --staged --config .gitleaks.toml --redact -v

# Verify the ruleset itself (proves each rule fires, placeholders ignored)
scripts/security/test-secret-scan.sh

Enable the local pre-commit hook (optional, per clone)

pip install pre-commit           # or: brew install pre-commit
pre-commit install               # wires it into git; runs automatically on every commit
pre-commit run --all-files       # test it once

The hook is opt-in and local (.git/hooks/ isn't committed), so it's a convenience — the CI gate is the real enforcement.

How it blocks a PR

The secret-scan workflow runs on every pull_request. If gitleaks finds a new secret, or a public LoadBalancer without an internal annotation, the job fails. Findings also appear in Security → Code scanning via SARIF upload. Once the checks are marked required (see next steps), a failing scan blocks merge.

AI reviewer

Copilot code review reads the new Security & secret-leak review section in .github/copilot-review-instructions.md and leaves 🔴 Critical comments / requests changes for the same categories. This is advisory (its comments aren't a required status check) — it complements, but does not replace, the deterministic gitleaks gate.

Next steps / repo settings required after merge

These are repo-admin actions, not code:

1. Make the checks required — Settings → Branches → branch protection for main → add as required status checks:

   • secret-scan / gitleaks config self-test
   • secret-scan / gitleaks (secrets, keys, certs)
   • secret-scan / public LoadBalancer / external exposure check

   This is what actually blocks merges.

2. (Optional) Enable GitHub Secret Scanning + Push Protection (Settings → Code security) as a native complement for known provider token formats.

Known follow-ups (not blockers)

• A full-history scan currently surfaces ~25 pre-existing findings (docs, tests, deployment examples, and 2 in source: internal/utils/constants.go, sidecar-injector config.go). These need separate triage — the diff-scoped gate means they won't block new PRs in the meantime. A gitleaks baseline can grandfather them if a scheduled full scan is added later.
• (Optional) Add a make secret-scan convenience target.

Testing done

• scripts/security/test-secret-scan.sh — 8/8 pass (7 rules fire, placeholders ignored)
• Verified diff-scoped scan catches a new secret in a PR range and ignores historical findings
• Validated workflow YAML, bash syntax, and gitleaks config parsing

[github-advanced-security]

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

• The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
• Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
• You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

[Copilot]

Pull request overview

Adds a defense-in-depth secret and exposure scanning setup for this repository, combining a required CI workflow (gitleaks + SARIF + LoadBalancer exposure check), a local pre-commit hook, and updated Copilot review guidance to reduce the chance of secret leaks or unintended public exposure reaching main.

Changes:

• Adds a secret-scan GitHub Actions workflow to run gitleaks diff-scoped scanning (with SARIF upload) and detect potentially-public Service type: LoadBalancer manifests.
• Introduces a custom .gitleaks.toml ruleset plus a bash self-test script to validate the rules fire and placeholders are ignored.
• Adds an opt-in .pre-commit-config.yaml and extends Copilot review instructions with secret/exposure review criteria.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
.github/workflows/secret-scan.yml	Adds CI jobs for gitleaks scanning + SARIF upload and a YAML-based LoadBalancer exposure check.
.gitleaks.toml	Defines custom gitleaks rules and allowlisting for placeholders and selected paths.
.pre-commit-config.yaml	Adds local pre-commit hooks for gitleaks and credential/key detectors.
scripts/security/test-secret-scan.sh	Implements a config self-test harness that generates fixtures and asserts expected gitleaks RuleIDs.
.github/copilot-review-instructions.md	Adds Copilot review guidance for secret-leak and exposure findings.

[documentdb-triage-tool]

🤖 Auto-triaged by documentdb-triage-tool.

Applied: documentation, CI/CD, enhancement
Project fields suggested: Component docs · Priority P0 · Effort M · Status Needs Review
Confidence: 0.95 (deterministic)

Reasoning

component from path globs (docs, ci); P0 keyword "security" in body; effort from diff stats (364+0 LOC, 5 files)

If a label is wrong, remove it manually and ping @patty-chow so the rules can be tuned. The bot will not re-label items that already have component labels.