elastic · eric-forte-elastic · Jul 14, 2026 · Jul 28, 2026
@@ -2,10 +2,11 @@
 
 Threat detection, alerting, and rule management workflows
 
-## Workflows (10)
+## Workflows (11)
 
 | Workflow | Description |
 |----------|-------------|
+| [DaC - VCS Rule Sync](./dac-vcs-rule-sync/dac-vcs-rule-sync.yaml) | Detection-as-Code sync to version control: rules tagged `vcs` in a dev space are exported through a detection-rules fork and opened as pull requests against main, with unit tests reported on the PR ([setup guide](./dac-vcs-rule-sync/README.md)) |
 | [Sync Rule Exceptions Across Spaces](./sync-rule-exceptions-across-spaces.yaml) | One-way sync of rule exceptions between Kibana spaces: exceptions added to any rule's default exception list in a source space (e.g. a "subset" space) are pushed to the matching rules (same rule_id) in a target space (e.g. the "main" space), skipping items already synced, with opt-in consts to overwrite items when the source copy is newer and to delete workflow-pushed copies whose source exception was removed. It is self-contained in Kibana, no external services |
 | [Disable Noisy Elastic Defend Behavior Rules](./disable-noisy-endpoint-rules-from-esql.yaml) | Detects noisy Elastic Defend behavior rules via ES\|QL (FP bursts) and auto-creates scoped Endpoint Security exceptions by rule version; new artifact versions are unaffected |
 | [␆ Mark Alert as Acknowledged](./mark-alert-as-acknowledged.yaml) | This workflow YAML defines a manual trigger for acknowledging security alerts |

@@ -0,0 +1,166 @@
+# Deploy & Test Guide — DaC VCS Rule Sync
+
+End-to-end walkthrough for deploying and validating the VCS rule sync
+pipeline (see [README.md](./README.md) for the architecture). It assumes you
+already have:
+
+- a fork of [elastic/detection-rules](https://github.com/elastic/detection-rules)
+- an Elastic stack that can reach GitHub, and GitHub Actions runners that can
+  reach your Kibana
+
+Everything on the Kibana side is configured through the UI — the GitHub
+connector does all the GitHub work (no PATs in YAML, no API-only connector
+setup).
+
+Note the sync direction: **Kibana is the source of truth for rules, not the
+detection-rules repo**. This pipeline pushes rule state from Kibana into
+GitHub so it can be reviewed, merged, and stored in VCS; it never writes
+rules back to Kibana. Additional GitHub Actions (e.g. importing to Kibana on
+merge) could turn this into a dual-sync architecture — in its example form
+it is one-way, Kibana → VCS/GitHub.
+
+---
+
+## Phase 0 — One-time prerequisites
+
+### 1. Enable Workflows in Kibana
+
+Kibana 9.3+ (tech preview): in the space where you'll work, go to
+**Stack Management → Advanced Settings**, search "workflows", and enable the
+Workflows UI. Your user needs the **Workflows: All** privilege.
+
+### 2. Create the dev space
+
+**Stack Management → Spaces** → create `dev` (or use an existing space). The
+workflow `consts` take the space **ID** (the lowercase URL slug), not the
+display name.
+
+### 3. Create the export API key (becomes `DR_API_KEY`)
+
+**Stack Management → API Keys → Create API key**, created as a user who can
+**read rules in the dev space** (Security feature privilege). That's all the
+pipeline needs — it never writes to Kibana or Elasticsearch from CI. Copy
+the **Encoded** value.
+
+### 4. Create the GitHub connector in Kibana
+
+Generate a GitHub credential that can write to your fork — fine-grained PAT
+scoped to the fork with **Contents: read/write** and **Pull requests:
+read/write**. Then **Stack Management → Connectors → Create connector →
+GitHub**, paste the credential. Grab the connector ID from the connector's
+details (or Dev Tools: `GET kbn:/api/actions/connectors`).
+
+The workflow only uses tools this connector is verified to expose:
+`create_branch`, `create_or_update_file`, `create_pull_request`, and PR
+search. It never dispatches GitHub Actions directly.
+
+### 5. Network sanity check
+
+GitHub-hosted runners must be able to reach your Kibana URL (for the rule
+export). If your stack is not internet-reachable, use a self-hosted runner.
+
+---
+
+## Phase 1 — Set up the fork
+
+1. Copy [`github/dac-vcs-rule-sync.yml`](./github/dac-vcs-rule-sync.yml) into
+   your fork at `.github/workflows/dac-vcs-rule-sync.yml` and push it to
+   `main`.
+2. In the fork, **Settings → Secrets and variables → Actions**, add the
+   connection secrets. Pick **one** connection form:
+
+   | Secret | Value |
+   |--------|-------|
+   | `DR_CLOUD_ID` | Cloud ID from the deployment overview page (Elastic Cloud) |
+   | `DR_KIBANA_URL` | …or the Kibana base URL, no `/s/<space>` suffix (self-managed) |
+   | `DR_API_KEY` | Always: the encoded key from Phase 0 |
+
+   Don't set both `DR_CLOUD_ID` and `DR_KIBANA_URL` — the Action fails fast
+   on that combination.
+3. (Recommended) Commit the custom rules directory so its config is
+   version-controlled:
+
+   ```bash
+   python -m detection_rules custom-rules setup-config dac-vcs-rules
+   git add dac-vcs-rules && git commit -m "Add DaC VCS custom rules dir" && git push
+   ```
+
+---
+
+## Phase 2 — Deploy the Elastic workflow
+
+1. In Kibana: **Workflows → Create workflow**, paste
+   [`dac-vcs-rule-sync.yaml`](./dac-vcs-rule-sync.yaml).
+
+   > **Workflows are space-specific.** The workflow is saved into whichever
+   > space you're in when you create it, and that's the only space where it
+   > (and its execution history) will show up — note which space you use.
+
+2. Edit `consts`: at minimum `github_owner` and `github_connector_id`; also
+   `dev_space`/`sync_tag` if you diverged from the defaults.
+3. Save, run it manually, and open the execution. With nothing tagged yet
+   you should see: `No rules tagged "vcs" found in space "dev" — nothing to
+   sync.` That proves the rule search and consts are wired correctly.
+
+---
+
+## Phase 3 — First sync (happy path)
+
+1. In the **dev** space, create or pick a custom rule and add the tag
+   **`vcs`**.
+2. Run the workflow manually (or wait for the schedule). The execution log
+   should show `VCS sync PR opened for 1 changed rule(s): <PR URL> ...`.
+3. On the fork, verify the chain:
+   - a branch `vcs-sync-<timestamp>` exists, containing
+     `.vcs-sync/vcs-sync-<timestamp>.json`;
+   - the **DaC - VCS rule sync** Action ran on that branch: exported the rule
+     (`Export tagged rules...`), committed it (`[VCS Sync] Export 'vcs'
+     tagged rules from space 'dev'`), and ran the unit tests;
+   - the PR against `main` now shows the rule TOML in its diff and a
+     ✅ *unit tests passed* comment.
+4. **Merge the PR.** The rule is now in version control.
+5. Run the workflow again without touching the rule: the log should show
+   `... none changed since the last sync ...` — the watermark works, no new
+   PR.
+6. Edit the tagged rule in dev (e.g. tweak the description) and run again: a
+   fresh sync PR opens with just that diff.
+
+---
+
+## Phase 4 — Failure and edge-case behavior
+
+- **Unit test failure:** if the exported rules fail the detection-rules test
+  suite, the Action run goes red on the PR and posts a ❌ comment with the
+  test output tail. The rules are still committed, so the diff stays
+  reviewable. Fix the rule in dev and let the next cycle re-sync (close the
+  failing PR first, or push a fix to its branch).
+- **No-change sync:** if a sync PR's export produces no file changes, the
+  Action closes the PR and deletes the branch automatically, with a comment.
+- **Open PR gate:** while any `[VCS Sync]` PR is open, the workflow skips
+  new syncs (`... a "[VCS Sync]" PR is already open ...`). Merge or close it
+  to resume.
+- **Failed sync retry:** the watermark only advances after the PR opens
+  successfully, so a failed branch/PR creation is retried on the next cycle.
+
+---
+
+## Phase 5 — Go live
+
+Nothing else to do — the workflow saved enabled with a 30-minute schedule,
+and idle cycles are no-ops. Adjust the cadence on the `scheduled` trigger if
+you want, and keep an eye on the workflow's execution history for the first
+day (it's only visible in the space where you saved the workflow).
+
+---
+
+## Troubleshooting map
+
+| Symptom | Likely cause |
+|---------|--------------|
+| Branch/file/PR steps fail in the Elastic workflow | Wrong `github_connector_id`, or the connector's credential lacks Contents/Pull requests write on the fork |
+| PR opened but the Action never ran | The Action file isn't at `.github/workflows/dac-vcs-rule-sync.yml` on `main`, or Actions are disabled on the fork; note the trigger only matches `vcs-sync-*` branches and `.vcs-sync/**` paths |
+| Action fails at "Validate connection settings" | Neither `DR_CLOUD_ID` nor `DR_KIBANA_URL` set, both set at once, or `DR_API_KEY` missing |
+| Action export finds 0 rules | Tag/space mismatch between workflow consts and reality, or the rule is prebuilt rather than custom |
+| Every run says "none changed" but you expect a sync | The watermark in `dac-vcs-sync-state` is ahead — check `GET dac-vcs-sync-state/_search` in Dev Tools; delete the index to force a full re-sync |
+| Sync PRs keep opening and closing with no changes | A rule's `updated_at` is being bumped without content changes (e.g. repeated tag toggling); harmless, the Action closes them |
+| Workflow or its execution history "missing" from the Workflows page | Workflows are space-specific — switch to the space where you created it |
@@ -0,0 +1,163 @@
+# DaC VCS Rule Sync (Dev → GitHub PR)
+
+A Detection-as-Code (DaC) sync into version control, built on the
+[detection-rules](https://github.com/elastic/detection-rules) CLI, following the
+[DaC quick start guide](https://dac-reference.readthedocs.io/en/latest/dac_quick_start_guide.html).
+
+Rules are authored in a **dev** Kibana space. Tagging a rule with **`vcs`**
+means it is *managed by the version control system* (GitHub). On a schedule,
+the Elastic workflow checks whether any tagged rule changed since the last
+sync, and if so uses the **Kibana GitHub connector** (created in the UI — no
+API-only setup, no tokens in YAML) to open a pull request against your
+detection-rules fork's `main`. The fork's GitHub Action populates the PR
+branch with the exported rules and runs the detection-rules unit tests, with
+results posted on the PR. Nothing is copied between Kibana spaces, and there
+is no Kibana-side alerting — review, test results, and history all live in
+GitHub.
+
+> **Sync direction: Kibana → VCS (one-way).** In this pipeline **Kibana is
+> the source of truth for rule content — not the detection-rules repo**.
+> Rules are authored and edited in the dev Kibana space, and version control
+> *receives* those updates so they can be reviewed, merged, and stored in
+> git. Nothing here writes rules back to Kibana. Adding further GitHub
+> Actions (for example, a `kibana import-rules` job that runs on merge to
+> `main`) could evolve this into a dual-sync architecture where VCS changes
+> also flow back into Kibana — but in its example form this is strictly
+> Kibana syncing to VCS/GitHub.
+
+```
+┌──────────────── Kibana ────────────────┐        ┌───────────── GitHub fork ─────────────┐
+│  dev space: rules tagged "vcs"         │        │                                       │
+│        │ 1. _find by tag               │        │  branch vcs-sync-<timestamp>          │
+│        ▼                               │        │   ├─ .vcs-sync/<branch>.json  ◀──┐    │
+│  [Elastic workflow]                    │        │   └─ dac-vcs-rules/ (exported) ◀─┼─┐  │
+│   2. changed since last sync?          │        │        ▲                         │ │  │
+│   3. GitHub connector:                 │        │        │ 4. Action (on push):    │ │  │
+│      create_branch ────────────────────┼────────┼────────┘   export-rules -e -ac ──┘ │  │
+│      create_or_update_file (request) ──┼────────┼───────────  commit + unit tests ───┘  │
+│      create_pull_request ──────────────┼────────┼──▶ PR vs main (tests commented) │     │
+│   5. record sync watermark (ES index)  │        │      merge = rules in VCS       │     │
+└────────────────────────────────────────┘        └───────────────────────────────────────┘
+```
+
+## Files
+
+| File | Purpose | Where it goes |
+|------|---------|---------------|
+| [`dac-vcs-rule-sync.yaml`](./dac-vcs-rule-sync.yaml) | Elastic workflow: finds `vcs`-tagged rules, detects changes, opens the sync branch/PR via the GitHub connector | Kibana → Workflows (space-specific — it lives in the space you create it in) |
+| [`github/dac-vcs-rule-sync.yml`](./github/dac-vcs-rule-sync.yml) | GitHub Action: exports tagged rules into the sync branch, commits, runs unit tests, comments on the PR | Your detection-rules fork, at `.github/workflows/dac-vcs-rule-sync.yml` |
+| [`DEPLOY_AND_TEST.md`](./DEPLOY_AND_TEST.md) | Step-by-step deployment walkthrough | Reference doc |
+
+## Configuration
+
+Everything environment-specific is parameterized in the Elastic workflow's
+`consts` block:
+
+| Const | Default | Meaning |
+|-------|---------|---------|
+| `dev_space` | `dev` | Kibana space rules are authored and tagged in |
+| `sync_tag` | `vcs` | Rule tag meaning "managed by version control" |
+| `github_owner` | — | Owner of your detection-rules fork (user or org) |
+| `github_repo_name` | `detection-rules` | Fork repository name |
+| `github_base_branch` | `main` | Branch the sync PRs target |
+| `github_connector_id` | — | Kibana GitHub connector used for branch/file/PR operations |
+| `sync_request_dir` | `.vcs-sync` | Where sync-request files are committed (also the Action's trigger path) |
+| `pr_title_prefix` | `[VCS Sync]` | PR title prefix; also used to detect an already-open sync PR |
+| `state_index` | `dac-vcs-sync-state` | Elasticsearch index storing the sync watermark |
+
+The sync cadence (default: every 30 minutes) is set on the `scheduled`
+trigger directly — trigger definitions cannot reference `consts`. Unchanged
+rules are skipped, so frequent runs are cheap no-ops.
+
+## Setup
+
+### 1. detection-rules fork
+
+1. Fork [elastic/detection-rules](https://github.com/elastic/detection-rules).
+2. Copy [`github/dac-vcs-rule-sync.yml`](./github/dac-vcs-rule-sync.yml) into
+   the fork at `.github/workflows/dac-vcs-rule-sync.yml` on `main`.
+3. (Recommended) Commit a custom rules directory so its config is
+   version-controlled — the Action expects it at `dac-vcs-rules/`:
+
+   ```bash
+   python -m detection_rules custom-rules setup-config dac-vcs-rules
+   git add dac-vcs-rules && git commit -m "Add DaC VCS custom rules dir" && git push
+   ```
+
+   If it is missing, the Action bootstraps a default one per run (but then
+   only the `rules/` content lands in the PR diff, not the config).
+4. Add the repository secrets (Settings → Secrets and variables → Actions).
+   Provide **either** the Cloud ID **or** the Kibana URL — not both:
+
+   | Secret | Value |
+   |--------|-------|
+   | `DR_CLOUD_ID` | Elastic Cloud ID (deployment overview page) — alternative to the URL |
+   | `DR_KIBANA_URL` | Kibana base URL (no `/s/<space>` suffix) |
+   | `DR_API_KEY` | Always required: base64 API key able to read rules in the dev space |
+
+### 2. Kibana
+
+1. Create a **GitHub connector** (Stack Management → Connectors → GitHub)
+   whose credential can write to the fork: create branches, commit files,
+   and open pull requests (fine-grained PAT: Contents read/write +
+   Pull requests read/write on the fork).
+2. Create a new workflow from
+   [`dac-vcs-rule-sync.yaml`](./dac-vcs-rule-sync.yaml) and fill in the
+   `consts` block (at minimum `github_owner` and `github_connector_id`).
+
+   > **Workflows are space-specific.** The workflow only appears — and its
+   > execution history only lives — in the space where you save it, so
+   > remember which space that is. The dev space is a natural home. Steps
+   > target the dev space explicitly via `/s/<space>/...` paths regardless
+   > of where the workflow lives.
+
+3. Run it once manually to confirm the consts are right.
+
+### 3. Sync a rule
+
+Tag any custom rule in the dev space with `vcs`. Within one schedule
+interval the workflow opens a PR (`vcs-sync-<timestamp>` → `main`), and the
+Action fills it with the exported rule TOML and comments the unit test
+results. Merge the PR to accept the rule state into version control. Edit
+the rule again and the next cycle opens a fresh sync PR with the diff.
+
+## Behavior notes
+
+- **One-way sync, Kibana as source of truth.** Rule content flows from
+  Kibana into git only. Editing a rule's TOML directly in the repo does not
+  change the rule in Kibana, and the next sync of that rule will overwrite
+  the repo copy with Kibana's version.
+- **Change detection.** The workflow stores a watermark (the max `updated_at`
+  across tagged rules) in `dac-vcs-sync-state` after each sync PR. Runs where
+  no tagged rule is newer than the watermark do nothing. Adding or removing
+  the tag bumps a rule's `updated_at`, so newly tagged rules sync on the next
+  cycle.
+- **One sync PR at a time.** If a `[VCS Sync]` PR is already open, the
+  workflow skips the cycle; changes are picked up on the first cycle after
+  the PR is merged or closed.
+- **Edits are included.** `export-rules` overwrites existing TOML files by
+  default (there is no `--overwrite` flag on export — that flag belongs to
+  `import-rules`), so edited rules update in place and show as diffs in the
+  PR.
+- **No-change PRs close themselves.** If the export produces no file changes
+  (e.g. a rule was touched but re-normalizes identically), the Action closes
+  the PR and deletes the branch.
+- **Unit test results live in GitHub.** A failing test suite fails the
+  Action run (red on the PR) and posts a comment with the test output tail.
+  There is no Kibana-side failure alerting in this design.
+- **Deletions are not synced.** Untagging (or deleting) a rule stops future
+  syncs but does not remove the TOML already in the repo — prune manually.
+- **`.vcs-sync/` is an audit log.** One small request JSON per sync rides
+  along in each PR; merged PRs accumulate them as a history of syncs. Prune
+  whenever you like.
+- **Custom rules only.** The export uses `--custom-rules-only`; prebuilt
+  Elastic rules are ignored even if tagged.
+
+## Migrating from the earlier "promote dev → prod" pipeline
+
+If you deployed the previous iteration of this example: delete the old
+`.github/workflows/dac-promote-rules.yml` from the fork, remove the old
+"DaC - Promote Tagged Rules from Dev to Prod" workflow in Kibana, and delete
+the "DaC Promotion Failure" detection rule and `dac-promotion-failures`
+index in the dev space if they were created. The `DR_ELASTICSEARCH_URL`
+secret is no longer needed.