ops: ship docker logs to Better Stack via Vector sidecar#23
Conversation
Vector scrapes the host docker socket read-only so a single agent covers both the free-tier stack (this compose) and the paid-backend stack (separate compose at /opt/paid-backend). Logs go to Better Stack via their HTTPS NDJSON ingest with a bearer token sourced from .env. The config tags each event with the originating container name so dashboard filters can split free-tier from paid-backend traffic, and explicitly excludes Vector's own logs from the source to prevent the agent from billing itself for its own chatter. Operator step: paste BETTERSTACK_SOURCE_TOKEN and BETTERSTACK_INGEST_HOST into /opt/github-store-backend/.env on the VPS before redeploying.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR adds Better Stack env vars, a Vector service to production docker-compose, and a Vector config to collect, tag, and forward container logs to Better Stack. ChangesBetter Stack Centralized Logging Infrastructure
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR introduces a Vector sidecar container that attaches to the host Docker socket and ships structured logs from every container on the VPS to Better Stack over HTTPS, using a bearer token injected via environment variables.
Confidence Score: 4/5Safe to deploy, but two concerns flagged in earlier review threads — the buffer overflow behaviour (drop_newest discards the most recent events) and the unrestricted Docker API access via the socket mount — remain unaddressed in this version of the code. The Vector sidecar is correctly wired: auth is injected via environment variables and never committed, the image is pinned to an exact tag, Vector's own logs are excluded from the shipping pipeline, and the .env.example guidance is clear. The open questions about buffer drop policy and Docker API scope noted in earlier review threads still exist in the code as written. ops/vector.toml — buffer configuration and the self-exclusion comment around matching semantics are worth a final read; docker-compose.prod.yml — Vector service is the only one without a Docker healthcheck. Important Files Changed
Sequence DiagramsequenceDiagram
participant DC as Docker Daemon
participant V as Vector (sidecar)
participant BS as Better Stack
DC->>V: stdout/stderr streams (via /var/run/docker.sock)
Note over V: docker_logs source excludes vector-* containers
V->>V: remap transform — adds .service + .host fields
V->>V: buffer (memory, 5 000 events)
V->>BS: POST NDJSON (gzip) Authorization: Bearer $TOKEN
BS-->>V: 2xx OK
Note over V: On failure: retry x5 — buffer full drop_newest
Reviews (3): Last reviewed commit: "Merge branch 'main' into ops/betterstack..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ops/vector.toml`:
- Line 15: The current exclusion exclude_containers = ["vector"] is brittle
because Docker Compose prefixes container names, so update the Vector source
configuration to exclude by a stable identifier instead of exact name: change
the exclusion to use container labels (e.g., exclude containers with the label
com.docker.compose.service=vector) or to a pattern match that covers Compose
prefixes (e.g., a wildcard/regex matching "vector" if the config supports it).
Locate the exclude_containers setting in vector.toml and replace or augment it
with label-based exclusion (or a supported glob/regex) so any Compose-prefixed
container like github-store-backend-vector-1 is reliably excluded.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: dc92b8e4-3dd4-422d-9198-8f763651fa82
📒 Files selected for processing (3)
.env.exampledocker-compose.prod.ymlops/vector.toml
CodeRabbit: exclude_containers = ["vector"] doesn't fire under Docker Compose because Compose prefixes container names with the project + index (github-store-backend-vector-1). Added all the prefixed variants so Vector consistently skips its own logs regardless of which Compose stack launches it. Greptile: documented the docker.sock :ro caveat — the read-only flag restricts the socket FILE, not the Docker API operations. A Vector compromise would have full container management privileges on the host. Mitigated by image pinning + single-tenant VPS; upgrade path via tecnativa/docker-socket-proxy documented in the header.
1 participant
Adds a Vector container that scrapes the host's docker socket (read-only) and ships every container's stdout/stderr to Better Stack as NDJSON. One agent covers both the free-tier stack and paid-backend stack because the socket sees every container regardless of which compose file launched them.
Files
Operator steps before deploy
```
BETTERSTACK_SOURCE_TOKEN=
BETTERSTACK_INGEST_HOST=<sNNNNNNN.eu-fsn-3.betterstackdata.com>
```
Post-merge alert wiring
Create in Better Stack UI → Logs → Alerts:
Also worth adding (low-effort follow-ups):
Summary by CodeRabbit