Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 18 additions & 14 deletions .agents/skills/chatnow-orienting/references/repository-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,21 +20,21 @@ Verified: 2026-07-22
| `presence/` | Presence aggregation, subscriptions, and typing coordination | `presence/source/presence_server.h`, `presence/source/presence_server.cc`, `proto/presence/presence_service.proto` |
| `push/` | WebSocket connections, routes, cross-instance delivery, resend, client ACK ingestion | `push/source/push_server.h`, `push/source/connection.hpp`, `push/source/push_server.cc`, `proto/push/notify.proto` |
| `odb/` | ODB entity definitions and durable relational fields | Affected entity, especially `message.hxx`, `user_timeline.hxx`, `conversation_member.hxx`, and `media_*.hxx` |
| `conf/` | Non-secret local/container flags and JSON configuration; tracked files are not a runtime secret source | `conf/local/`, `conf/docker/`, `conf/auth.json`, `conf/media.json` |
| `sql/` | Versioned schema migrations | `sql/V4__media.sql` and any migration matching affected ODB entities |
| `docker/` | Separate MinIO topology and initialization; not wired into the root application network | `docker/docker-compose.yml`, `docker/minio-init/entrypoint.sh` |
| `docker-compose.yml` | Application stack declaration; Media object-storage wiring is incomplete | Root `docker-compose.yml`, then affected `Dockerfile` and `conf/docker` file |
| `scripts/` | Operational support and monitoring configuration | `scripts/install_aws_sdk_linux.sh`, `scripts/prometheus/redis_alerts.yml` |
| `conf/` | Non-secret local/container flags and JSON configuration; tracked files are not a runtime secret source | `conf/local/`, `conf/docker/`, `conf/auth.json`, local `conf/media.json`, container `conf/docker/media.json` |
| `sql/` | Versioned forward-only schema migrations for all current ODB objects | `sql/V1__core.sql`, `sql/V4__media.sql`, `scripts/init_mysql.sh` |
| `docker/` | Reusable MinIO initialization script plus a supplemental standalone topology that is not combined with root Compose | `docker/minio-init/entrypoint.sh`, `docker/docker-compose.yml` |
| `docker-compose.yml` | Integrated local application topology, health conditions, and one-shot convergence services | Root `docker-compose.yml`, affected `Dockerfile`, `conf/docker`, and initializer script |
| `scripts/` | Runtime convergence/readiness, operational support, and monitoring | `scripts/init_mysql.sh`, `scripts/converge_mysql_users.sh`, `scripts/init_redis_cluster.sh`, `scripts/init_rabbitmq.py`, `scripts/wait_for_services.sh`, `scripts/prometheus/redis_alerts.yml` |
| `tests/` | Pure-Go L1-L4 plus Redis-focused Reliability framework, clients, fixtures, cleanup, and store verification | `tests/Makefile`, `tests/config.yaml`, affected `tests/bvt`, `tests/func`, `tests/perf`, `tests/reliability`, `tests/pkg` |
| `docs/` | Secondary architecture/API context and canonical operations guidance | `docs/operations/runtime-secrets.md`, affected `docs/api/*.yaml`, then relevant architecture documents |
| `docs/` | Secondary architecture/API context and canonical operations guidance | `docs/operations/compose-runtime.md`, `docs/operations/runtime-secrets.md`, affected `docs/api/*.yaml`, then relevant architecture documents |

## Verified ports and infrastructure endpoints

Application ports come from `conf/local`, `conf/docker`, and root `docker-compose.yml`. MinIO ports come from the separate `docker/docker-compose.yml`; this is not an integrated container endpoint map.
Application and integrated infrastructure ports come from `conf/local`, `conf/docker`, and root `docker-compose.yml`. Published root-profile infrastructure ports bind to loopback.

| Owner | Local endpoint/port | Container endpoint/port | Evidence |
|---|---:|---:|---|
| Gateway HTTP | `127.0.0.1:9000` | `gateway_server:9000` | `gateway_server.conf` `http_listen_port` |
| Gateway HTTP and readiness | `127.0.0.1:9000`; unauthenticated `GET /health` | `gateway_server:9000` | `gateway_server.conf` `http_listen_port`; `GatewayServer::dependencies_ready` |
| Media brpc | `127.0.0.1:10002` | `media_server:10002` | `media_server.conf` |
| Identity brpc | `127.0.0.1:10003` | `identity_server:10003` | `identity_server.conf` |
| Transmite brpc | `127.0.0.1:10004` | `transmite_server:10004` | `transmite_server.conf` |
Expand All @@ -47,14 +47,18 @@ Application ports come from `conf/local`, `conf/docker`, and root `docker-compos
| etcd | `127.0.0.1:2379` | `etcd:2379` | all service configs |
| MySQL | `127.0.0.1:3306` | `mysql:3306` | root Compose |
| Redis cluster | `127.0.0.1:6379`, `:6380`-`:6384` | `redis-node1:6379`, `redis-node2:6380` through `redis-node6:6384` | root Compose; service seed flags |
| RabbitMQ | `127.0.0.1:5672` | `rabbitmq:5672` | Transmite, Message, Push configs |
| RabbitMQ | `127.0.0.1:5672` | `rabbitmq:5672` | Transmite, Message, Push host-only `mq_host` configs; builders append `5672` |
| Elasticsearch | HTTP `127.0.0.1:9200`; transport `:9300` | `elasticsearch:9200`; transport `:9300` | root Compose; service configs |
| MinIO S3 | Host `127.0.0.1:9000`, conflicting with Gateway | `minio:9000` only inside the separate MinIO Compose network | `conf/media.json`; supplemental Compose |
| MinIO console | Host `127.0.0.1:9001`, conflicting with Push WebSocket | `minio:9001` only inside the separate MinIO Compose network | supplemental Compose |
| MinIO S3 | `127.0.0.1:19000` for local access and presigned URLs | `minio:9000` for Media internal operations | root Compose; `conf/media.json`; `conf/docker/media.json` |
| MinIO console | `127.0.0.1:19001` | `minio:9001` | root Compose |

MySQL service configs set `mysql_port=0`, while root Compose exposes MySQL on `3306` and service entrypoints wait on `mysql:3306`; preserve that distinction when diagnosing driver defaults. Gateway's `websocket_listen_port=0` is not the client WebSocket endpoint; Push owns `ws_port=9001`.
MySQL service configs set `mysql_port=0`, while root Compose exposes MySQL on `3306` and service entrypoints wait on `mysql:3306`; preserve that distinction when diagnosing driver defaults. Gateway's `websocket_listen_port=0` is not the client WebSocket endpoint; Push owns `ws_port=9001`. Gateway `GET /health` returns `200` only when all eight discovered business-service channels are available and returns `503` otherwise; it is not a process-only liveness response.

Root Compose mounts `conf/media.json` into Media, but `s3.endpoint=http://127.0.0.1:9000` addresses the Media container itself. Root Compose has no MinIO service/dependency, while the supplemental MinIO Compose project has no declared shared external network with the root project. Do not present these declarations as a working integrated Media topology or recommend their current commands as a functional Media runtime. Any repair must explicitly reconcile the network, endpoint, dependency, and `9000`/`9001` host-port conflicts, then be verified from the affected containers.
Root Compose mounts `conf/docker/media.json` read-only. Media uses `s3.endpoint=http://minio:9000` for server-side S3 operations and `s3.public_endpoint=http://127.0.0.1:19000` to generate URLs reachable by local host clients. `common/infra/s3_client.hpp` owns separate internal and presign clients. Do not use the loopback public endpoint for a remote deployment without replacing it with a client-reachable address.

`mysql-init` applies read-only `V*.sql` migrations through `scripts/init_mysql.sh` and a checksum ledger. `V1__core.sql` plus `V4__media.sql` cover all 17 current ODB object tables; `scripts/converge_mysql_users.sh` owns the five table-scoped application identities. Redis, RabbitMQ, and MinIO use their own bounded one-shot initializers. `scripts/wait_for_services.sh` is the cross-stack semantic gate; `entrypoint.sh` is only bounded TCP prerequisite polling.

Root infrastructure state is bind-mounted under `middle/data`. `docker compose down -v` does not remove that state, and the current runtime PR does not include the separate CI clean-state override or workflow integration. Do not claim a repeatable cold start or passing runtime gate from the source topology or static contracts.

## Runtime credential ownership

Expand All @@ -64,7 +68,7 @@ Current consumers at the verified commit are:
- Conversation, Identity, Media, Message, and Relationship resolve service-specific MySQL password inputs through `common/config/secret_resolver.hpp`.
- Transmite, Message, and Push resolve service-specific RabbitMQ password inputs through the same resolver.
- Identity resolves its SMTP password; Media resolves separate S3 access-key and secret-key inputs. Non-secret S3 settings remain in `conf/media.json`.
- Root Compose requires MySQL, RabbitMQ, and supplemental MinIO bootstrap values through deployment environment references. These are separate from least-privileged application inputs. Redis has no configured password or ACL consumer.
- Root Compose requires MySQL, RabbitMQ, and MinIO bootstrap values through deployment environment references. One-shot initializers use those bootstrap inputs to converge least-privileged MySQL, RabbitMQ, and MinIO application identities; application containers consume only their own resolver inputs. Redis has no configured password or ACL consumer.

Tracked runtime credential literals have been removed from the scoped source, configuration, Compose, and test-runtime surfaces. Do not reintroduce values in documentation, logs, test output, Issues, or PRs. Synthetic test-only credentials and API examples require narrow scanner exemptions rather than broad path allowlists.

Expand Down
13 changes: 8 additions & 5 deletions .agents/skills/chatnow-orienting/references/technology-stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Use this reference for the `3.0-dev` architecture line, then verify task-sensiti
| Cache and coordination | Redis 7 Cluster plus local L1 caches | `common/dao/data_redis.hpp`; `common/utils/local_cache.hpp` |
| Message broker | RabbitMQ through AMQP-CPP/libev | `common/mq/`; Transmite, Message, and Push source |
| Search | Elasticsearch 7 | `common/dao/data_es.hpp`; Message, Identity, Relationship source |
| Object storage | MinIO through the S3-compatible AWS C++ SDK | `common/infra/s3_client.hpp`; Media source; `docker/docker-compose.yml` |
| Object storage | MinIO through the S3-compatible AWS C++ SDK | `common/infra/s3_client.hpp`; Media source; root `docker-compose.yml`; `conf/docker/media.json` |
| Service discovery and leases | etcd | `common/infra/etcd.hpp`; service entry files |
| Authentication | JWT HS256 with multi-key rotation support | `common/auth/`; `conf/auth.json` |
| Logging | spdlog JSON lines with propagated trace context | `common/infra/logger.hpp`; `common/log/`; `gateway/source/gateway_trace.hpp` |
Expand All @@ -42,18 +42,21 @@ Inspect root `CMakeLists.txt`, the affected service's `CMakeLists.txt`, and its
- Local service flags: `conf/local/*_server.conf`.
- Container service flags: `conf/docker/*_server.conf`.
- JWT keys and TTLs: Identity, Gateway, and Push resolve `CHATNOW_JWT_CONFIG` or `CHATNOW_JWT_CONFIG_FILE` at process startup. The value is the complete JSON document.
- Media S3 application credentials are resolved through the common secret resolver. Buckets, endpoint, presign, and MIME policy remain in `conf/media.json` plus Media flags.
- Media S3 application credentials are resolved through the common secret resolver. `conf/media.json` is the local-process configuration; root Compose mounts `conf/docker/media.json` read-only. Container-internal S3 operations use `http://minio:9000`, while client-facing local presigned URLs use the published `http://127.0.0.1:19000` endpoint.
- Example Transmite flags: `conf/transmite_server.conf.example`.
- Service defaults and flag definitions: each `<service>/source/<service>_server.cc`.
- MySQL passwords for Conversation, Identity, Media, Message, and Relationship use service-specific direct-environment or `_FILE` inputs through `common/config/secret_resolver.hpp`.
- RabbitMQ passwords for Transmite, Message, and Push use the same resolver contract. Identity SMTP and Media S3 application credentials are also migrated.
- The `mq_host` flag is host-only for Transmite, Message, and Push. Their builders append the fixed AMQP port `5672`; configuration must not include a port.
- The resolver accepts exactly one allowlisted direct environment variable or `_FILE` locator, fails closed on missing/conflicting input, and validates secret-file type, owner, mode, size, and content. It reads once at startup; there is no hot reload.
- Bootstrap credentials in Compose remain deployment environment references rather than application resolver inputs. Redis has no configured password or ACL consumer.
- The canonical names, consumers, deployment rules, and limitations are maintained in `docs/operations/runtime-secrets.md`.
- Root `docker-compose.yml` declares the application stack used by CI, but it is not a complete integrated Media/MinIO topology: it starts Media without a MinIO service or dependency.
- `docker/docker-compose.yml` separately declares MinIO and its initialization sidecar on a different default Compose network. Media mounts `conf/media.json`, whose `http://127.0.0.1:9000` endpoint resolves to the Media container itself, not to that separate MinIO container.
- Root `docker-compose.yml` is the integrated local application topology. It includes health-checked MySQL, six-node Redis Cluster, RabbitMQ, Elasticsearch, etcd, and MinIO plus one-shot `mysql-init`, `redis-cluster-init`, `rabbitmq-init`, and `minio-init` convergence services. Application dependencies use health or `service_completed_successfully` conditions instead of fixed startup delays.
- `mysql-init` mounts `sql/` read-only, applies ordered `V*.sql` files through a checksum ledger, rejects changes to an applied version, covers the 17 current ODB object tables, and converges five table-scoped MySQL application users. This is a forward-only repository bootstrap mechanism, not a general rollback engine.
- MinIO S3 and console ports are published on loopback `19000` and `19001`; Gateway HTTP remains `9000` and Push WebSocket remains `9001`. The supplemental `docker/docker-compose.yml` is not part of the root topology and must not be combined with it.
- `scripts/wait_for_services.sh` is the bounded semantic cross-stack readiness entry point. It checks Redis Cluster state and slots, the MySQL schema and users, RabbitMQ alarms, Elasticsearch health, MinIO health and buckets, eight exact etcd registrations, dependency-aware Gateway health, and Push listener reachability. The shared service `entrypoint.sh` performs bounded TCP prerequisite polling only.

The two Compose declarations also conflict on host ports: Gateway HTTP and MinIO S3 both publish `9000`; Push WebSocket and the MinIO console both publish `9001`. Therefore, neither `docker compose up -d --build` nor running both Compose files as written proves a functional containerized Media flow. Treat the network, Media S3 endpoint, service dependency, and host-port mapping as unresolved executable contradictions that must be fixed and verified before documenting a working container runtime command.
The source topology and static contracts do not prove a successful clean-slate start. Root infrastructure state uses bind mounts under `middle/data`, so `docker compose down -v` does not remove it. CI workflow integration and isolated clean-state storage remain a separate follow-up PR; until fresh dynamic evidence exists, report full-stack cold start and Go runtime gates as unverified. The canonical operating contract is `docs/operations/compose-runtime.md`.

## Verification entry points

Expand Down
12 changes: 10 additions & 2 deletions .agents/skills/chatnow-testing/references/framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Read this reference before selecting, implementing, running, or reporting a test
| L4 Performance | `tests/perf`, `perf` | Throughput and latency baselines | `make -C tests test-perf` |
| Reliability | `tests/reliability`, `reliability` | Redis failure injection, recovery, and Push unacked convergence | `make -C tests test-reliability` |

The current `tests/Makefile` also provides `make -C tests proto`, `make -C tests deps`, `make -C tests test-agent-policy`, and `make -C tests clean`. Run `proto` only when generated Go protobuf is required; `clean` removes generated `tests/proto/chatnow` content. Repository policy checks are static evidence and never substitute for a behavior RED or runtime gate.
The current `tests/Makefile` also provides `make -C tests proto`, `make -C tests deps`, `make -C tests test-agent-policy`, and `make -C tests clean`. Run `proto` only when generated Go protobuf is required; `clean` removes generated `tests/proto/chatnow` content. Repository policy checks and `tests/pkg/contracts` Compose/runtime contracts are static evidence and never substitute for a behavior RED or runtime gate. Repository contracts do not consume a BVT/Functional case ID namespace.

The Reliability target runs the complete tagged package and does not consume `TEST_RUN`. For an exact test, invoke the same tagged Go package with an anchored `-run` expression, then run `make -C tests test-reliability` for the layer regression. The current fault controller is Redis-only; there is no RabbitMQ, MySQL, arbitrary-service, or general-network controller.

Expand All @@ -42,6 +42,14 @@ CI runs `build` independently and builds reusable service artifacts. BVT needs `

The dedicated Reliability job exists, but the inspected PR run was skipped after an upstream failure. Its existence is executable-surface evidence, not a successful runtime result.

## Full-stack readiness boundary

`scripts/wait_for_services.sh` is the bounded pre-suite gate for the root Compose runtime when a full-stack job or operator explicitly invokes it. It verifies Redis Cluster state and slots, all 17 ODB tables and five MySQL application users, RabbitMQ running/alarm state, Elasticsearch yellow-or-green health, MinIO readiness and both media buckets, eight exact etcd service registrations, dependency-aware Gateway `GET /health`, and Push listener reachability. The Push check is TCP reachability, not WebSocket delivery evidence.

Container health, one-shot initializer completion, and the shared `entrypoint.sh` bounded TCP polling are startup prerequisites; none replaces `scripts/wait_for_services.sh`. Conversely, a passing static contract for the helper or Compose shape does not prove that any container started or that a runtime gate passed.

The current Issue #78 runtime PR does not own the CI workflow or an isolated clean-state storage override. CI integration remains a separate follow-up PR. Until a fresh run exists for the exact commit, report cold start, BVT, Functional, Reliability, and Performance as `not run` or `blocked`, not passed.

## Shared framework

- `tests/pkg/client`: configuration plus shared HTTP and WebSocket clients. Use `client.NewRequestID()` and `client.NewDeviceID()` for collision-resistant request, idempotency, device, and test-data suffixes.
Expand All @@ -55,7 +63,7 @@ Use unique IDs for every request and collision-prone resource. Do not rely on a

Poll the externally observable condition with a bounded deadline and useful failure message. Suitable conditions include service reachability, a WebSocket event, a database row/state, an Elasticsearch hit, a MinIO object, or an API state transition. A polling interval is allowed; a fixed delay used as proof of readiness or convergence is not.

Assign exactly one owner for each created resource. Prefer suite-level cleanup through `tests/pkg/cleanup`; add `t.Cleanup` for per-test resources such as clients, sockets, temporary objects, or state that suite cleanup cannot safely own. Cleanup must run on assertion failure. CI owns `docker compose down -v` in its full-stack jobs.
Assign exactly one owner for each created resource. Prefer suite-level cleanup through `tests/pkg/cleanup`; add `t.Cleanup` for per-test resources such as clients, sockets, temporary objects, or state that suite cleanup cannot safely own. Cleanup must run on assertion failure. Root Compose persists infrastructure through bind mounts under `middle/data`; `docker compose down -v` does not remove that state. A clean-slate test must use an explicitly disposable storage path or the separate CI override and must never delete a shared tree.

## Change-to-layer matrix

Expand Down
Loading