Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
058fd10
feat(stack): foundational image-tag deps + GIP-0088 stack profiles
RembrandtK May 11, 2026
2893cfb
feat(GIP-0088): deploy contracts + indexing-payments subgraph
RembrandtK May 11, 2026
67f83c0
refactor: rename test secrets from ACCOUNT-n to role-named
RembrandtK May 11, 2026
ee8c7c3
feat(infra): recipe overlay system for composable network configs
RembrandtK May 11, 2026
ac9bac9
feat(test-infra): per-test IndexerHandle + REO test wiring; mark brok…
RembrandtK May 11, 2026
f4a4de3
feat(infra): dump-state + bake/restore-snapshot scripts for fast stac…
RembrandtK May 11, 2026
f2ed3b7
fix(block-oracle): keep epoch subgraph fresh under test load
RembrandtK May 11, 2026
5b8bc38
docs: rewrite README around recipes; sweep stale planning/status docs
RembrandtK May 11, 2026
17ca4da
refactor(graph-contracts): consume published contracts image, pin sha…
RembrandtK Jun 12, 2026
0859325
fix(test): per-test indexer isolation + anvil archive state; un-ignor…
RembrandtK Jun 12, 2026
2f23443
feat(test): reo-live recipe + REO test gating and robustness
RembrandtK Jun 12, 2026
335dbd4
feat(recipes): recipe-driven nextest profiles + pre-GIP-0088 main recipe
RembrandtK Jun 12, 2026
bf95f00
feat(test): baked-baseline test runner with fingerprint-keyed cache
RembrandtK Jun 12, 2026
5e54f63
feat(dips): wire full on-chain DIPs lifecycle for indexing-payments
RembrandtK Jun 12, 2026
8a0153b
feat(recipes): default baseline to the mainnet-deployed contract surface
RembrandtK Jun 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 0 additions & 134 deletions .env

This file was deleted.

18 changes: 17 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,20 @@
.claude
.idea

# Environment overrides
# Environment files
# .env: auto-generated by scripts/resolve-recipe.sh from a recipe; picked up
# automatically by `docker compose`. Regenerate with `just resolve` or `just up`.
# .env.local: optional per-checkout overrides (highest precedence in the resolver).
# .env.secrets: optional secrets fragment (legacy; new code uses .env.local).
.env
.env.local
.env.secrets

# Recipe selection: .recipe (optional, committed per-branch default — branches
# pin their target scenario by committing one); .recipe.local (per-checkout
# override, never committed).
.recipe.local

# OS
.DS_Store
Thumbs.db
Expand All @@ -20,6 +30,12 @@ Thumbs.db
# Rust build artifacts
tests/target/

# State dumps from scripts/dump-state.sh (captured for offline debugging)
_dumps/

# Volume snapshots from scripts/bake-snapshot.sh
_snapshots/

# Legacy local config directory (now uses config-local Docker volume)
config/local/

Expand Down
1 change: 0 additions & 1 deletion CHEATSHEET.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,3 @@ curl "http://localhost:7700/api/subgraphs/id/BFr2mx7FgkJ36Y6pE5BiXs1KmNUmVDCnL82
```bash
docker exec -it redpanda rpk topic consume gateway_client_query_results --brokers="localhost:9092"
```

167 changes: 103 additions & 64 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,95 +2,94 @@

A local Graph network for debugging & integration tests.

## Usage
Requires Docker & Docker Compose v2.24+, plus [`just`](https://github.com/casey/just)
for the entry-point commands.

Requires Docker & Docker Compose v2.24+ and [just](https://github.com/casey/just).
## Quick start

```bash
# Show all recipes
just --list

# Start (or resume) the network — skips already-completed setup steps
just up

# Rebuild a single service after code changes
just up --build ${service}

# Get logs for a service
just logs ${service}

# Re-initialise from scratch (removes all persisted state)
just reset && just up
just up # resolve active recipe → .env, then docker compose up -d --build
just logs gateway # docker compose logs -f gateway
just rebuild gateway # rebuild + restart one service after code changes
just down # docker compose down
just reset && just up # re-initialise from scratch (removes all persisted state)
```

__Note__: State is persisted in named volumes, so the network restarts where it left off. Use `just reset` only when you want a clean slate.
The first `just up` materialises a recipe (see below) into a gitignored `.env`
file. After that, bare `docker compose` commands work directly — `just up` just
chains recipe resolution + a build-aware compose up. `docker compose` halts
with a clear error if `.env` is missing.

More useful commands for each component can be found at [CHEATSHEET.md](CHEATSHEET.md).
The resolved `.env` is read by three consumers:

## Configuration

The `.env` file holds all configuration and is read by three consumers:

- **docker-compose** — for `${VAR}` substitution in `docker-compose.yaml` (auto-loaded from the project directory).
- **docker compose** — for `${VAR}` substitution in `docker-compose.yaml` (auto-loaded from the project directory).
- **host scripts** — scripts that run on the host source this file via `source .env`.
- **containers** — volume-mounted at `/opt/config/.env` and typically sourced by each service's `run.sh`.

## Local Overrides

Create `.env.local` (gitignored) to override defaults without touching `.env`:

```bash
# .env.local — your local settings
COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer,indexing-payments
GRAPH_NODE_VERSION=v0.38.0-rc1
```

`.env.local` overrides `.env` for:
- `docker compose` but only when invoked via `just`. Bare `docker compose` reads only `.env`.
- host scripts (typically sourced automatically after `.env`)
- it DOES NOT override `.env` for container scripts.
## Recipes

## Service Profiles
A **recipe** selects which env fragments compose, which compose profiles enable,
and which image versions pin. Recipes live in [recipes/](recipes/) and reference
fragments in [config/](config/).

Optional services are controlled via `COMPOSE_PROFILES` in `.env`. By default, profiles that work out of the box are enabled:
| Recipe | Profile set | Includes |
| ------------------- | ------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| `baseline` | `block-oracle,explorer` | Default: the mainnet-deployed contract surface, stable image versions, no GIP-0088 components |
| `gip-0088` | `block-oracle,explorer,rewards-eligibility` | Full GIP-0088 contract deployment (REO + IA + RAM) on a post-audit contract image |
| `indexing-payments` | `explorer,rewards-eligibility,indexing-payments` | gip-0088 + WIP DIPs services (dipper, IISA, indexing-payments subgraph, dips-fork indexer-rs) |
| `reo-live` | `block-oracle,explorer,rewards-eligibility` | gip-0088 with production REO wiring (REO_MOCK=0) for the REO-A integration tests |

```bash
COMPOSE_PROFILES=block-oracle,explorer
just recipes # list available recipes
just recipe-active # show which one is currently selected
just up indexing-payments # one-shot recipe override
echo indexing-payments > .recipe.local # per-checkout sticky default (gitignored)
```

Available profiles:
Recipe selection precedence: CLI arg → `RECIPE` env → `.recipe.local` →
`.recipe` (committed per-branch default) → `baseline`.

To add a recipe, create `recipes/my-recipe.json` with the fragments + override
`env` to apply. See existing JSON files for the shape.

| Profile | Services | Prerequisites |
| --------------------- | --------------------------------- | -------------------------- |
| `block-oracle` | block-oracle | none |
| `explorer` | block-explorer UI | none |
| `rewards-eligibility` | eligibility-oracle-node | none (clones from GitHub) |
| `indexing-payments` | dipper, iisa, iisa-scoring | GHCR auth (below) |
## Local overrides

To enable all profiles, uncomment the full line in `.env`:
Create `.env.local` (gitignored) for machine-specific values that layer on top
of the resolved recipe — image-tag overrides, compose-profile additions, etc:

```bash
COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer,indexing-payments
# .env.local
GRAPH_NODE_VERSION=v0.43.0
COMPOSE_PROFILES=block-oracle,explorer,rewards-eligibility,indexing-payments,my-extra
```

### GHCR authentication (indexing-payments)
`.env.local` is sourced last during recipe resolution, so its values win.

The `indexing-payments` profile pulls private images from `ghcr.io/edgeandnode`. Create a GitHub **classic** Personal Access Token with `read:packages` scope (https://github.com/settings/tokens — fine-grained tokens do not support packages) and log in once:
## Iterating on upstream source

Most services run from prebuilt images pinned by `${SERVICE_VERSION}` vars
in [config/services.env](config/services.env) and
[config/indexing-payments.env](config/indexing-payments.env). To iterate on
upstream source, build a locally-tagged image in the upstream repo and
override the version pin in `.env.local`:

```bash
echo $GITHUB_TOKEN | docker login ghcr.io -u YOUR_USERNAME --password-stdin
# .env.local
INDEXING_PAYMENTS_SUBGRAPH_VERSION=local
```

Then set the image versions in `.env` or `.env.local`:
Then `just rebuild <service>` here to pick up the change.

```bash
DIPPER_VERSION=<tag>
IISA_VERSION=<tag>
```
How each upstream produces a `:local` tag is repo-specific. Convention is a
`just build-image` recipe that tags `<image>:local` — e.g.
graphprotocol/indexing-payments-subgraph builds
`ghcr.io/graphprotocol/indexing-payments-subgraph:local` via its `justfile`.

## Building from source - Dev overrides (compose/dev/)
## Building from source — dev overrides (compose/dev/)

For local development, mount locally-built binaries into running containers. Set `COMPOSE_FILE` in `.env` (or `.env.local`, when using `just`) to include dev override files:
For faster iteration than image rebuilds, mount locally-built binaries into
running containers. Set `COMPOSE_FILE` in `.env.local` to include dev override
files:

```bash
# Mount local indexer-service binary
Expand All @@ -105,24 +104,64 @@ Each override requires a binary path env var. Source repos own their own build;
local-network just wraps the published image with `run.sh` and utilities.
See [compose/dev/README.md](compose/dev/README.md) for details.

## Rebuilding after edits

```bash
just rebuild indexer-agent # rebuild + restart one service
just rebuild # rebuild + restart all
just up # equivalent if recipe hasn't changed (defaults to --build)
```

`run.sh` and `Dockerfile` changes only take effect after a rebuild.

## State persistence

Volumes (`chain-data`, `postgres-data`, `ipfs-data`, `redpanda-data`,
`iisa-scores`, `config-local`) survive `just down`. To start clean:

```bash
just reset # docker compose down -v
just up
```

## GHCR authentication (indexing-payments)

The `indexing-payments` profile pulls private images from `ghcr.io/edgeandnode`.
Create a GitHub **classic** Personal Access Token with `read:packages` scope
([fine-grained tokens don't support packages](https://github.com/settings/tokens))
and log in once:

```bash
echo $GITHUB_TOKEN | docker login ghcr.io -u YOUR_USERNAME --password-stdin
```

## Devcontainer usage

When running inside a devcontainer, service names (gateway, redpanda, etc.) won't resolve by default because the devcontainer is on a different Docker network. Connect it to the compose network once per session:
Inside a devcontainer, service names won't resolve by default because the
devcontainer is on a different Docker network. Connect once per session:

```bash
scripts/connect-network.sh
```

The script auto-detects the compose project network. You can also pass a network name explicitly: `scripts/connect-network.sh my-network_default`.
The script auto-detects the compose project network. Pass a name explicitly with
`scripts/connect-network.sh my-network_default`.

## Component cheatsheet

See [CHEATSHEET.md](CHEATSHEET.md) for per-component commands.

## Common issues

### `too far behind`

Gateway error:

```
ERROR graph_gateway::network::subgraph_client: network_subgraph_query_err="response too far behind"
```

This happens when subgraphs fall behind the chain head. With automine (default), this is a harmless warning during startup. Run `scripts/mine-block.sh 10` to advance blocks manually if needed.
Subgraphs fell behind the chain head. With automine (default), this is harmless
during startup. `scripts/mine-block.sh 10` to advance blocks manually.

### `LOCAL_NETWORK_RECIPE is missing a value`

`.env` hasn't been generated yet. Run `just resolve` (or any `just up`).
Loading