Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 34 additions & 1 deletion .env
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,45 @@
# devcontainer environments to set *_HOST env vars (e.g. CHAIN_HOST=chain)
# to reach services on the Docker network instead of localhost.

# --- Service profiles ---
# Controls which optional service groups are started.
# Available profiles:
# block-oracle epoch block oracle
# explorer block explorer UI
# rewards-eligibility REO eligibility oracle node
# indexing-payments dipper + iisa (requires GHCR auth — see README)
# Default: profiles that work out of the box.
COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer
# All profiles (indexing-payments requires GHCR auth — see README):
#COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer,indexing-payments

# --- Dev overrides ---
# Uncomment and extend to build services from local source.
# See compose/dev/README.md for available overrides.
#COMPOSE_FILE=docker-compose.yaml:compose/dev/graph-node.yaml

# indexer components versions
GRAPH_NODE_VERSION=v0.37.0
INDEXER_AGENT_VERSION=v0.25.4
INDEXER_SERVICE_RS_VERSION=v1.8.0
INDEXER_TAP_AGENT_VERSION=v1.12.2

# indexing-payments image versions (requires GHCR auth — see README)
# Set real tags in .env.local when enabling the indexing-payments profile.
DIPPER_VERSION=sha-24d10d4
IISA_VERSION=

# gateway components versions
GATEWAY_COMMIT=b37acb4976313316a2bc0a488ca98749da51c61d
TAP_AGGREGATOR_VERSION=sha-d38d0b9
TAP_ESCROW_MANAGER_COMMIT=530a5a72da7592b8d442b94d82a5a5f57d4a2b40

# eligibility oracle (clone-and-build — requires published repo)
ELIGIBILITY_ORACLE_COMMIT=84710857394d3419f83dcbf6687a91f415cc1625

# network components versions
BLOCK_ORACLE_COMMIT=3a3a425ff96130c3842cee7e43d06bbe3d729aed
CONTRACTS_COMMIT=908bc32597fc946d5f76ad64eef41fedded82e70
CONTRACTS_COMMIT=0003fe3adf7394df5c9ac1cf4ef600c96a61259f
NETWORK_SUBGRAPH_COMMIT=5b6c22089a2e55db16586a19cbf6e1d73a93c7b9
TAP_CONTRACTS_COMMIT=e3351e70b3e5d9821bc0aaa90bb2173ca2a77af7
TAP_SUBGRAPH_COMMIT=cf7279f60433bf9a9d897ec2548c13c0607234cc
Expand Down Expand Up @@ -97,5 +122,13 @@ RECEIVER_SECRET="0x2ee789a68207020b45607f5adb71933de0946baebbaaab74af7cbd69c8a90
SUBGRAPH="BFr2mx7FgkJ36Y6pE5BiXs1KmNUmVDCnL82KUSdcLW1g"
SUBGRAPH_2="9p1TRzaccKzWBN4P6YEwEUxYwJn6HwPxf5dKXK2NYxgS"

# REO (Rewards Eligibility Oracle)
# Set to 1 to deploy and configure the REO contract (Phase 4). Unset or 0 to skip.
REO_ENABLED=1
# eligibilityPeriod: how long an indexer stays eligible after renewal (seconds)
REO_ELIGIBILITY_PERIOD=300
# oracleUpdateTimeout: fail-safe — if no oracle update for this long, all indexers eligible (seconds)
REO_ORACLE_UPDATE_TIMEOUT=86400

# Gateway
GATEWAY_API_KEY="deadbeefdeadbeefdeadbeefdeadbeef"
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,8 @@ Thumbs.db
*.swo
*~

# Local config files
# Rust build artifacts
tests/target/

# Legacy local config directory (now uses config-local Docker volume)
config/local/
7 changes: 0 additions & 7 deletions .gitmodules

This file was deleted.

93 changes: 54 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Epochs are set up to be 554 blocks long, use `scripts/mine-block.sh` to advance

## Usage

Requires Docker & Docker Compose. Install foundry on the host for mining blocks.
Requires Docker & Docker Compose v2.24+. Install foundry on the host for mining blocks.

```bash
# Start (or resume) the network — skips already-completed setup steps
Expand All @@ -25,11 +25,22 @@ restarts where it left off. Use `down -v` only when you want a clean slate.

Add `--build` to rebuild after changes to Docker build context, including modifying `run.sh` or `Dockerfile`, or changed source code.

## Local Overrides

Create `.env.local` (gitignored) to override defaults without touching `.env`:

```bash
# .env.local — your local settings
COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer,indexing-payments
GRAPH_NODE_VERSION=v0.38.0-rc1
```

Host scripts source `.env.local` automatically after `.env`.

## Useful commands

- `docker compose up -d --build ${service}` — rebuild a single service after code changes
- `docker compose logs -f ${service}`
- `scripts/clean.sh` — interactive cleanup (volumes + generated config)
- `source .env`

## Components
Expand Down Expand Up @@ -188,63 +199,67 @@ docker exec -it redpanda rpk topic consume gateway_client_query_results --broker
}
```

## Optional: Indexing Payments
## Service Profiles

To enable payments for indexing work (alternative to TAP allocations):
Optional services are controlled via `COMPOSE_PROFILES` in `.env`.
By default, profiles that work out of the box are enabled:

```bash
# Start with indexing payments
docker compose -f docker-compose.yaml -f overrides/indexing-payments/docker-compose.yaml up

# Or use helper script
./overrides/indexing-payments/start.sh
COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer
```

See [overrides/indexing-payments/README.md](overrides/indexing-payments/README.md) for details.
Available profiles:

## Building components from source
| Profile | Services | Prerequisites |
| --------------------- | --------------------------------- | -------------------------- |
| `block-oracle` | block-oracle | none |
| `explorer` | block-explorer UI | none |
| `rewards-eligibility` | eligibility-oracle-node | none (clones from GitHub) |
| `indexing-payments` | dipper, iisa, iisa-scoring | GHCR auth (below) |

### docker compose overrides
To enable all profiles, uncomment the full line in `.env`:

The following components allow building from source by overriding `docker-compose.yml`:
```bash
COMPOSE_PROFILES=rewards-eligibility,block-oracle,explorer,indexing-payments
```

- graph-node
- graph-contracts (contracts)
- indexer-agent
### GHCR authentication (indexing-payments)

Please refer to `overrides/README.md` for instructions.
The `indexing-payments` profile pulls private images from `ghcr.io/edgeandnode`.
Create a GitHub **classic** Personal Access Token with `read:packages` scope
(https://github.com/settings/tokens — fine-grained tokens do not support packages) and log in once:

### git submodules source
```bash
echo $GITHUB_TOKEN | docker login ghcr.io -u YOUR_USERNAME --password-stdin
```

The following components allow building from source by cloning them with submodules:
Then set the image versions in `.env` or `.env.local`:

- indexer-service
- tap-agent
```bash
DIPPER_VERSION=<tag>
IISA_VERSION=<tag>
```

Building from source requires the Git submodules to be initialized first:
## Building Components from Source

- `git submodule update --init --recursive`
### Dev overrides (compose/dev/)

And then select the `wrapper-dev` target when building the Docker image in the `docker-compose.yaml` file.
For local development, mount locally-built binaries into running containers.
Set `COMPOSE_FILE` in `.env` to include dev override files:

```diff
indexer-service:
container_name: indexer-service
build: {
- target: "wrapper", # Set to "wrapper-dev" for building from source
+ target: "wrapper-dev", # Set to "wrapper-dev" for building from source
context: indexer-service,
}
```bash
# Mount local indexer-service binary
INDEXER_SERVICE_BINARY=/path/to/indexer-rs/target/release/indexer-service-rs
COMPOSE_FILE=docker-compose.yaml:compose/dev/indexer-service.yaml

tap-agent:
container_name: tap-agent
build: {
- target: "wrapper", # Set to "wrapper-dev" for building from source
+ target: "wrapper-dev", # Set to "wrapper-dev" for building from source
context: tap-agent,
}
# Multiple overrides
COMPOSE_FILE=docker-compose.yaml:compose/dev/indexer-service.yaml:compose/dev/tap-agent.yaml
```

Each override requires a binary path env var. Source repos own their own build;
local-network just wraps the published image with `run.sh` and utilities.
See [compose/dev/README.md](compose/dev/README.md) for details.

## Common issues

### `too far behind`
Expand Down
35 changes: 35 additions & 0 deletions compose/dev/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Dev Overrides

Compose override files for local development. Most mount a locally-built binary
into the running container, avoiding full image rebuilds.

## Usage

Set `COMPOSE_FILE` in `.env` (or `.env.local`) to include the override:

```bash
COMPOSE_FILE=docker-compose.yaml:compose/dev/graph-node.yaml
```

Chain multiple overrides:

```bash
COMPOSE_FILE=docker-compose.yaml:compose/dev/graph-node.yaml:compose/dev/indexer-agent.yaml
```

Then `docker compose up -d` applies the overrides automatically.

## Available Overrides

| File | Service | Required Env Var |
| ------------------------- | -------------------------------- | ------------------------------------------------------ |
| `graph-node.yaml` | graph-node | `GRAPH_NODE_SOURCE_ROOT` |
| `graph-contracts.yaml` | graph-contracts, subgraph-deploy | `CONTRACTS_SOURCE_ROOT`, `GRAPH_CONTRACTS_SOURCE_ROOT` |
| `indexer-agent.yaml` | indexer-agent | `INDEXER_AGENT_SOURCE_ROOT` |
| `indexer-service.yaml` | indexer-service | `INDEXER_SERVICE_BINARY` |
| `tap-agent.yaml` | tap-agent | `TAP_AGENT_BINARY` |
| `eligibility-oracle.yaml` | eligibility-oracle-node | `REO_BINARY` |
| `dipper.yaml` | dipper | `DIPPER_BINARY` |
| `iisa.yaml` | iisa | `IISA_VERSION=local` |

See each file's header comments for details.
17 changes: 17 additions & 0 deletions compose/dev/dipper.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Dipper Dev Override
# Mounts a locally-built binary for WIP development (skip image rebuild).
#
# Set DIPPER_BINARY to the path of the locally-built binary, e.g.:
# DIPPER_BINARY=/path/to/dipper/target/release/dipper-service
#
# Build the binary locally first:
# cargo build --release --bin dipper-service
#
# Activate via COMPOSE_FILE in .env (requires indexing-payments profile):
# COMPOSE_PROFILES=indexing-payments,block-oracle
# COMPOSE_FILE=docker-compose.yaml:compose/dev/dipper.yaml

services:
dipper:
volumes:
- ${DIPPER_BINARY:?Set DIPPER_BINARY to locally-built dipper-service binary}:/usr/local/bin/dipper-service:ro
17 changes: 17 additions & 0 deletions compose/dev/eligibility-oracle.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Eligibility Oracle Dev Override
# Mounts a locally-built binary for WIP development (skip image rebuild).
#
# Set REO_BINARY to the path of the locally-built binary, e.g.:
# REO_BINARY=/git/local/eligibility-oracle-node/eligibility-oracle-node/target/release/eligibility-oracle
#
# Build the binary locally first:
# cargo build --release -p eligibility-oracle
#
# Activate via COMPOSE_FILE in .env (requires rewards-eligibility profile):
# COMPOSE_PROFILES=rewards-eligibility
# COMPOSE_FILE=docker-compose.yaml:compose/dev/eligibility-oracle.yaml

services:
eligibility-oracle-node:
volumes:
- ${REO_BINARY:?Set REO_BINARY to locally-built eligibility-oracle binary}:/usr/local/bin/eligibility-oracle:ro
20 changes: 20 additions & 0 deletions compose/dev/graph-contracts.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Graph Contracts Dev Override
# Mounts local contracts repo for WIP development (skip image rebuild).
#
# Set CONTRACTS_SOURCE_ROOT to the local contracts repo path, e.g.:
# CONTRACTS_SOURCE_ROOT=/git/graphprotocol/contracts/post-audit
# The repo must have pnpm install and pnpm build already run.
#
# Set GRAPH_CONTRACTS_SOURCE_ROOT to the local graph-network-subgraph repo, e.g.:
# GRAPH_CONTRACTS_SOURCE_ROOT=/git/graphprotocol/graph-network-subgraph
#
# Activate via COMPOSE_FILE in .env:
# COMPOSE_FILE=docker-compose.yaml:compose/dev/graph-contracts.yaml

services:
graph-contracts:
volumes:
- ${CONTRACTS_SOURCE_ROOT:?Set CONTRACTS_SOURCE_ROOT to local contracts repo}:/opt/contracts
subgraph-deploy:
volumes:
- ${GRAPH_CONTRACTS_SOURCE_ROOT:?Set GRAPH_CONTRACTS_SOURCE_ROOT to local graph-network-subgraph repo}:/opt/graph-network-subgraph
Original file line number Diff line number Diff line change
@@ -1,11 +1,20 @@
# Graph Node Dev Override
# Builds graph-node from local source with optional gdb debugging.
#
# Set GRAPH_NODE_SOURCE_ROOT to the local repo path, e.g.:
# GRAPH_NODE_SOURCE_ROOT=/path/to/graph-node
#
# Activate via COMPOSE_FILE in .env:
# COMPOSE_FILE=docker-compose.yaml:compose/dev/graph-node.yaml

services:
graph-node:
entrypoint: bash -cl /opt/run-override.sh
build:
context: "./overrides/graph-node-dev"
build:
context: "./containers/indexer/graph-node/dev"
dockerfile: Dockerfile
volumes:
- ./overrides/graph-node-dev/run-override.sh:/opt/run-override.sh:ro
- ./containers/indexer/graph-node/dev/run-override.sh:/opt/run-override.sh:ro
- /tmp/graph-node-cargo-home:/tmp/graph-node-cargo-home
- /tmp/graph-node-docker-build:/tmp/graph-node-docker-build
- ${GRAPH_NODE_SOURCE_ROOT}:/opt/graph-node-source-root
Expand Down
13 changes: 13 additions & 0 deletions compose/dev/iisa.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# IISA Dev Override
# Uses a locally-built image instead of pulling from GHCR.
#
# Build the image in the IISA repo first:
# cd /path/to/subgraph-dips-indexer-selection && docker compose build
#
# Then set IISA_VERSION=local in .env and activate via COMPOSE_FILE:
# COMPOSE_PROFILES=indexing-payments,block-oracle
# COMPOSE_FILE=docker-compose.yaml:compose/dev/iisa.yaml

services:
iisa:
image: ghcr.io/edgeandnode/subgraph-dips-indexer-selection:local
19 changes: 19 additions & 0 deletions compose/dev/indexer-agent.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Indexer Agent Dev Override
# Mounts local indexer-agent source for hot-reload development.
#
# Set INDEXER_AGENT_SOURCE_ROOT to the local repo path, e.g.:
# INDEXER_AGENT_SOURCE_ROOT=$HOME/Development/en/indexer
#
# Activate via COMPOSE_FILE in .env:
# COMPOSE_FILE=docker-compose.yaml:compose/dev/indexer-agent.yaml

services:
indexer-agent:
entrypoint: bash -cl /opt/run-override.sh
ports:
- "${INDEXER_MANAGEMENT}:7600"
# Nodejs debugger
- 9230:9230
volumes:
- ./containers/indexer/indexer-agent/dev/run-override.sh:/opt/run-override.sh:ro
- ${INDEXER_AGENT_SOURCE_ROOT}:/opt/indexer-agent-source-root
16 changes: 16 additions & 0 deletions compose/dev/indexer-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Indexer Service Dev Override
# Mounts a locally-built binary for WIP development (skip image rebuild).
#
# Set INDEXER_SERVICE_BINARY to the path of the locally-built binary, e.g.:
# INDEXER_SERVICE_BINARY=/path/to/indexer-rs/target/release/indexer-service-rs
#
# Build the binary locally first:
# cargo build --release --bin indexer-service-rs
#
# Activate via COMPOSE_FILE in .env:
# COMPOSE_FILE=docker-compose.yaml:compose/dev/indexer-service.yaml

services:
indexer-service:
volumes:
- ${INDEXER_SERVICE_BINARY:?Set INDEXER_SERVICE_BINARY to locally-built indexer-service-rs binary}:/usr/local/bin/indexer-service-rs:ro
Loading