feat(tools/talis): vendor talis deployment tool + Fibre experiment runner

Brings the celestia-app talis multi-cloud deploy tool into ev-node,
plus the wiring needed to deploy a working Fibre DA aggregator
end-to-end on top of it. Verified via a fresh AWS run from this
branch state — talis up → genesis → deploy → setup-fibre → start-fibre
→ fibre-bootstrap-evnode → loadgen reaches 24.57 MB/s @ 99.7 % ok
on 60 s (3-validator c6in.4xlarge + c6in.2xlarge bridge + c6in.8xlarge
ev-node + c6in.2xlarge load-gen, all us-east-1).

What this adds:

  • tools/talis/                — vendored from celestia-app's
    feat/fibre-payments. Provisions AWS / DO / GCP boxes for one or
    more validators + bridge + ev-node + load-gen, deploys binaries
    + init scripts, drives a Fibre setup-fibre + start-fibre flow,
    and ships an evnode-bootstrap step that scp's the bridge JWT and
    Fibre payment keyring onto each ev-node before its init script
    starts the daemon.
  • tools/celestia-node-fiber/cmd/evnode-fibre/  — long-lived
    aggregator runner that wires block.NewFiberDAClient on top of
    the celestia-node-fiber adapter. Compiled on each `talis genesis`
    and shipped to evnode-* hosts.
  • tools/talis/cmd/evnode-txsim/ — small Go load-gen that pumps the
    runner's HTTP /tx ingress for a fixed duration; deployed to the
    load-gen boxes and prints a single TXSIM: line on completion.
  • tools/talis/Makefile         — cross-compiles celestia-appd, the
    fibre server + load tool, the bridge/light celestia binary, and
    both runner binaries to linux/amd64 for talis genesis -b.

setup-fibre fixes uncovered during the verified run:

  • bash script for set-host now retries until the validator's host
    appears in `query valaddr providers`. The previous one-shot call
    relied on `--yes` returning the txhash before block inclusion;
    if the chain wasn't ready (validator just-started, mempool full,
    parallel set-host on other validators contending) the tx silently
    bounced and the validator never registered. Fibre client cached
    the partial set on startup and uploads cascaded to "host not
    found" → "voting power: collected 0".
  • talis-CLI side polls `query valaddr providers` after the per-
    validator scripts finish and refuses to return until all
    validators are registered. Up to 5-minute deadline, then errors
    so the operator can re-run instead of pressing on with a broken
    Fibre setup.

What this changes in ev-node itself (load-bearing for the runner):

  • pkg/config/config.go: new ApplyFiberDefaults() profile that flips
    the DA client to Fibre-friendly defaults (adaptive batching,
    1 s DA.BlockTime, 50-deep pending-cache window) so a runner can
    opt in with one call.
  • block/public.go: SetMaxBlobSize override so the runner can lift
    Celestia's 5 MiB default to Fibre's 120 MiB headroom; new
    NewFiberDAClient + Fibre type re-exports for the adapter.
  • block/internal/da/{fiber_client.go,fiber/types.go,fibremock/}:
    Fibre adapter wired through the DA client interface, with a
    matching mock used by the testing tree.
  • block/internal/submitting/da_submitter.go: per-stream upload
    workers, splitByBlobSize chunking, parallel signing pool, and
    an oversized-blob safety net that advances the cache instead
    of looping forever (which OOM'd the daemon under sustained
    Fibre stalls).
  • core/sequencer/sequencing.go: ErrQueueFull sentinel for sequencer
    backpressure; pkg/sequencers/solo/sequencer.go honours a
    configurable mempool cap and returns it on overflow so the
    reaper can back off instead of feeding an unbounded queue
    while the executor is paused on the pending-cache cap.

External dependency (documented in tools/talis/fibre.md):

  • Sibling clone of celestia-app on a branch with feat/fibre-payments
    + sysrex/fibre_url_fix cherry-picked. Without the URL-parse fix
    the Fibre client rejects every host:port registration.

Tested:
  - go test ./block/... ./pkg/... ./types/... — all green
  - go build ./... + Makefile build-bins — all 6 binaries cross-
    compile cleanly to linux/amd64
  - End-to-end AWS run from this branch state — TXSIM 24.57 MB/s,
    99.7 % ok on a 60 s sustained run

Author: walldiss

.claude/scheduled_tasks.lock

@@ -0,0 +1 @@
+{"sessionId":"e50bfb66-350f-42ce-a67f-1053bf478384","pid":32397,"procStart":"Sun Apr 26 13:39:16 2026","acquiredAt":1777338483302}

.gitignore

@@ -1,5 +1,7 @@
 coverage.txt
 *.out
+go.work
+go.work.sum
 proto/pb
 proto/tendermint
 types/pb/tendermint

apps/evm/cmd/run.go

@@ -131,7 +131,10 @@ var RunCmd = &cobra.Command{
 			}()
 		}
 
-		return rollcmd.StartNode(logger, cmd, executor, sequencer, nodeKey, datastore, nodeConfig, genesis, node.NodeOptions{})
+		// fibre-experiment: StartNode now takes a FiberClient as the
+		// trailing arg. The EVM app doesn't wire Fiber — nil is fine
+		// as long as DA.Fiber.Enabled stays false in its config.
+		return rollcmd.StartNode(logger, cmd, executor, sequencer, nodeKey, datastore, nodeConfig, genesis, node.NodeOptions{}, nil)
 	},
 }
 

apps/grpc/cmd/run.go

@@ -86,8 +86,10 @@ The execution client must implement the Evolve execution gRPC interface.`,
 			return err
 		}
 
-		// Start the node
-		return rollcmd.StartNode(logger, cmd, executor, sequencer, nodeKey, datastore, nodeConfig, genesis, node.NodeOptions{})
+		// Start the node. fibre-experiment: StartNode now takes a
+		// FiberClient as the trailing arg. The grpc app doesn't wire
+		// Fiber — nil is fine as long as DA.Fiber.Enabled stays false.
+		return rollcmd.StartNode(logger, cmd, executor, sequencer, nodeKey, datastore, nodeConfig, genesis, node.NodeOptions{}, nil)
 	},
 }
 

apps/testapp/cmd/run.go

@@ -97,7 +97,12 @@ var RunCmd = &cobra.Command{
 			return err
 		}
 
-		return cmd.StartNode(logger, command, executor, sequencer, nodeKey, datastore, nodeConfig, genesis, node.NodeOptions{})
+		// fibre-experiment: StartNode now takes a FiberClient as the
+		// trailing arg. testapp doesn't wire Fiber here (the Fiber-
+		// aware runner lives in tools/talis/cmd/evnode-fibre); nil is
+		// fine as long as DA.Fiber.Enabled stays false in testapp's
+		// config.
+		return cmd.StartNode(logger, command, executor, sequencer, nodeKey, datastore, nodeConfig, genesis, node.NodeOptions{}, nil)
 	},
 }
 

block/internal/cache/manager.go

@@ -76,10 +76,8 @@ type PendingManager interface {
 	GetPendingData(ctx context.Context) ([]*types.SignedData, [][]byte, error)
 	SetLastSubmittedHeaderHeight(ctx context.Context, height uint64)
 	GetLastSubmittedHeaderHeight() uint64
-	ResetInFlightHeaderRange(start, end uint64)
 	SetLastSubmittedDataHeight(ctx context.Context, height uint64)
 	GetLastSubmittedDataHeight() uint64
-	ResetInFlightDataRange(start, end uint64)
 	NumPendingHeaders() uint64
 	NumPendingData() uint64
 }
@@ -312,14 +310,6 @@ func (m *implementation) SetLastSubmittedHeaderHeight(ctx context.Context, heigh
 	m.pendingHeaders.SetLastSubmittedHeaderHeight(ctx, height)
 }
 
-func (m *implementation) ResetInFlightHeaderHeight() {
-	m.pendingHeaders.ResetInFlightHeaderRange(0, 0)
-}
-
-func (m *implementation) ResetInFlightHeaderRange(start, end uint64) {
-	m.pendingHeaders.ResetInFlightHeaderRange(start, end)
-}
-
 func (m *implementation) GetLastSubmittedDataHeight() uint64 {
 	return m.pendingData.GetLastSubmittedDataHeight()
 }
@@ -328,10 +318,6 @@ func (m *implementation) SetLastSubmittedDataHeight(ctx context.Context, height
 	m.pendingData.SetLastSubmittedDataHeight(ctx, height)
 }
 
-func (m *implementation) ResetInFlightDataRange(start, end uint64) {
-	m.pendingData.ResetInFlightDataRange(start, end)
-}
-
 func (m *implementation) NumPendingHeaders() uint64 {
 	return m.pendingHeaders.NumPendingHeaders()
 }

block/internal/cache/manager_test.go

@@ -221,12 +221,6 @@ func TestPendingHeadersAndData_Flow(t *testing.T) {
 	// update last submitted heights and re-check
 	cm.SetLastSubmittedHeaderHeight(ctx, 1)
 	cm.SetLastSubmittedDataHeight(ctx, 2)
-	cm.ResetInFlightHeaderRange(1, 3)
-	cm.ResetInFlightDataRange(2, 3)
-
-	// numPending views (before getPending claims items)
-	assert.Equal(t, uint64(2), cm.NumPendingHeaders())
-	assert.Equal(t, uint64(1), cm.NumPendingData())
 
 	headers, _, err = cm.GetPendingHeaders(ctx)
 	require.NoError(t, err)
@@ -237,6 +231,10 @@ func TestPendingHeadersAndData_Flow(t *testing.T) {
 	require.NoError(t, err)
 	require.Len(t, signedData, 1)
 	assert.Equal(t, uint64(3), signedData[0].Height())
+
+	// numPending views
+	assert.Equal(t, uint64(2), cm.NumPendingHeaders())
+	assert.Equal(t, uint64(1), cm.NumPendingData())
 }
 
 func TestManager_TxOperations(t *testing.T) {