Skip to content

Commit a88b176

Browse files
walldissclaude
andauthored
test(celestia-node-fiber): in-process Upload/Listen/Download showcase (#3280)
* test(celestia-node-fiber): showcase end-to-end Upload/Listen/Download Adds tools/celestia-node-fiber/testing/, a single-validator in-process showcase that boots a fibre-tagged Celestia chain + in-process Fibre server + celestia-node bridge, registers the validator's FSP via valaddr (with the dns:/// URI scheme the client's gRPC resolver expects), funds an escrow account, and drives the full adapter surface. TestShowcase proves the round-trip: subscribe via Listen, Upload a blob, wait for the share-version-2 BlobEvent that lands after the async MsgPayForFibre commits, assert the BlobID from Listen matches Upload's return, Download and diff the payload bytes. The harness is intentionally single-validator — a 2-validator Docker Compose showcase is planned as a follow-up for exercising real quorum collection. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(celestia-node-fiber): scale showcase to 10 blobs, document DataSize gap Upload 10 distinct-payload blobs through adapter.Upload, collect BlobEvents via adapter.Listen until every BlobID is accounted for (order-insensitive, rejects duplicates), then round-trip each blob through adapter.Download to diff bytes. Catches routing bugs (wrong blob returned for a BlobID) and duplicate-event bugs that a single-blob test can't see. Scaling the test also exposed a semantic issue: the v2 share carries only (fibre_blob_version + commitment), so b.DataLen() — what listen.go's fibreBlobToEvent reports today — is always 36, not the original payload length ev-node's fibermock conveys. The adapter can't derive the payload size from the subscription stream alone; surfacing it correctly needs an x/fibre PaymentPromise lookup (tracked as a TODO on fibreBlobToEvent). The test therefore asserts DataSize is non-zero rather than matching len(payload). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent a4a46e7 commit a88b176

6 files changed

Lines changed: 549 additions & 4 deletions

File tree

tools/celestia-node-fiber/go.mod

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -30,12 +30,16 @@ replace (
3030
replace github.com/evstack/ev-node => ../../.
3131

3232
require (
33+
cosmossdk.io/math v1.5.3
3334
github.com/celestiaorg/celestia-app/v8 v8.0.0-20260408095837-ab08b6d5e54e
3435
github.com/celestiaorg/celestia-node v0.0.0-20260423103931-b07242fbfec8
3536
github.com/celestiaorg/go-square/v4 v4.0.0-rc4.0.20260318002530-1ca8ff7b42ea
37+
github.com/cometbft/cometbft v1.0.1
3638
github.com/cosmos/cosmos-sdk v0.50.13
39+
github.com/cristalhq/jwt/v5 v5.4.0
3740
github.com/evstack/ev-node v1.1.0
3841
github.com/stretchr/testify v1.11.1
42+
go.uber.org/fx v1.24.0
3943
)
4044

4145
require (
@@ -58,7 +62,6 @@ require (
5862
cosmossdk.io/depinject v1.2.1 // indirect
5963
cosmossdk.io/errors v1.0.2 // indirect
6064
cosmossdk.io/log v1.6.1 // indirect
61-
cosmossdk.io/math v1.5.3 // indirect
6265
cosmossdk.io/store v1.1.2 // indirect
6366
cosmossdk.io/x/circuit v0.1.1 // indirect
6467
cosmossdk.io/x/evidence v0.1.1 // indirect
@@ -70,6 +73,7 @@ require (
7073
filippo.io/keygen v0.0.0-20260114151900-8e2790ea4c5b // indirect
7174
github.com/99designs/go-keychain v0.0.0-20191008050251-8e49817e8af4 // indirect
7275
github.com/99designs/keyring v1.2.2 // indirect
76+
github.com/BurntSushi/toml v1.6.0 // indirect
7377
github.com/DataDog/datadog-go v4.8.3+incompatible // indirect
7478
github.com/DataDog/zstd v1.5.7 // indirect
7579
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.31.0 // indirect
@@ -131,7 +135,6 @@ require (
131135
github.com/cockroachdb/redact v1.1.6 // indirect
132136
github.com/cockroachdb/swiss v0.0.0-20251224182025-b0f6560f979b // indirect
133137
github.com/cockroachdb/tokenbucket v0.0.0-20250429170803-42689b6311bb // indirect
134-
github.com/cometbft/cometbft v1.0.1 // indirect
135138
github.com/cometbft/cometbft-db v1.0.4 // indirect
136139
github.com/consensys/gnark v0.14.0 // indirect
137140
github.com/consensys/gnark-crypto v0.19.2 // indirect
@@ -147,7 +150,6 @@ require (
147150
github.com/cosmos/ibc-go/v8 v8.7.0 // indirect
148151
github.com/cosmos/ics23/go v0.11.0 // indirect
149152
github.com/cosmos/ledger-cosmos-go v0.15.0 // indirect
150-
github.com/cristalhq/jwt/v5 v5.4.0 // indirect
151153
github.com/cskr/pubsub v1.0.2 // indirect
152154
github.com/danieljoos/wincred v1.2.1 // indirect
153155
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
@@ -184,6 +186,7 @@ require (
184186
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
185187
github.com/goccy/go-yaml v1.19.2 // indirect
186188
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
189+
github.com/gofrs/flock v0.13.0 // indirect
187190
github.com/gogo/googleapis v1.4.1 // indirect
188191
github.com/gogo/protobuf v1.3.3 // indirect
189192
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
@@ -232,6 +235,7 @@ require (
232235
github.com/huin/goupnp v1.3.0 // indirect
233236
github.com/iancoleman/orderedmap v0.3.0 // indirect
234237
github.com/iancoleman/strcase v0.3.0 // indirect
238+
github.com/imdario/mergo v0.3.16 // indirect
235239
github.com/improbable-eng/grpc-web v0.15.0 // indirect
236240
github.com/inconshreveable/mousetrap v1.1.0 // indirect
237241
github.com/ingonyama-zk/icicle-gnark/v3 v3.2.2 // indirect
@@ -335,6 +339,7 @@ require (
335339
github.com/quic-go/webtransport-go v0.10.0 // indirect
336340
github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 // indirect
337341
github.com/rogpeppe/go-internal v1.14.1 // indirect
342+
github.com/rollkit/go-da v0.9.0 // indirect
338343
github.com/ronanh/intcomp v1.1.1 // indirect
339344
github.com/rs/cors v1.11.1 // indirect
340345
github.com/rs/zerolog v1.35.0 // indirect
@@ -363,9 +368,11 @@ require (
363368
go.etcd.io/bbolt v1.4.0 // indirect
364369
go.opencensus.io v0.24.0 // indirect
365370
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
371+
go.opentelemetry.io/contrib/bridges/prometheus v0.67.0 // indirect
366372
go.opentelemetry.io/contrib/detectors/gcp v1.39.0 // indirect
367373
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.63.0 // indirect
368374
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.65.0 // indirect
375+
go.opentelemetry.io/contrib/instrumentation/runtime v0.67.0 // indirect
369376
go.opentelemetry.io/otel v1.43.0 // indirect
370377
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.43.0 // indirect
371378
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
@@ -377,7 +384,6 @@ require (
377384
go.opentelemetry.io/otel/trace v1.43.0 // indirect
378385
go.opentelemetry.io/proto/otlp v1.10.0 // indirect
379386
go.uber.org/dig v1.19.0 // indirect
380-
go.uber.org/fx v1.24.0 // indirect
381387
go.uber.org/mock v0.5.2 // indirect
382388
go.uber.org/multierr v1.11.0 // indirect
383389
go.uber.org/zap v1.27.1 // indirect

tools/celestia-node-fiber/listen.go

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,14 @@ func resolveHeight(resp *blob.SubscriptionResponse) uint64 {
8484
// fibreBlobToEvent reconstructs the Fibre BlobID (version byte + 32-byte
8585
// commitment) from a share-version-2 libshare.Blob and wraps it as a
8686
// BlobEvent.
87+
//
88+
// DataSize caveat: a v2 share carries only (fibre_blob_version + commitment),
89+
// not the original blob payload, so b.DataLen() is the on-chain share size
90+
// (a fixed constant), not the user-facing "how big is this blob" number
91+
// that ev-node's fibermock and its consumers typically expect. Reporting
92+
// the true payload size requires an on-chain query against x/fibre's
93+
// PaymentPromise keyed by commitment. Tracked as a follow-up; for now we
94+
// report the share size so the field is non-zero.
8795
func fibreBlobToEvent(b *libshare.Blob, height uint64) (block.FiberBlobEvent, error) {
8896
version, err := b.FibreBlobVersion()
8997
if err != nil {
Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
//go:build fibre
2+
3+
package cnfibertest
4+
5+
import (
6+
"context"
7+
"crypto/rand"
8+
"net"
9+
"testing"
10+
"time"
11+
12+
"github.com/cosmos/cosmos-sdk/crypto/hd"
13+
"github.com/cosmos/cosmos-sdk/crypto/keyring"
14+
"github.com/cristalhq/jwt/v5"
15+
"github.com/stretchr/testify/require"
16+
"go.uber.org/fx"
17+
18+
appfibre "github.com/celestiaorg/celestia-app/v8/fibre"
19+
20+
"github.com/celestiaorg/celestia-node/api/client"
21+
"github.com/celestiaorg/celestia-node/api/rpc/perms"
22+
"github.com/celestiaorg/celestia-node/nodebuilder"
23+
"github.com/celestiaorg/celestia-node/nodebuilder/node"
24+
"github.com/celestiaorg/celestia-node/nodebuilder/p2p"
25+
stateapi "github.com/celestiaorg/celestia-node/nodebuilder/state"
26+
)
27+
28+
// Bridge bundles an in-process celestia-node bridge node and the admin
29+
// JWT that grants it authenticated RPC access. The adapter's ReadConfig
30+
// needs both the address and the token for Blob.Subscribe to work.
31+
type Bridge struct {
32+
Node *nodebuilder.Node
33+
AdminToken string
34+
}
35+
36+
// RPCAddr returns a WebSocket URL the adapter uses in
37+
// Config.ReadConfig.BridgeDAAddr. WebSocket (not HTTP) is required
38+
// because Blob.Subscribe returns a channel; go-jsonrpc only supports
39+
// channel-returning methods over a streaming transport.
40+
func (b *Bridge) RPCAddr() string {
41+
return "ws://" + b.Node.RPCServer.ListenAddr()
42+
}
43+
44+
// StartBridge brings up an in-process celestia-node bridge connected to
45+
// the Network's consensus gRPC endpoint. Mirrors celestia-node's
46+
// api/client test helpers so TestShowcase has a real JSON-RPC server for
47+
// Blob.Subscribe.
48+
func StartBridge(t *testing.T, ctx context.Context, network *Network) *Bridge {
49+
t.Helper()
50+
51+
cfg := nodebuilder.DefaultConfig(node.Bridge)
52+
53+
ip, port, err := net.SplitHostPort(network.ConsensusGRPCAddr())
54+
require.NoError(t, err, "splitting consensus gRPC addr")
55+
cfg.Core.IP = ip
56+
cfg.Core.Port = port
57+
// Pin the bridge RPC to an ephemeral port; the test discovers it via
58+
// Node.RPCServer.ListenAddr() after Start.
59+
cfg.RPC.Port = "0"
60+
61+
tempDir := t.TempDir()
62+
store := nodebuilder.MockStore(t, cfg)
63+
64+
auth, adminToken := bridgeAuth(t)
65+
kr := bridgeKeyring(t, tempDir)
66+
67+
bn, err := nodebuilder.New(node.Bridge, p2p.Private, store,
68+
auth,
69+
stateapi.WithKeyring(kr),
70+
stateapi.WithKeyName(stateapi.AccountName(bridgeSigningKey)),
71+
fx.Replace(node.StorePath(tempDir)),
72+
)
73+
require.NoError(t, err, "constructing bridge node")
74+
75+
require.NoError(t, bn.Start(ctx), "starting bridge node")
76+
t.Cleanup(func() {
77+
stopCtx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
78+
defer cancel()
79+
_ = bn.Stop(stopCtx)
80+
})
81+
82+
return &Bridge{
83+
Node: bn,
84+
AdminToken: adminToken,
85+
}
86+
}
87+
88+
// bridgeSigningKey is the keyring account the bridge uses for its own
89+
// tx submissions. Distinct from the client's account so the two keyrings
90+
// don't collide.
91+
const bridgeSigningKey = "bridge-signer"
92+
93+
func bridgeKeyring(t *testing.T, tempDir string) keyring.Keyring {
94+
t.Helper()
95+
96+
kr, err := client.KeyringWithNewKey(client.KeyringConfig{
97+
KeyName: bridgeSigningKey,
98+
BackendName: keyring.BackendTest,
99+
}, tempDir)
100+
require.NoError(t, err, "creating bridge keyring")
101+
102+
// The Fibre module on the bridge expects a key under
103+
// appfibre.DefaultKeyName to exist, even though our client never uses
104+
// the bridge's Fibre module for Upload/Download. Without it,
105+
// fx.Start fails during fibre module wiring.
106+
_, _, err = kr.NewMnemonic(
107+
appfibre.DefaultKeyName,
108+
keyring.English, "", "", hd.Secp256k1,
109+
)
110+
require.NoError(t, err, "provisioning bridge fibre key")
111+
return kr
112+
}
113+
114+
// bridgeAuth creates an HS256 JWT signer pair and an admin token. The
115+
// returned fx option injects the signer/verifier into the node; the
116+
// token is what the adapter puts in ReadConfig.DAAuthToken.
117+
func bridgeAuth(t *testing.T) (fx.Option, string) {
118+
t.Helper()
119+
120+
key := make([]byte, 32)
121+
_, err := rand.Read(key)
122+
require.NoError(t, err, "rand.Read jwt key")
123+
124+
signer, err := jwt.NewSignerHS(jwt.HS256, key)
125+
require.NoError(t, err)
126+
verifier, err := jwt.NewVerifierHS(jwt.HS256, key)
127+
require.NoError(t, err)
128+
129+
token, err := perms.NewTokenWithPerms(signer, perms.AllPerms)
130+
require.NoError(t, err)
131+
132+
return fx.Decorate(func() (jwt.Signer, jwt.Verifier, error) {
133+
return signer, verifier, nil
134+
}), string(token)
135+
}
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
//go:build fibre
2+
3+
// Package cnfibertest wires a single-validator Celestia chain, an in-process
4+
// Fibre server, a celestia-node bridge and the celestia-node-fiber adapter
5+
// together so Upload → Listen → Download can be exercised end-to-end in a
6+
// Go test.
7+
//
8+
// The chain is a celestia-app testnode built with -tags fibre. The Fibre
9+
// server runs in the same process and its FSP endpoint is registered with
10+
// the valaddr module so the client's host registry can find it. The
11+
// underlying adapter talks directly to consensus gRPC and the Fibre server;
12+
// only Listen goes through the bridge node's blob subscription.
13+
//
14+
// This is the "fast sanity" variant. A multi-validator showcase is planned
15+
// as a Docker Compose follow-up that exercises real quorum collection.
16+
package cnfibertest

0 commit comments

Comments
 (0)