Skip to content

Commit ef84e01

Browse files
committed
perf(fiber-da): skip flatten allocation on single-item Submit; honor ctx
Three changes in fiber_client.go::Submit, all hot-path correctness/ efficiency wins surfaced while debugging Fibre throughput: 1. Single-item fast path that bypasses flattenBlobs. For data blobs, limitBatchBySize already caps each Submit call at one item (each block's data already saturates MaxBlobBytes). The flatten step was therefore allocating MaxBlobSize bytes and memcpy'ing the entire payload solely to prepend the 8-byte count/length prefix used by splitBlobs. At 128 MiB blocks that's ~128 MB held in two places at once during every Upload. The fast path passes data[0] straight through and saves the full copy. Wire-format caveat: a retriever (full-node syncer or light client) downloading a blob written via this fast path can't decode it — splitBlobs always expects the prefix. The right fix is to pair this with a per-item Upload model so flatten falls away entirely; tracked as a TODO in the source pointing at the concurrent-uploads work where that lands naturally. 2. Honor caller's ctx in Upload. The previous context.Background() kept Uploads alive past node shutdown and was the proximate cause of the "payment promise already processed" warnings — a stale Upload would settle on-chain after ev-node had already moved on. Threading the caller's ctx makes shutdown promptly cancel in-flight Uploads. 3. Correct SubmittedCount on error. On a full-Upload failure the result reported len(data)-1 as submitted, which both reads weirdly for len==1 (uint64 underflow risk in any future arithmetic) and lies to submitToDA's prefix-of-success retry advance. Reset to 0 on error. No behaviour change for the multi-item retrieve path (flatten still runs when len > 1). Validated via go build / go vet.
1 parent 44e977a commit ef84e01

1 file changed

Lines changed: 29 additions & 5 deletions

File tree

block/internal/da/fiber_client.go

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -87,9 +87,30 @@ func (c *fiberDAClient) Submit(ctx context.Context, data [][]byte, _ float64, na
8787
}
8888
}
8989

90-
flat := flattenBlobs(data)
90+
// Single-item fast path: avoid the MaxBlobSize-sized allocation +
91+
// memcpy that flattenBlobs would do just to wrap one item in the
92+
// 8-byte count/length prefix. With per-item caps already saturating
93+
// MaxBlobBytes for data blobs, this is the steady-state path.
94+
//
95+
// TODO: wire-format compat — splitBlobs always expects the prefix,
96+
// so any retriever (full node syncer, light client) downloading a
97+
// blob written via this fast path will fail to decode. Address
98+
// alongside the concurrent-uploads change by switching to a
99+
// per-item Upload model where flatten is no longer needed.
100+
var blob []byte
101+
if len(data) == 1 {
102+
blob = data[0]
103+
} else {
104+
blob = flattenBlobs(data)
105+
}
91106

92-
result, err := c.fiber.Upload(context.Background(), namespace[len(namespace)-10:], flat)
107+
// Honor the caller's context so Upload returns promptly on
108+
// shutdown / parent cancellation. The previous context.Background()
109+
// kept Uploads alive past node shutdown and contributed to the
110+
// "payment promise already processed" warnings we saw in early
111+
// runs (a stale Upload would settle after the node had stopped
112+
// tracking it).
113+
result, err := c.fiber.Upload(ctx, namespace[len(namespace)-10:], blob)
93114
if err != nil {
94115
code := datypes.StatusError
95116
switch {
@@ -103,9 +124,12 @@ func (c *fiberDAClient) Submit(ctx context.Context, data [][]byte, _ float64, na
103124

104125
return datypes.ResultSubmit{
105126
BaseResult: datypes.BaseResult{
106-
Code: code,
107-
Message: fmt.Sprintf("fiber upload failed for blob: %v", err),
108-
SubmittedCount: uint64(len(data) - 1),
127+
Code: code,
128+
Message: fmt.Sprintf("fiber upload failed for blob: %v", err),
129+
// On error nothing settled — the previous len(data)-1
130+
// reported all-but-one as submitted on full failure,
131+
// which lied to the caller's retry/postSubmit logic.
132+
SubmittedCount: 0,
109133
BlobSize: blobSize,
110134
Timestamp: time.Now(),
111135
},

0 commit comments

Comments
 (0)