MQ-1202 Include metrics metadata in queue() handler message batch by KennethRuan · Pull Request #6339 · cloudflare/workerd

KennethRuan · 2026-03-16T20:39:15Z

Summary

Adds a new metadata property to queue message batches (MessageBatch / QueueEvent), exposing queue metrics with each batch delivered to a worker's queue() handler.

Changes

This PR includes changes to queue.c++ and queue.h to include metrics on each message batch. The metadata contains three metrics from the upstream queue broker worker (backlogCount, backlogBytes, oldestMessageTimestamp). Below is what the generated types will look like:

// NEW
interface MessageBatchMetrics {
  backlogCount: number;
  backlogBytes: number;
  oldestMessageTimestamp: number;
}
// NEW
interface MessageBatchMetadata { 
  metrics: MessageBatchMetrics;
}

interface QueueEvent<Body = unknown> extends ExtendableEvent {
  readonly messages: readonly Message<Body>[];
  readonly queue: string;
  readonly metadata: MessageBatchMetadata;  // NEW FIELD
  retryAll(options?: QueueRetryOptions): void;
  ackAll(): void;
}
interface MessageBatch<Body = unknown> {
  readonly messages: readonly Message<Body>[];
  readonly queue: string;
  readonly metadata: MessageBatchMetadata;  // NEW FIELD
  retryAll(options?: QueueRetryOptions): void;
  ackAll(): void;
}

The new type changes are gated behind a queue_batch_metadata compat flag so that it can be tested internally before release. Additionally, changes were made to http.c++ and http.h so that the Fetcher used for testing can accept an optional metadata argument.

Upstream changes can be found here: https://gitlab.cfdata.org/cloudflare/mq/queue-broker-worker/-/merge_requests/1765

Testing

bazel test //src/workerd/api/tests:queue-test@
bazel test //src/workerd/api/tests:queue-test@all-compat-flags
bazel test //src/workerd/api/tests:queue-metadata-test@
bazel test //src/workerd/api/tests:queue-metadata-test@all-compat-flags
bazel test //types:test/types/rpc

jasnell · 2026-03-17T13:49:07Z

/bigbonk review this

ask-bonk

Review summary

Overall this is a solid, well-structured change. The compat flag gating, capnp schema additions, and test coverage are all done correctly. A few items to address:

[Medium] backlogCount and backlogBytes would read more naturally as integer types rather than double at the JS API level — consider whether the upstream data truly requires floating-point representation.
[Low] QueueEvent::getMetadata() and QueueController::getMetadata() return MessageBatchMetadata by value, creating a copy on every property access. For a struct of three doubles this is cheap, but worth noting for consistency with lazy-evaluation patterns used for messages.
[Low] The Fetcher::queue() TS override shows metadata?: MessageBatchMetadata unconditionally, meaning the type signature is visible in tooling even for workers without the queue_batch_metadata flag. This is fine for a testing API, but worth confirming it's intentional.
[Low] The disabled-flag test service could use a compatibilityDate to be more explicit.

This review was generated by an AI assistant and may contain inaccuracies. Please verify all suggestions before applying.

src/workerd/api/queue.h

src/workerd/api/tests/queue-metadata-test.wd-test

src/workerd/api/queue.c++

src/workerd/io/worker-interface.capnp

src/workerd/api/http.h

ask-bonk · 2026-03-17T13:56:59Z

Review posted successfully. Here's a summary of the findings:

Key findings posted on PR #6339:

[Medium] Type choice for metrics fields — backlogCount and backlogBytes are conceptually unsigned integers but use double/Float64. This works for JS interop but is worth a design discussion.
[Low] Property accessor pattern — getMetadata() returns by value and uses JSG_READONLY_INSTANCE_PROPERTY (eager). This is fine for 3 doubles, just noted the difference from the lazy pattern used by messages.
[Low] Fetcher TS override not flag-gated — The metadata? parameter on Fetcher::queue() is always visible when service_binding_extra_handlers is on, regardless of the queue_batch_metadata flag. Reasonable for a testing API but worth confirming.
[Low] Test module naming — The disabled-test service uses name = "worker-disabled" for the embedded module, which is a bit unusual given it embeds the same file.
[Positive] Capnp default-to-zero comment — Good documentation of the forward-compatibility behavior when metadata is omitted.
[Low] Timestamp unit documentation — Flagged verifying that "ms since epoch" in the capnp comment matches the upstream broker's actual semantics.

No safety, memory, or correctness issues were found. The capnp ordinal (@173) is correct, the compat flag is properly $experimental, both test variants (enabled/disabled) are exercised, and the EW_QUEUE_ISOLATE_TYPES macro correctly includes the new types.

github run

codecov-commenter · 2026-03-26T15:47:02Z

Codecov Report

❌ Patch coverage is 14.28571% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.70%. Comparing base (c2e2dbc) to head (8676265).
⚠️ Report is 5 commits behind head on main.

Files with missing lines	Patch %	Lines
src/workerd/api/queue.c++	5.55%	17 Missing ⚠️
src/workerd/api/queue.h	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6339      +/-   ##
==========================================
- Coverage   70.70%   70.70%   -0.01%     
==========================================
  Files         427      427              
  Lines      117285   117313      +28     
  Branches    18905    18907       +2     
==========================================
+ Hits        82926    82944      +18     
- Misses      23103    23114      +11     
+ Partials    11256    11255       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

a-robinson · 2026-03-26T21:03:28Z

src/workerd/api/queue.h

+struct MessageBatchMetrics {
+  double backlogCount;
+  double backlogBytes;
+  double oldestMessageTimestamp;


If you buy my argument on #6246 (comment), then this should be a Date here too. As well as in any equivalent new data types introduced in #6354.

Although I'm not totally sure why we're using a different struct here than on the "metrics" endpoint in the first place. Are we worried that eventually we'll end up exposing different metrics from each place?

Agreed with the comment! All of these changes are behind the experimental flag right now. Would it be alright if I modified all of them to use Date in a follow-up PR after #6354 is in?

That was my thought process, although I imagined they would likely stay in sync, I didn't want to permanently tie all of the metrics types together.

That's totally fine, thanks!

Here is the follow-up PR: #6445

KennethRuan requested review from a team as code owners March 16, 2026 20:39

KennethRuan marked this pull request as draft March 16, 2026 21:02

KennethRuan marked this pull request as ready for review March 16, 2026 23:59

ask-bonk bot reviewed Mar 17, 2026

View reviewed changes

KennethRuan force-pushed the kruan/MQ-1202-add-metrics-metadata-to-queue-handler branch from 6ea1850 to c232a53 Compare March 18, 2026 07:41

KennethRuan mentioned this pull request Mar 20, 2026

MQ-1200 Add response body to queue send() and sendBatch() #6354

Merged

4 tasks

KennethRuan force-pushed the kruan/MQ-1202-add-metrics-metadata-to-queue-handler branch from c232a53 to 244cd5c Compare March 20, 2026 22:49

KennethRuan requested a review from a team as a code owner March 20, 2026 22:49

KennethRuan force-pushed the kruan/MQ-1202-add-metrics-metadata-to-queue-handler branch from 244cd5c to c395270 Compare March 20, 2026 22:57

jasnell approved these changes Mar 26, 2026

View reviewed changes

feat(MQ-1202): include metrics metadata in queue() handler message batch

8676265

KennethRuan force-pushed the kruan/MQ-1202-add-metrics-metadata-to-queue-handler branch from c395270 to 8676265 Compare March 26, 2026 18:21

jasnell merged commit a9fc5bb into cloudflare:main Mar 26, 2026
28 of 30 checks passed

a-robinson reviewed Mar 26, 2026

View reviewed changes

KennethRuan mentioned this pull request Mar 27, 2026

MQ-1267 Change queues oldestMessageTimestamp to use kj::Date #6445

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MQ-1202 Include metrics metadata in queue() handler message batch#6339

MQ-1202 Include metrics metadata in queue() handler message batch#6339
jasnell merged 1 commit intocloudflare:mainfrom
KennethRuan:kruan/MQ-1202-add-metrics-metadata-to-queue-handler

KennethRuan commented Mar 16, 2026 •

edited

Loading

Uh oh!

jasnell commented Mar 17, 2026

Uh oh!

ask-bonk bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ask-bonk bot commented Mar 17, 2026

Uh oh!

codecov-commenter commented Mar 26, 2026 •

edited

Loading

Uh oh!

Uh oh!

a-robinson Mar 26, 2026

Uh oh!

KennethRuan Mar 26, 2026

Uh oh!

a-robinson Mar 26, 2026

Uh oh!

KennethRuan Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

KennethRuan commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Testing

Uh oh!

jasnell commented Mar 17, 2026

Uh oh!

ask-bonk bot left a comment

Choose a reason for hiding this comment

Review summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ask-bonk bot commented Mar 17, 2026

Uh oh!

codecov-commenter commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

a-robinson Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

KennethRuan Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

a-robinson Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

KennethRuan Mar 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

KennethRuan commented Mar 16, 2026 •

edited

Loading

codecov-commenter commented Mar 26, 2026 •

edited

Loading