Skip to content

fix gl2_input_message_size accuracy for inputs with field duplication#25533

Open
AntonEbel wants to merge 4 commits intoinputLicensefrom
inputLicenseDuplicateFields
Open

fix gl2_input_message_size accuracy for inputs with field duplication#25533
AntonEbel wants to merge 4 commits intoinputLicensefrom
inputLicenseDuplicateFields

Conversation

@AntonEbel
Copy link
Copy Markdown
Contributor

@AntonEbel AntonEbel commented Apr 1, 2026

Summary

  • Add utility for proportional size distribution across split messages
  • OTelGrpcInput, OTelHttpInput, CollectorIngestHttpInput : Distribute original request size proportionally by each log record's size
  • Kinesis/CloudWatch: Use decompressed payload size instead of compressed Kinesis record size
  • DecodingProcessor: Refactor to use shared distribution utility

Context

Inputs that split one inbound payload into multiple Graylog messages were overcounting gl2_input_message_size due to shared fields being duplicated into every RawMessage payload. For Daily Usage Raw license accounting, the total input size must equal the original payload size, distributed proportionally across the resulting messages.

/nocl
Refs Graylog2/graylog-plugin-enterprise#13577

@AntonEbel AntonEbel requested review from a team and kroepke April 1, 2026 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants