Skip to content

Conversation

@sjchmiela
Copy link
Contributor

@sjchmiela sjchmiela commented Feb 12, 2026

Why

For the new logs experience we need to identify every log with a unique ID.

Also, in the new logs experience we're going to be handling logs in plaintext, uncompressed, so size starts to matter.

How

Decorating every log with a unique ID is surprisingly difficult with bunyan. It only runs serializers on the object passed in, I tried adding a log_id serializer serializers: { log_id: () => uuidv7() } and a non-empty value to build logger, but it was getting serialized in the moment of logger creation.

Instead, I'm adding log_id in the Transform that also masks secrets. The overall architecture looks like this now:

Bez tytułu-2026-02-04-1436-3

Also removes service: 'worker' and hostname from logs. service was only being used here which I don't think is a thing right now.

Test Plan

Added unit test that ensures Transform adds logId.

Zrzut ekranu 2026-02-12 o 16 02 32

@github-actions
Copy link

Subscribed to pull request

File Patterns Mentions
**/* @douglowder

Generated by CodeMention

@sjchmiela sjchmiela added the no changelog PR that doesn't require a changelog entry label Feb 12, 2026
@codecov
Copy link

codecov bot commented Feb 12, 2026

Codecov Report

❌ Patch coverage is 83.87097% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 52.43%. Comparing base (13a6356) to head (598691b).

Files with missing lines Patch % Lines
packages/worker/src/logger.ts 83.88% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3397      +/-   ##
==========================================
+ Coverage   52.30%   52.43%   +0.14%     
==========================================
  Files         804      803       -1     
  Lines       33421    33416       -5     
  Branches     6972     6967       -5     
==========================================
+ Hits        17478    17519      +41     
- Misses      14555    15815    +1260     
+ Partials     1388       82    -1306     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.


await waitForStreamFlush();

expect(outputStream.readableLength).toBe(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to add a test with a filtered secret

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it('obfuscates secrets in logs', async () => {
const { logger, outputStream } = await createBuildLoggerWithSecretsFilter([
{ name: 'TEST_SECRET', value: 'secret', type: EnvironmentSecretType.STRING },
{
name: 'ANOTHER_SECRET_BASE64',
value: 'YW5vdGhlclNlY3JldA==',
type: EnvironmentSecretType.STRING,
},
]);
const logs: any[] = [];
const writable = new Writable({
objectMode: true,
write(chunk: any, _encoding: BufferEncoding, callback: TransformCallback) {
logs.push(chunk);
callback(null, chunk);
},
});
outputStream.pipe(writable);
logger.info('this is a secret');
logger.info(`another secret in base64 is ${Buffer.from('anotherSecret').toString('base64')}`);
await waitForStreamFlush();
expect(logs.length).toBe(2);
expect(logs[0].msg).toBe('this is a ******');
expect(logs[1].msg).toBe('another ****** in base64 is ********************');
});

@github-actions
Copy link

⏩ The changelog entry check has been skipped since the "no changelog" label is present.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no changelog PR that doesn't require a changelog entry

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants