Skip to content

Refactor logging for efficiency, add fifo, context printing in text mode#4133

Closed
invisig0th wants to merge 68 commits intomasterfrom
visi-link-errinfo
Closed

Refactor logging for efficiency, add fifo, context printing in text mode#4133
invisig0th wants to merge 68 commits intomasterfrom
visi-link-errinfo

Conversation

@invisig0th
Copy link
Copy Markdown
Contributor

No description provided.

@codecov
Copy link
Copy Markdown

codecov bot commented Feb 14, 2025

Codecov Report

Attention: Patch coverage is 96.29630% with 21 lines in your changes missing coverage. Please review.

Project coverage is 97.29%. Comparing base (e6d8cef) to head (1432c5c).

Files with missing lines Patch % Lines
synapse/lib/logging.py 94.11% 9 Missing ⚠️
synapse/cortex.py 95.34% 4 Missing ⚠️
synapse/lib/agenda.py 95.31% 3 Missing ⚠️
synapse/lib/coro.py 81.25% 3 Missing ⚠️
synapse/tools/cmdr.py 0.00% 1 Missing ⚠️
synapse/tools/rstorm.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4133      +/-   ##
==========================================
- Coverage   97.35%   97.29%   -0.06%     
==========================================
  Files         261      261              
  Lines       59821    59954     +133     
==========================================
+ Hits        58237    58331      +94     
- Misses       1584     1623      +39     
Flag Coverage Δ
linux 97.29% <96.29%> (+0.01%) ⬆️
linux_replay ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@invisig0th invisig0th changed the title VERY WIP: Refactor logging for efficiency, add fifo, context printing in text mode, and additional link exception context WIP: Refactor logging for efficiency, add fifo, context printing in text mode Feb 18, 2025
@invisig0th invisig0th requested a review from Cisphyx March 7, 2025 15:10
MichaelSquires
MichaelSquires previously approved these changes Mar 7, 2025
@invisig0th invisig0th changed the title WIP: Refactor logging for efficiency, add fifo, context printing in text mode Refactor logging for efficiency, add fifo, context printing in text mode Mar 7, 2025
@invisig0th invisig0th requested a review from vEpiphyte March 7, 2025 18:16
Cisphyx
Cisphyx previously approved these changes Mar 7, 2025
Comment thread synapse/axon.py
@vEpiphyte vEpiphyte dismissed stale reviews from Cisphyx and MichaelSquires via 1432c5c March 11, 2025 18:46
Comment thread synapse/lib/httpapi.py Outdated
Comment thread synapse/lib/httpapi.py
self.clear()
self.set_status(500)
self.sendRestErr('StormRuntimeError', f'Extended HTTP API {iden} never set status code.')
self.sendRestErr('StormRuntimeError', 'Extended HTTP API never set status code.')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the iden being removed here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed like an unnecessary disclosure of "internals" to a caller who knows what API endpoint they hit. Happy to discuss. 👍

Comment thread synapse/lib/httpapi.py Outdated
Comment thread synapse/lib/cell.py
Comment thread synapse/lib/logging.py

logtodo = []
logbase = None
loglock = asyncio.Lock()
Copy link
Copy Markdown
Contributor

@vEpiphyte vEpiphyte Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These structures ( loglock / logevnt )will bind to the currently running loop when they are first locked / waited on; meaning we'll end up with isolation issues if we move to isolated ioloops for testing.

Comment thread synapse/lib/cell.py
s_logging.logbase = await s_base.Base.anit()
s_logging.logbase._fini_at_exit = True

s_logging.logbase.schedCoro(s_logging._feedLogTask())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The logbase feels like something we should be able to refcount & teardown on cell fini to prevent the feedtask from running beyond the lifetime of a cell; similar to how we handle other global tasks.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a bit of a "not necessarily via a single cell in one python interpreter" issue here that would need to be discussed, but def worth talking through 👍

Comment thread synapse/cortex.py
'''
await self.agenda.enable(iden)
await self.feedBeholder('cron:enable', {'iden': iden}, gates=[iden])
logger.info(f'Enabled cron job {iden}', extra=await self.getLogExtra(iden=iden, status='MODIFY'))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to retain these status keys.

Comment thread synapse/lib/aha.py Outdated
Comment thread synapse/lib/logging.py Outdated
@vEpiphyte vEpiphyte added this to the v3.0.0 milestone Apr 4, 2025
@vEpiphyte vEpiphyte self-assigned this Jun 27, 2025
@vEpiphyte
Copy link
Copy Markdown
Contributor

Closing this in favor of #4686

@vEpiphyte vEpiphyte closed this Apr 2, 2026
@vEpiphyte vEpiphyte deleted the visi-link-errinfo branch April 9, 2026 20:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants