fix(perf-detection): handle missing description in MN+1 DB detector; downgrade noisy DetectorGroup log#119449
Draft
billyvg wants to merge 1 commit into
Draft
fix(perf-detection): handle missing description in MN+1 DB detector; downgrade noisy DetectorGroup log#119449billyvg wants to merge 1 commit into
billyvg wants to merge 1 commit into
Conversation
… detector Spans processed by the segment consumer may not always include a `description` key. Accessing `db_span["description"]` raised KeyError causing the entire performance problem detection to fail (SENTRY-5QVD, 210K+ events). Changed both callsites to use `.get("description", "")`. Also downgrade `logger.exception` to `logger.warning` for `DetectorGroup.DoesNotExist` in `_get_detector_for_group` — this is an expected condition for legacy groups that predate DetectorGroup associations, and the code already handles it gracefully by falling through to alternative detector lookups (SENTRY-5R4N, 24K events, 4K users). Using `logger.exception` was generating noisy error-level log events with full tracebacks for a known benign case. Fixes SENTRY-5QVD Ref SENTRY-5R4N
github-actions
Bot
added
the
Scope: Backend
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes two Sentry-reported production errors found during automated alert triage.
Fix 1 — KeyError: 'description' in MN+1 DB span detector (SENTRY-5QVD, high severity)
Issue:
SENTRY-5QVD— 210K+ events, ongoing since 2026-06-25.Root cause:
ContinuingMNPlusOne._maybe_performance_problemaccesseddb_span["description"]directly in two places. The segment consumer (spans.consumers.process_segments) processes spans from theprocess-segmentsKafka consumer, and some spans in this path do not include adescriptionkey (e.g. downsampled spans, spans with stripped fields). This causedKeyError: 'description', which was caught by the outerdetect_performance_problemshandler and logged as "Failed to detect performance problems" — silently dropping all N+1 detection for affected segments.Fix: Use
db_span.get("description", "")in both callsites (desc=argument andget_notification_attachment_bodycall) inmn_plus_one_db_span_detector.py:258,283.Fix 2 — DetectorGroup.DoesNotExist logged as exception (SENTRY-5R4N, medium severity)
Issue:
SENTRY-5R4N— 24K+ events, 4K+ users impacted, ongoing since 2026-07-01.Root cause:
_get_detector_for_groupinworkflow_engine/processors/detector.pyusedlogger.exception(...)inside theexcept DetectorGroup.DoesNotExisthandler.logger.exceptionlogs at ERROR level and includes the full traceback. The Seer root cause analysis confirms this is an expected condition for legacy groups that predate DetectorGroup associations — the code already handles the missing record gracefully by falling through to look up a detector by(project, type)or the issue stream detector.Fix: Change
logger.exception→logger.warningso the expected fallback path doesn't generate noisy ERROR-level events with full tracebacks.Evidence
Session
https://claude.ai/code/session_01AY6kMQotceHm8vtMtSkdBc
Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
Generated by Claude Code