Skip to content

refactor: simplify/reformat existing grafana alerts#142

Draft
amdove wants to merge 20 commits intomainfrom
alert-reformat
Draft

refactor: simplify/reformat existing grafana alerts#142
amdove wants to merge 20 commits intomainfrom
alert-reformat

Conversation

@amdove
Copy link
Contributor

@amdove amdove commented Feb 25, 2026

Description

  • Standardized alert format across all 15 Grafana alerts with consistent WHERE/DETAILS sections
  • Added tenant_name field to WorkloadConfig and Alloy external_labels for tenant identification in alerts
  • Alerts now use severity emojis (🔴 CRITICAL, 🟡 WARNING) and include runbook/dashboard links

Slack alerts should now look like this:
image

Test plan

  • Just test-python-pulumi passes (111 tests)
  • Deploy to staging and verify alert format in Grafana

No new tests added since tenant_name is a cosmetic field with simple fallback logic (tenant_name or compound_name), testing would only verify basic Python language features.

Category of change

  • Bug fix (non-breaking change which fixes an issue)
  • Version upgrade (upgrading the version of a service or product)
  • New feature (non-breaking change which adds functionality)
  • Build: a code change that affects the build system or external dependencies
  • Performance: a code change that improves performance
  • Refactor: a code change that neither fixes a bug nor adds a feature
  • Documentation: documentation changes
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

@amdove amdove changed the title Simplify existing grafana alerts refactor: simplify/reformat existing grafana alerts Feb 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant