Skip to content

Commit 4aa283c

Browse files
committed
Add: alertmanager-mattermost in k8s document
Signed-off-by: Ali Mehraji <a.mehraji75@gmail.com>
1 parent 1fe8dd5 commit 4aa283c

File tree

1 file changed

+59
-0
lines changed

1 file changed

+59
-0
lines changed
Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
title: Mattermost Receiver For Alertmanager in Prometheus-Stack in K8S
3+
published: true
4+
description: Add Mattermost Receiver for alertmanager in prometheus-stack in k8s
5+
tags: 'alertmanager,prometheus,monitoring,kubernetes'
6+
---
7+
8+
### Why
9+
10+
Since Prometheus-Stack Alertmanager doesn’t include Mattermost as a built-in receiver, we will integrate it using a webhook via creating a Python relay application that accepts Alertmanager webhook notifications, formats them into a Mattermost-compatible payload, and sends them to a Mattermost incoming webhook URL.
11+
12+
Then we will deploy this relay inside Kubernetes using a Deployment and expose it with a Service so Alertmanager can reliably reach it.
13+
14+
Finally, we will configure Alertmanager using an AlertmanagerConfig resource to define a new receiver and route, so alerts matching our rules (severity/namespace/etc.) get delivered to Mattermost through the relay.
15+
16+
### How
17+
18+
- Prometheus Alert Rules trigger alerts.
19+
- Alertmanager processes and routes the alerts.
20+
- Alertmanager sends webhook notifications to the Python Relay Service.
21+
- The relay formats the alert payload.
22+
- The relay forwards the formatted message to Mattermost Incoming Webhook.
23+
- Alerts appear in the designated Mattermost channel.
24+
25+
### Steps
26+
27+
- Create an [Incoming Webhook in Mattermost][mattermost-incoming-webhook]. This webhook URL will be used by the Python relay to deliver alert messages to the desired channel.
28+
> Ensure you securely store the generated webhook URL, as it will be injected into the relay application via environment variables or Kubernetes secrets.
29+
30+
- [Python Relay][python-relay-app] Application
31+
32+
> A lightweight webhook server that receives Alertmanager alerts and forwards them to Mattermost.
33+
34+
- Build the Docker Image [Dockerfile][dockerfile]
35+
36+
Once the relay application is complete, build the image and push it to a container registry accessible by your Kubernetes cluster.
37+
38+
- Deploy in Kubernetes: [K8S Manifests][k8s-manifests]
39+
- Configure Alertmanager with [AlertmanagerConfig][alertmanagerconfig-manifest]
40+
> Ensure having a [label][alertmanagerconfig-label] in AlertmanagerConfig to select via prometheus operator
41+
42+
- Add the Alertmanagerconfig to prometheus `values.yaml`
43+
44+
```yaml
45+
46+
alertmanager:
47+
alertmanagerSpec:
48+
alertmanagerConfigSelector:
49+
matchLabels:
50+
alertmanagerConfig: alertmanager-mattermost
51+
52+
```
53+
54+
[mattermost-incoming-webhook]:https://docs.mattermost.com/integrations-guide/incoming-webhooks.html
55+
[python-relay-app]: https://github.com/cod3mas0n/k8s-alertmanager-mattermost/blob/main/src/app.py
56+
[dockerfile]: https://github.com/cod3mas0n/k8s-alertmanager-mattermost/blob/main/src/Dockerfile
57+
[k8s-manifests]: https://github.com/cod3mas0n/k8s-alertmanager-mattermost/tree/main/kubernetes
58+
[alertmanagerconfig-manifest]: https://github.com/cod3mas0n/k8s-alertmanager-mattermost/blob/main/kubernetes/alertmanagerConfig.yaml
59+
[alertmanagerconfig-label]: https://github.com/cod3mas0n/k8s-alertmanager-mattermost/blob/main/kubernetes/alertmanagerConfig.yaml#L6-L7

0 commit comments

Comments
 (0)