Configuring Alert Manager To Send Slack Message
- Open the Alertmanager configuration:
$ sudo $EDITOR /etc/alertmanager/alertmanager.yml
- Set the default route’s
repeat_interval
to one minute and update the receiver to use our Slack endpoint:route: receiver: 'slack' group_by: ['alertname'] group_wait: 10s group_interval: 10s repeat_interval: 1m
- Create a secondary route that will send
severe: page
alerts to the Slack receiver; group by theteam
label:route: receiver: 'slack' group_by: ['alertname'] group_wait: 10s group_interval: 10s repeat_interval: 1m routes: - match: severity: page group_by: ['team'] receiver: 'slack'
- Add a tertiary route that sends all alerts for the
devops
team to Slack:route: receiver: 'slack' group_by: ['alertname'] repeat_interval: 1m routes: - match: severity: page group_by: ['team'] receiver: 'slack' routes: - match: team: devops receiver: 'slack'
- Update the receiver to use Slack:
receivers: - name: 'slack' slack_configs: - channel: "#prometheus" api_url: APIKEY text: "Overview: {{ .CommonAnnotations.overview }}"
- Update the
inhibit_rules
so that any alerts with the severity ofticket
for the DevOps team are suppressed when apage
-level alert is happening:inhibit_rules: - source_match: severity: 'page' target_match: severity: 'ticket' equal: ['team']
Save and exit.
- Restart Alertmanager:
$ sudo systemctl restart alertmanager
- View your Slack chat and wait to see the firing alert.