Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Extract fields from AlertManager labels into top level context #704

Open
bilbof opened this issue Apr 12, 2022 · 1 comment

Comments

@bilbof
Copy link
Contributor

bilbof commented Apr 12, 2022

Hey folks, I'm liking Iris so far. I'd like to check receptiveness to this change - I'm happy to put up a PR for this, as it would help us.

Problem

The AlertManager integration doesn't currently support the "override incident title" and "plan dynamic targets" features.

AlertManager sends a payload in this (slightly redacted) format:

{
  "groupKey": "Demo system down",
  "status": "firing",
  "groupLabels": {"iris_plan": "page-engineer", "severity":"critical", "service":"demo-system"},
  "commonLabels": {"iris_plan": "page-engineer", "severity":"critical", "service":"demo-system"},
  "commonAnnotations": {"team":"demo-team", "runbook_url": "https://example.org", "description": "The service is down!"}
}

Since the groupLabels.iris_plan is set to an existing plan (page-engineer) the plan will run and the context (the above payload) will be available in the incident summary and context templates and the message templates.

Unfortunately, if we want to react to all incidents in the same way (i.e. page an on-call primary and, if they don't answer, the secondary) but want to pages to be received by different teams, we must create a plan (or potentially plans) for each team, and set the iris_plan label on each alert to be team specific (i.e. iris_plan: "page-engineer-demo-team". Ideally we'd be able to use the existing dynamic_targets feature on plans so that we only need to create one plan which could be reused by all teams.

Also, each incident has the title "alertmanager" (the name of the application). Though we can set an <h1>{{groupLabels.alertname}}</h1> in the context templates, it would be nice if this feature was supported for the AlertManager integration.

Feature request

In the AlertManager integration it'd be great if we could pull up the alertname field from groupLabels if it exists, and allow users to set dynamic targets in alert labels.

labels:
    alertname: "Demo service is down"
    iris_plan: page-engineer
    iris_targets: '[{"role": "oncall-primary","target": "demo-team"},{"role": "oncall-secondary","target": "demo-team"}]'
@diegocepedaw
Copy link
Collaborator

@bilbof I don't think we have any issues with these feature enhancements. We basically don't leverage the alertmanager integration at all within linkedin (except for a couple of very niche usecases) which is why it's not something we've put a lot of resources towards. If you want to open a PR to implement the capabilities you mention in this issue I would be more than happy to review it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants