The Alerts API allows you to easily integrate a monitoring system with BigPanda. Monitoring systems generally send out events when problems are detected and when problems have been resolved (fixed).

The API receives event payloads from a monitoring system. Incoming events sent via the REST API are processed according to the BigPanda alert correlation logic. Depending on the results of the correlation, the system then creates a new incident or updates an existing incident based on Event Aggregation rules.

🚧

Authentication

All BigPanda APIs require Bearer Token Authorization in the call headers.

This API uses the Org Token type of Authorization token.

If the tool you are integrating with does not support HTTP headers, you can use the access_token URL parameter for authentication. For example:
<https://api.bigpanda.io/data/v2/alerts?access_token=><YOUR TOKEN>

❗

️ Sending Multiple Alerts

BigPanda uses the timestamp to determine the latest status of an incident. If it is not included, BigPanda uses the time when the alert is received. To ensure that BigPanda accurately reflects the current status, when sending multiple alerts, you must include the timestamp for each alert or sort the alerts array by when the events occurred, in ascending order.

❗

️ Sending Duplicate Events

When an event sent via the Alerts API is an exact duplicate of a previous event, BigPanda returns an HTTP response code of 204 No Content.

👍

Resolving Alerts

To maintain only the most relevant information in the incident feed, we recommend configuring monitoring systems to send resolving events to BigPanda when an alert is no longer active.

Available Methods

Integrating Monitoring Systems

You can use the Alerts API to build a custom integration between BigPanda and your monitoring system. If the monitoring system supports custom HTTP callbacks, the easiest way to integrate it is by using a webhook. If it doesn't, you can write a custom script.

Using a Webhook to Integrate a Monitoring System

If the monitoring system supports custom HTTP callbacks, you can follow this general process to integrate it with BigPanda:

1. In BigPanda, create an App Key. Each integration must have an App Key in BigPanda to identify it as a unique source.
2. In your monitoring system, configure the webhook to send alerts to BigPanda.
3. Test the integration by sending a test alert.

Using a Custom Script to Integrate a Monitoring System

If the monitoring system does not support webhooks, you can write a custom script. Follow this general process to integrate it with BigPanda:

1. In BigPanda, create an App Key. Each integration must have an App Key in BigPanda to identify it as a unique source.
2. Configure your monitoring system to call your custom script.
3. Test the integration by sending a test alert.

To learn more about integrating tools using the BigPanda Rest API, see the instructions in BigPanda.

Best Practices

Follow these guidelines when building custom integrations with the BigPanda Alerts API.

When to Send Events

• Send an event every time the status of an alert changes. For example, if a new Critical alert is opened or a Critical alert becomes a Warning. The status of an alert typically changes when a metric goes above or below a certain threshold.
• Send an updated event when you need to change tag values for an alert. In some rare cases, you may want to send an event for an alert with the same status but a different value for one or more tags. For example, assume you have an alert with status of Warning and description of problem1. You can send a new Warning event with a description of problem2, and the alert status remains the same but the description is updated.
• Send an OK resolution event when an alert is no longer active. BigPanda does not resolve alerts automatically. Alerts that have not been resolved remain open, and the corresponding incident also remains open and continues to appear in the incident feed.

❗

️

Some monitoring tools don't trigger resolution events. In this case, you must manually resolve incidents to remove them from the incident feed.

What Tags to Send

The attributes included in the JSON payload become tags in BigPanda.

You can add any number of custom JSON attributes with a string, integer, or array value to the payload. Common fields include host, host, service, application, device, check, sensor, cluster, node, data center, region, and description.

When configuring the data to send through the Alerts API, send attributes that:

• Drive deduplication, correlation, and UI titling. Event attributes enable BigPanda to turn noisy events into high-quality alerts and actionable incidents.
• Add context about the event, including where or when the alert triggered. Additional information helps your team understand the full situation while investigating an issue.
• Enable automation and categorization. Tags such as business element or team help you build environments, autoshare rules, analytics, and workflows.
• Set next steps for triage and remediation. Including links to runbook wikis or investigation tools helps your team take action quicker. If a tag value starts with http, BigPanda automatically adds a link button.

❗

️

Nested objects are not supported when sending events to the Rest API

🚧

Incident_identifier

The incident_identifier is a unique ID used in BigPanda to recognize if two events are related to each other. By default BigPanda will use a combination of the primary and secondary properties to create a value for incident_identifier. Adding an incident_identifier into the payload will override the default value, and allows you to configure specific deduplication rules.

Read more about how incident_identifiers work in the Alert Identification documentation.

How Many Events to Send (Load Limitations)

• If an integration generates more than a few hundred events per hour, review the guidelines for when to send events and ensure that the integration is properly configured. Even with hundreds of thousands of monitoring metrics, status changes do not happen very often. If an integration exceeds this guideline, it may be sending duplicate or erroneous events to BigPanda.
• Under maximum load, the API can support up to a few hundred events per minute.

❗

️Payload Size

Alert payloads must be 6MB or smaller. Larger payloads will fail to process with BigPanda. We recommend reviewing your configurations to ensure that only actionable, useful information is being sent to BigPanda.