Alert Enrichment V2.1

Enrichment is the process of adding contextual information to alerts in BigPanda.

Use the Alert Enrichment API to define alert tags that help users understand incidents more quickly and enrich BigPanda functionality, including correlation patterns and maintenance plans.

Read more about how alerts are contextualized and the potential correlation and automation possibilities of alert enrichment in the Alert Intelligence documentation.

📘

Mapping Enrichment UI

Mapping Enrichments can now be managed right in the BigPanda UI. Read more in the Enrichment Maps documentation.

🚧

Updating Existing Processes

If you have previously configured API management of Alert Enrichment using the V1 or V2.0 API, you can update those processes to this new API by following the steps in the Migrating to Alert Enrichment V2.1 documentation.

No additional steps are necessary to configure a new API call using the V2.1 API.

Relevant Permissions

Roles with the following permissions can access the Alert Enrichment API:

PermissionDescription
Alert EnrichmentView and use the Alert Enrichments UI and API.
Mapping Enrichment (API Only)Use the Mapping Enrichment API.

Permission access levels can be adjusted by selecting either View or Full Access. To learn more about how BigPanda's permissions work, see the Roles Management guide.

🚧

Rate Limitations

To maintain quality of service, the Alert Enrichment API is limited to 5 requests per second.
Additional requests will return a 429 response code and the request will need to be retried.

🚧

Tag Limitations

To maintain quality of service, BigPanda limits the number of alert tags and enrichment items available. Each organization can have:

  • 1000 alert tags
  • 500 enrichment items per alert tag
  • 20,000 alert enrichment items total
  • 200 mapping enrichment results tags

If more alert tags or enrichment items are needed, we recommend exploring normalization options to help streamline your alert data and improve incident quality.

Alert Tag Object

BigPanda normalizes alert data from integrated monitoring systems into standard key-value pairs, called tags. Alert tags are the fundamental data model for your alerts and drive alert deduplication, correlation, and enrichment.

In addition to the core properties such as name, active status, and description, all alert tags also include at least one enrichment item. Enrichment items set rules and conditions to populate values for the tag based on event-payload fields.

There are 3 types of enrichment items:

  • Composition
  • Extraction
  • Mapping

Enrichment items are tied to their tag. If all enrichment items are deleted from a tag, the tag will also be removed from the system. Deleting a tag will remove all of the tag’s enrichment items as well.

❗️

Deleting Mapping Enrichment Items

To delete a tag with mapping enrichment items, you must first delete the associated mapping enrichment schema.

Mapping enrichment schemas apply to multiple tags, and deleting a schema will remove the mapping enrichment from all associated tags. If any associated tags should remain in BigPanda, we recommend first creating the new schema, then deleting the old one.

Run order is an important part of successful alert enrichment, allowing you to create tags that leverage the values of other tags. Alert tags and the enrichment items within each tag can be reordered so that tag values are populated in the needed order.

Enrichment item order is based on the order it appears in the API call. When merging new or changing enrichment items, new enrichment items will be added to the top of the order.

Composition Enrichment Items

Composition enrichment items are created by defining a value template.

Composition Template: A simple formula to populate tag value using strings and other tag values.

Use any tag value as a variable, in the format ${<tag_name>}. For example:

mywiki.com/${host}/${check}

Special formatting is required for tag values that contain encoded characters for URLs. If the tag value contains encoded values for the characters %, +, or a space, use the format ${exact(<tagname>)} to protect the values from being re-encoded.

Extraction Enrichment Items

Extraction enrichment items are created by defining a regular expression (regex formula).

Extraction Regex: The Regular expression to extract the new tag value from the source tag value.

  • Use a caret ^ to indicate starts with and a dollar sign $ to indicate ends with.
  • Use parentheses to surround the capture group for the extraction. BigPanda uses the contents of the first capture group to create the value of the alert enrichment tag.
  • Use a non-capturing group to ignore part of the contents.

If a tag value doesn't match the pattern, the alert is not enriched with the alert enrichment item.

Mapping Enrichment Items

Mapping enrichment allows users to upload a CSV to map tag values to an enrichment table. Incoming alert data will be compared to the values in query_tag columns. If a match is found, the related tag will be updated with a value from the result_tag column.

Mapping enrichment items are added automatically to existing tags upon the upload of a mapping enrichment table with matching result_tag column names. If a map contains a result_tag column name that does not match an existing tag, a new tag is automatically created for the enrichment item.

👍

If a result_tag column should match to a BigPanda tag with a name different from the column title, use the tag_name attribute. This will map the column to the BigPanda tag. No new tag will be created for the result_tag column title.

When the values of a mapping table should be updated, a new CSV file can be uploaded. All future incoming alerts will be mapped to the updated values.

Mapping enrichment items are created using two separate elements:

  • Mapping Enrichment Schema: The configuration of the Mapping Table to be uploaded. Defines properties such as: active status, source tag names (query_tag), BigPanda tag names (result_tag), and override settings. Individual tag values do not need to be configured, as values will be extracted from the uploaded table.
  • Mapping Enrichment Table: The dynamic table of values to populate into BigPanda based on the predefined query_tag and result_tag rules. This defines the actual data that will be used when enriching alerts.

📘

Mapping Enrichment UI

Mapping Enrichments can now be managed right in the BigPanda UI. Read more in the Enrichment Maps documentation.

JSON Format

The Alert Enrichment API accepts only standard double-quoted JSON formatting. Python JSON uses single quotes and will not be accepted by the API.

If using Python, please sanitize your JSON to adhere to standard double-quoted JSON formatting.

Example JSON formatted payload:

curl --request POST \
  --url https://api.bigpanda.io/resources/v2.1/alert-enrichments \
  --header "Authorization: Bearer ${your_api_key}" \
  --header 'Content-Type: application/json' \
  --data '{
	"field": "states",
	"secondary_field": true,
	"object": {
		"subobject": [
			{
				"field": "string",
				"secondary_field": "also string"
			]
	},
	"another_field": "url.com"
}'

Available Objects and Actions

Objects

ObjectDescriptionSupported MethodsAPI Endpoint
Alert Tag ObjectDefines the schema for an alert tag.POST, GET, PATCH, DELETE/v2.1/enrichments-config/tags/

/v2.1/enrichments-config/tags/{TAG_NAME}
Composition or Extraction Enrichment ItemDefines the schema for a composition or extraction enrichment item.POST, GET, PATCH, DELETE/v2.1/enrichments-config/tags/{TAG_NAME}/enrichments
Mapping Enrichment SchemaDefines the schema for a mapping enrichment item.POST, GET, PATCH, DELETE/v2.1/mapping-enrichment/

/v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}
Mapping Enrichment TableUploaded enrichment map files.GET, POST, PATCH/v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}/map
Job StatusThe schema for the return on a job status call.GET/v2.1/alert-enrichments-jobs/{JOB_ID}
Alert Tag OrderThe schema for an alert tag order call.PUT/v2.1/enrichments-config
Enrichment Item OrderThe schema for an enrichment item order call.PUT/v2.1/enrichments-config/{tag_name}/order

Alert Tag Routes

ActionEndpointDescription
Create Alert TagPOST /v2.1/enrichments-config/tags/Creates a new tag
Retrieve Alert TagGET /v2.1/enrichments-config/tags/{tag_name}Retrieves a specific alert tag.
Retrieve All Alert TagsGET /v2.1/enrichments-config/tags/Lists all alert tags configured in BigPanda.
Update Alert Tag DataPATCH /v2.1/enrichments-config/tags/{tag_name}Updates tag fields for name, active state, or description.
Delete Alert TagDELETE /v2.1/enrichments-config/tags/{tag_name}Deletes an alert tag.

Enrichment Item Routes (Composition and Extraction Only)

ActionDescription
Composition or Extraction Enrichment ItemDefines the schema for a composition or extraction enrichment item
Create Alert Enrichment ItemCreates a new composition or extraction enrichment item for the tag.
Update Alert Enrichment ItemUpdates a composition or extraction enrichment item for the tag.
Delete Alert Enrichment ItemDeletes a composition or extraction enrichment item for the tag.
If all enrichment items for a tag are deleted, the tag will also be removed from the system

Mapping Enrichment Routes

ActionEndpointDescription
Create Mapping Enrichment SchemaPOST /v2.1/mapping-enrichment/Creates a mapping enrichment schema
Retrieve Mapping Enrichment SchemaGET /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}Retrieves a specific mapping enrichment schema.
Retrieve All Mapping Enrichment SchemaGET /v2.1/mapping-enrichment/Lists all mapping enrichment schemas in BigPanda.
Update Mapping Enrichment SchemaPATCH /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}Updates a mapping enrichment schema.
Delete Mapping Enrichment SchemaDELETE /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}Deletes a mapping enrichment schema.

Mapping Enrichment Table Routes

ActionEndpointDescription
Create Mapping Enrichment Table using CSVPOST /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}/mapUploads a mapping enrichment table.
Retrieve Mapping Enrichment TableGET /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}/mapRetrieve a specific enrichment map.
Create Mapping Enrichment RowPATCH /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}/mapUpdates an enrichment map to add a new row.
Update Mapping Enrichment RowPATCH /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}/mapUpdates an enrichment map to update a specific map row.
Delete Mapping Enrichment RowPATCH /v2.1/mapping-enrichment/{MAPPING_ENRICHMENT_ID}/mapUpdates an enrichment map to delete a specific map row.

Job Routes

ActionEndpointDescription
Retrieve Mapping Table Job StatusGET /v2.1/alert-enrichments-jobs/{job_id}Retrieves the status of a previous enrichment change.

Ordering Routes (Alert Tags, and Alert Tag Enrichment Items)

ActionEndpointDescription
Update Alert Tag OrderPUT /v2.1/enrichments-configUpdates tag run order.
Update Enrichment Item Order for an Alert TagPUT /v2.1/enrichments-config/{TAG_NAME}/orderUpdates enrichment item order for a composition or extraction tag.