Enrichment Maps
Enrichment Maps add dynamic context to incoming alerts.
Mapping enrichment imports dynamic contextual information from external sources and adds that data to matching alerts. By leveraging existing relationship information, mapping enrichments quickly improve alert quality and reduce time to triage.
Enrichment maps match incoming alert data to lookup columns in the enrichment file. When there is a match, the map additional data to the alert from that row of the mapping table.
For example: A mapping table might have the lookup column as application name, with result columns for tool owner, and runbook URL. If an alert enters BigPanda with a matching application name, the alert is automatically enriched with the associated owner and runbook URL.
Lookup items are not added to alerts by enrichment maps, but should already exist in raw event data or alert tags. Result items are added only to alerts that match a value from a lookup item column.
Mapping Enrichment
Because of the interconnected nature of the data, mapping enrichment for a map is managed together. Only the run order of mapping enrichment items is managed within the Alert Tags screen. To learn more about updating tag or enrichment rule run order, see the Manage Alert Enrichment documentation.
New Tag Creation
When adding or updating an enrichment map, enrichment items will automatically be added to matching alert tags for each result item column. If a tag doesn’t already exist in the system, a new alert tag will be created for the enrichment item.
To change which tag is enriched with a result item column, change the name of the tag in the Mapping Schema editor.
Mapping Schema and File
Each enrichment map is made up of a map schema, and a mapping file.
The mapping schema defines rules about how the system should interpret the mapping file. Mapping schemas determine:
- The name and description of the enrichment map
- A condition for when mapping rules should apply
- Which columns contain the lookup values
- Result tag names
- Override settings for result values
The mapping file is the actual data that should populate alert item values. Mapping files are uploaded to BigPanda as a .csv. The map does not have to be fully populated on initial upload, but it must include headers for all columns. Map reuploads must match the existing schema, with the same column headers that were included in the initial upload.
Each mapping schema has only one map, and each map must be associated with a mapping schema.
Enriched Tag for Analytics
If your organization would like to accurately visualize enrichment rates in Unified Analytics, you must include 2 enrichment flag columns in the CSV:
- An
enrichedorenrichmentcolumn with true/false rows, where true means alerts that match on that row should be considered enriched.- A
<map name>column withenrichedorenrichmentin each row (whichever was the name of the previous column), to add the enriched quality to any alert matched on the map.
Map File Requirements
- The CSV file cannot be more than 200 MB
- The CSV must use commas as the delimiter
- The table contains at least two columns - at least one lookup column and one result column
- The table contains at least one header row
- Each row is unique; the table must not contain duplicate rows
- Each row has the same number of columns as the header row
- The field values do not exceed 32K in length
- CSV must use standard line feed characters
Size Limit
When uploading maps through the BigPanda UI, maps must be 200 MB or smaller. To upload a larger map, up to 512 MB, use the Alert Enrichment API.
Mapping Value Limitations
Result values within the map must meet the character requirements for alert tag values in BigPanda.
- Most tag values must be 512 characters or less
- The
descriptiontag can contain values up to 2048 characters - New line characters (\n or \r) or line breaks are not supported
Create New Mapping Schema
Create a New Map
Enrichment maps are managed in the Settings > Enrichment Maps page.
To create an enrichment map:
- Click New Map
- In the Create a new Map screen, give your map a short, descriptive Map Name. The map name will appear in the description for each mapping enrichment item when viewing or editing alert tag settings.
- (Optional) Click Add description to add a note with context for the map contents.
- Add a Condition to define which alerts should be enriched with mapping data. The condition must define which source_systems (inbound integrations) the map should apply to, and can include other BPQL-enabled alert fields.
- Click Select CSV File to open a file selection tool. Select your map .csv file.
- Once the map is uploaded, adjust tag configuration as needed.
- (Optional) Select Create as inactive to save the map without beginning alert enrichment from the map rules.
- Click Create and upload to save the map.
Map uploads can take several minutes. During upload, a map may have 2 statuses:
- Pending - the map is scheduled to upload, but another map is ahead in the queue
- Uploading - the map is actively uploading and will be available soon
Adjust Tags Before Saving
Any changes to tag configuration must be made before the map is saved. After saving, only the Map name, Description, and Map file can be changed.
File Name
The file name is not visible in BigPanda after upload. Use the optional description to add information on which file populated the mapping schema.
Recent Tags Only
Dropdowns only include alert tags that have been updated or included in an alert in the last 90 days. To use an older tag, simply type the tag name.
Adjust the Mapping Schema
BigPanda will generate a mapping schema based on the column headers in the uploaded .csv
Each column in a mapping table pairs to a BigPanda alert tag. The tag name will be defined by the column header. If a tag with that name exists in BigPanda, the map will add an automatic enrichment item for that tag. If that tag doesn’t yet exist, a new tag will be created for the mapping enrichment item.
The Preview pane will show the first 50 lines of the map, making it easy to visualize what information would be added to alerts based on each lookup column.
BigPanda will automatically define lookup and result items based on the column headers within the map. By default the first column creates a lookup item and all other columns create result items. You can choose different columns to act as lookup items. A mapping schema must have at least one lookup column.
Multiple Lookup Columns
Mapping schemas can have up to 3 lookup columns. However, lookup columns are additive. Alerts must match on the combined value of all lookup columns. If either column fails to match alert data, the alert will not be enriched with result items.
To enrich using one lookup item OR another, use the same map file to create a new mapping enrichment, and select a different lookup column than the first.
In the mapping schema, you can change whether a column is a lookup or result item, adjust the tag name, and set the item to override previous alert tag values. Hover over a tag item to see additional configuration options.
Schema Column Options
| Option | Description |
|---|---|
| Change a column from result to lookup | All lookup columns must match on an alert for the map to add result values. Toggle to Lookup Tag |
| Change a column from lookup to result | Result columns will be automatically added as automatic enrichment items for any alert tags with the same name. If no tag exists, a new one will be made. Toggle to Result Tag |
| Rename a tag | Result columns enrich the alert tag that matches the column header name. Rename a result column to enrich a different alert tag, or to create a new alert tag. Rename a lookup column to match raw event data or alert tags earlier in the run order. a. Select the name of the column b. Enter the new tag name. The tag name must meet alert tag naming requirements. If a result tag name matches an existing tag, the mapping enrichment will be added to the bottom of the list of enrichment items for that tag. |
| Override existing values | The map value will replace any previous enrichment for this tag. If multiple maps have override selected for a single field, the enrichment item that runs last will be the tag value used. H Select the Override checkbox. |
Mapping items follow the enrichment order for the tags that each column is added to. To adjust run order, adjust the related tag and enrichment item order in Alert Enrichment.
Renaming Lookup Tags
The lookup name should match the tag name that appears in incoming events or earlier alert tags. The lookup tag is not added to alerts and should only be renamed to match incoming event data.
Tag Naming Requirements
By default, tag names will be automatically created from the mapping file headers. Tag names must meet the following requirements:
- Maximum length of 64 characters.
- Start with a letter from a to z.
- Contain only lowercase letters (a-z), numbers (0-9), and some special characters, including underscores ( _ ) and hyphens ( - ).
- Cannot contain spaces.
Some names are reserved in BigPanda, and may not be used as a tag name, or may cause issues in the system if used as a tag name. See the list of limitations in the Tag Naming documentation.
You are able to rename tag names in BigPanda to meet requirements or to make mapping values clearer.
Renaming Query Tags
The query tag name should match the tag name that will appear in incoming events or earlier alert tags. The query tag is not added to alerts and should only be renamed to match incoming event data.
API Options
Enrichment maps can also be created and managed with the Alert Enrichment API. Enrichment maps created in the API will appear in the UI after a short delay.
To use the API to update a map created in the UI, you’ll need the map ID. When you’ve selected a map schema from the list, the map ID is the last 36 characters of the URL. You can also see the map ID in the details pane after selecting a map from the list.
Manage Enrichment Maps
As systems and processes change, enrichment maps may need to be updated.
Maps can be renamed or toggled active or inactive, and you are able to edit the description or condition.
Map files can be replaced with new map files. The new file must have the same headers as the previous file.
Mapping schemas can be duplicated, activated, inactivated, or deleted.
Enrichment maps are managed at Settings > Enrichment Maps.
Manage a Mapping Schema
Mapping schemas can be edited, activated, deactivated, or deleted.
| Option | Description |
|---|---|
| Edit | Edit an enrichment map to upload a new mapping file. The file must have the same headers as the original file. a. Click the Pencil icon b. In the Enrichment Map editor, click Upload New Map to add a new file. c. Click Update map to apply the changes If a map is saved as Active, new matched alerts will be enriched with the result tag values. Existing alert values will not be updated. |
| Activate or Deactivate | Using the Active dropdown, select Active or Deactivate. Inactivating an enrichment map stops BigPanda from enriching alerts with all related mapping enrichments. To instead inactivate only the mapping enrichments for specific result tags, inactivate the enrichment rule in the Alert Tag editor. |
| Delete | Delete an enrichment map when you are sure that it no longer applies to your infrastructure or process. A deleted enrichment map cannot be recovered. Deleting an enrichment map will also delete all enrichment items tied to the map. a. Click the Trash icon. b. Click Delete Map to confirm, or Cancel to return to the previous page. Any enrichments added to existing alerts will remain associated with that alert. New alerts will not be matched to the enrichment map enrichments. |
Mapping Item Reordering
Any enrichment order changes will need to be re-adjusted when updating mapping enrichment.
Next Steps
Start Managing Alert Enrichment
Learn more about Navigating the Settings Menu
Dig into the Alert Enrichment API
Updated 5 months ago