Mapping Enrichment Items
Mapping enrichment allows users to upload a CSV to map tag values to an enrichment table. Incoming alert data will be compared to the values in query_tag columns. If a match is found, the related tag will be updated with a value from the result_tag column.
Mapping enrichment items are added automatically to existing tags upon the upload of a mapping enrichment table with matching result_tag column names. If a map contains a result_tag column name that does not match an existing tag, a new tag is automatically created for the enrichment item.
result_tag column
If a result_tag column should match to a BigPanda tag with a name different from the column title, use the tag_name attribute. This will map the column to the BigPanda tag. No new tag will be created for the result_tag column title.
When the values of a mapping table should be updated, a new CSV file can be uploaded. All future incoming alerts will be mapped to the updated values.
Mapping enrichment items are created using two separate elements:
Mapping Enrichment Schema: The configuration of the Mapping Table to be uploaded. Defines properties such as: active status, source tag names (
query_tag), BigPanda tag names (result_tag), and override settings. Individual tag values do not need to be configured, as values will be extracted from the uploaded table.Mapping Enrichment Table: The dynamic table of values to populate into BigPanda based on the predefined
query_tagandresult_tagrules. This defines the actual data that will be used when enriching alerts.
Naming limitations
Alert tag names and values must meet character and size limits.
Some words are already used for tagging and backend functions in BigPanda. These words may have limited functionality within BigPanda when used as tag names.
When creating new alert or incident tags, we recommend users use an alternate name (i.e. "short_description") for the tag to bring that data into the BigPanda system.
To see the full list of naming limitations, refer to Tag Naming.
Enrichment Map Parameters
Attribute | Description | Type |
|---|---|---|
total_count | Total number of results. Returned with a GET call only. | Integer |
items | The mapping data, in a JSON body | Array of strings |
Mapping enrichment tables
Mapping Enrichment Tables are initially uploaded as CSV files. When managing individual map rows or retrieving mapping enrichment tables, the results will be returned as a JSON object.
Mapping File Guidelines
When uploading a CSV map to create a new table for a mapping enrichment or to completely replace an existing table, send the entire table as comma-separated values (CSV) in the body of the request or in a CSV file.
Upload options
The mapping table can be copied into the body of the call, or uploaded as a separate CSV file. To upload a separate CSV, replace --data with --data-binary in the call parameters.
Empty new line required
The payload must end with an empty new line. Make sure there are no spaces before the closing quotation marks on the last line.
Table in separate CSV file
The CSV file must use standard line feed characters for line endings and must end with an empty new line. Some programs use different line ending formats. If you receive the following error: Stream finished but there was a truncated final frame in the buffer, you may need to convert the line endings or add an empty new line to your file.
Table requirements
The data table must meet these requirements:
The structure matches the mapping schema definition. For example, the column names must match the titles in the schema definition. Similarly, the table must include all of the columns in the definition
The table contains at least two rows—the title row and at least one row of enrichment values
Each row is unique; the table must not contain duplicate rows
The field values do not exceed 32K in length
Not contain any line breaks: New line characters (
\nor\r) or line breaks are not supported.The CSV file cannot be larger than 512 MB.
All map values must meet tag naming and size limits.
Warning
If your organization would like to accurately visualize enrichment rates in Unified Analytics, you must include 2 enrichment flag columns in the CSV:
An
enrichedorenrichmentcolumn with true/false rows, where true means alerts that match on that row should be considered enriched.A
<map name>column with `enriched` or `enrichment` in each row (whichever was the name of the previous column), to add the enriched quality to any alert matched on the map.
Asynchronous upload
Because it is a potentially long-running action, the table upload is performed asynchronously. Therefore, the immediate response indicates only whether the request was properly formatted and, if it was, provides a URL for checking the status of the upload.
The entire table upload must complete successfully for the changes to take effect; the API does not support partial success.
A typical asynchronous upload negotiation consists of these steps:
Upload the table: A Job object is created and a
locationURL for checking the status is returned.Use the URL to periodically check the job status until it is set to done or failed.
If the job was not successful, you can retry the request. If necessary, debug any connectivity issues or data formatting issues that may have contributed to the failed upload. For example, ensure the CSV file follows the enrichment schema definition.
Sample Object
{
"items":
[
{
"application": "sale24s",
"region": "us-east",
"team": "east-ops"
},
{
"application": "billing23",
"region": "us-west",
"team": "west-ops"
}
]
}
Parameters
The Mapping Enrichment object schema includes the following attributes:
Attribute | Description | Type |
|---|---|---|
id | System-generated unique identifier for the enrichment map schema. | String |
type | Enrichment function type used to create enrichment items. Must be “mapping” for mapping enrichment endpoints. | String |
active | Whether the mapping rules should be applied to incoming alerts. | Boolean |
When | BPQL object query to filter incoming alerts. Only matched alerts will receive a value based on the automatic enrichment item. | String |
Version | Internal version number of the current schema. This number is incremented automatically each time the schema is updated. | Integer |
config | Rules for mapping tag values to event data. | Object |
created_by | The unique ID associated with the user who created the mapping enrichment rule. | String |
updated_by | The unique ID associated with the user who last updated the mapping enrichment rule. | String |
note (optional) | A description of the mapping rule purpose or functionality. | String |
Mapping Enrichment Configuration Parameters
The mapping enrichment configuration schema sets mapping rules for the related mapping tables and includes the following attributes:
Attribute | Description | Type |
|---|---|---|
name | A system name for the mapping table. This name will appear in the UI. | String |
fields | Settings for each field included in the mapping table. Attributes: title - column name on the mapping table type - whether query_tag or result_tag tag_name - (Optional) the name of the tag in BigPanda override_existing - (Optional) if a value has been populated for this tag by another map or enrichment rule, whether this value should replace that value optional - (optional) whether or not the tag must be applied to all incoming alerts. Default is false. | Array of Objects |
id_generation_function | Method for id creation in system. System field. | String |
case_insensitive | Whether the field names for this configuration take character case into consideration. | Boolean |
override_existing
If override_existing is set to false, mapping values will not be added to any alert where the field is already populated, even if the field is populated with a “null.”
Update limitations for mapping enrichments
The only fields that can be modified in the PATCH request are the active, when, and note parameters. The mapping schema config cannot be edited.
If any other fields outside of these parameter are sent in the request, they will be ignored.
Sample Objects
{
"type":"mapping",
"active": true,
"when": {},
"config":{
"name": "mapping_rule",
"fields":[
{
"title":"application",
"type":"query_tag"
},
{
"title":"yes",
"type":"result_tag",
"override_existing":false
},
{
"title":"very_cool",
"type":"result_tag",
"tag_name":"very_cool",
"override_existing":false
}
],
"case_insensitive": false
},
"note": ""
}