Splunk

Splunk allows you to search, monitor, and analyze machine data that is generated by your infrastructure. Install this integration to correlate alerts from Splunk into high-level incidents in BigPanda, and see insights from Splunk alongside the problems detected by other tools in your monitoring stack.

📘

Looking for our old Splunk docs?

This page has been updated with documentation for our new Splunk integration. If you are looking for docs for the previous Splunk integration please find them here.

Supported Versions:
Splunk Enterprise, Splunk Cloud

Type:
Native App

Key Features


  • Easily send Splunk alerts to BigPanda using the native BigPanda action in Splunk
  • Keep your team focused on what matters with auto-resolution of Splunk alerts in BigPanda
  • Simple & efficient management of all alerts going to BigPanda via custom alert management dashboard and search commands
  • Customizable alert properties
  • Proxy support for on-premises Splunk deployments

How It Works


BigPanda provides a native Splunk App to let you easily send Splunk alerts to BigPanda. The app provides a native Splunk alert action which will forward the Splunk alert to the BigPanda integration. The integration will take the Splunk alert and normalize it into one or more BigPanda alerts, one alert for each row in the search result.

Auto-Resolve

Splunk does not send notifications when alerts are resolved. However, BigPanda will auto-resolve the alerts in BigPanda based on the expiration time defined for the alert in Splunk. This helps you keep your BigPanda incident feed clear and keep your team focused on solving incidents that matter.

Splunk Data Model


BigPanda normalizes the search alert results into multiple BigPanda alerts. Each of these results becomes a BigPanda alert where their data from Splunk is turned into tags. You can use tag values to filter the incident feed and to define filter conditions for Environments. The primary and secondary properties are also used during the correlation process.

Standard Tags

Tag

Description

Attributes

host

The host of the Splunk search result

Primary Property

search_name

Name of the search in Splunk.

Secondary Property

splunk_source

Source field value in the first result row of the Splunk search, if available.

server_host

Host name of the Splunk server.

server_url

Management port of the Splunk server, identified by host, port, and protocol.

app

Name of the Splunk app that contains the alert.

results_link

Link to the search results in Splunk.

results_file

Path to the raw results in a CSV file on your Splunk server.

Additional Tags (Varies)

If available, tag for each additional data field in the alert payload from Splunk.

Installing the Integration


Pre-requisites

  • Create an integration in BigPanda for Splunk

Install and Configure Splunk Add-on

  1. Install the BigPanda for Splunk add-on from Splunkbase in your Splunk instance. (If you are using a distributed Splunk search environment with multiple instances please install the app on your search head instance)
  2. Open the BigPanda app in Splunk and navigate to the Configuration > Global Settings page
  3. Fill in your BigPanda App Key and API Token (BigPanda Bearer Token) inputs
  4. Click Save
  5. From the BigPanda add-on navigate to the Action Manager page
  6. Use the action management dashboard to configure which alerts to send to BigPanda

Customizing Splunk


For any Splunk alert that is sent to BigPanda, you can define a custom description, primary property, or secondary property. You can also specify which integration to use for a given alert, if you have more than one Splunk integration with BigPanda.

Defining Custom Alert Tags

You can enrich Splunk alerts with additional information by defining custom alert tags.

  1. From the Alerts page in Splunk, select an alert to open its detail page, and then use the Click to edit actions link. Alternatively, run a search, and go to Save As > Alert.
  2. At the bottom of the window, click Add Actions, and then select BigPanda.
  3. Enter the custom alert values you want to send to BigPanda when the alert is triggered.

Field

Description

Primary

Adds a custom tag with the name custom_primary and the value specified. You can use Splunk variables in the tag value. If defined, this tag is treated as the primary property for alert correlation and in the incident title.

Secondary

Adds a custom tag with the name custom_secondary and the value specified. You can use Splunk variables in the alert value. If defined, this tag is treated as the secondary property for alert correlation and in the incident subtitle.

Description

Overrides the alert description. You can use Splunk variables in the description value. The default description is the search name.

📘

Custom tag fields are optional.

Using Splunk Variables in Custom Tag Values

To enrich Splunk alerts in BigPanda, you can leverage any search data that is available as a Splunk Token, including search metadata and values from the first row the search results.

  • To access search metadata, use the format $<fieldname>$. For example, $name$ for the search name.

  • To access field values from the first result row that a search returns, use the format $result.<fieldname>$. For example, $result.host$ for the host value and $result.sourcetype$ for the source type.

  • You can leverage any of the Selected Fields in the Splunk search.

  • Add any of the Interesting Fields to the Selected Fields to make the data available as a variable

The following example uses $result.host$ in the primary property value and $result.sourcetype$ for the secondary property value.

Enable the Add-On for the Alerts


The BigPanda App includes multiple ways to Enable the Add-On by leveraging the Splunk Search, the BigPanda Search Commands, and Splunk Dashboards.

Action Manager: Basic Assignment Management

  1. Go to BigPanda > Action Manager

  2. The first row contains information about which alerts are sending to BigPanda and whcih are not. Use the respective buttons to either add or remove BigPanda from all the alerts.

Action Manager: Advanced Assignment Management

  1. Go to BigPanda > Action Manager

  2. The second row contains a filter text box where you can provide a more detailed search query.

  3. Review the query result, then click on the button with the action you want to take.

BigPanda Search Commands

BigPanda also provides search commands to add and to remove the BigPanda Add-On to alerts. The two available search commands are addbigpanda and removebigpanda.

  1. Go to Search & Reporting.

  2. In the search bar do a query following this search structure: | rest /services/saved/searches | YOUR FILTER | SEARCH COMMAND.

Updated about a month ago

Splunk


Splunk allows you to search, monitor, and analyze machine data that is generated by your infrastructure. Install this integration to correlate alerts from Splunk into high-level incidents in BigPanda, and see insights from Splunk alongside the problems detected by other tools in your monitoring stack.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.