Install a Splunk Cloud Integration

BigPanda provides a native Splunk App to let you easily send Splunk alerts to BigPanda. The app provides a native Splunk alert action which will forward the Splunk alert to the BigPanda integration. The integration will take the Splunk alert and normalize it into one or more BigPanda alerts, one alert for each row in the search result.

Splunk Enterprise and Splunk Cloud have similar but different requirements when integrating with BigPanda.

• See the Install a Splunk Enterprise Integration documentation for details on configuring and installing Splunk Enterprise.
• See the Install a Splunk Cloud Integration documentation for details on configuring and installing Splunk Cloud.

👍

Open Integration Manager

The Open Integration Manager is available for use with Splunk Cloud. For more information, see the Open Integration Manager documentation.

❗️

Considerations before updating Splunk to OIM

The Splunk OIM integration uses Time Based Alert Resolution to resolve alerts. Do not switch from the legacy solution if you require variable resolve times based on the Splunk expire time.

The legacy endpoint for Splunk is https://integrations.bigpanda.io/splunk/alerts, and the new OIM endpoint is https://integrations.bigpanda.io/oim/splunk_v2/alerts.

❗️

Noisy Integration

Splunk can be a noisy monitoring system. For organizations using the BigPanda consumption based pricing model, it can consume large amounts of credits very quickly. For help configuring Splunk to be less noisy, see the Splunk FAQ.

Key Features

• Easily send Splunk alerts to BigPanda using the native BigPanda action in Splunk
• Keep your team focused on what matters with auto-resolution of Splunk alerts in BigPanda
• Simple & efficient management of all alerts going to BigPanda via custom alert management dashboard and search commands
• Customizable alert properties

Install the Integration

The Splunk Cloud integration can be installed from the Splunk Web Page

Before You Start

• Splunk admin permissions are required to install the integration
• Users who own searches sent to BigPanda must have the list_storage_passwords permission in Splunk

Create an App Key

Create an app key in BigPanda.

👍

Integration specific

You'll need a separate app key for each integrated system.

Install the BigPanda Splunk Add-on from the Web Page

1. From the Splunk Web home page, click the Apps gear icon.
2. Click Install Apps.
3. Select Install to install an app. If the app that you want is not listed, or if the app indicates self-service installation is not supported, contact Splunk Support.
4. Follow the prompts to complete the installation.

Configure the Integration

Users who own searches which are sent to BigPanda must have the list_storage_passwords permission in Splunk.

The Splunk Cloud integration can be configured through the BigPanda App Configuration Tab.

Configure Through the Configuration Tab

1. Open the BigPanda app in Splunk and navigate to the Configuration > Global Settings tab

   

2. Fill in your BigPanda App Key and API Token (BigPanda Bearer Token) inputs

3. (Optional) Add a Basic Auth header in the Authorization Override field. This may be required by a third-party server when routing traffic through it.

4. Click Save

5. (Optional) If you are using a Proxy, enter the Proxy information in the Proxy tab

   

6. Navigate to the Action Manager page and configure which alerts to send to BigPanda

Send Alerts to BigPanda

Once BigPanda is configured in Splunk, and the BigPanda action has been configured for alerts, you will need to enable the alerts to send to BigPanda.

When defining Trigger Conditions for Alerts to send to Bigpanda, make sure to select the Trigger as Once. BigPanda will extract all individual events from the search results as unique alerts, and does not need to send for each result.

❗️

Triggered Alerts action required

If the Triggered Alerts action has not been added to your Splunk configuration, you may need to first add this in addition to the BigPanda add on.

There are two types of saved searches in Splunk: alerts and reports. You can only use the add-on with alerts that you created, or that were shared with you by the owner.

The alert does not automatically start triggering when the add-on is used. The user must manually enable the alert for it to begin working.

Enable alerts using either the Action Manager, or Search Commands.

Action Manager

The Splunk Action Manager lists all alerts that have been created from previously saved searches. Each of these alerts can be enabled or disabled to send to BigPanda based on the configuration of the alert.

1. Navigate to BigPanda > Action Manager
2. The top left panel lists alerts that are currently enabled to send to BigPanda. The top right panel lists any alerts that are not enabled.
3. Click Stop Sending All Alerts to BigPanda to disable all alerts from sending to BigPanda
4. Click Send All Alerts to BigPanda to enable the alerts listed in the left pane to send to BigPanda

🚧

Potential Noise

The Send All Alerts to BigPanda option will attempt to send every search return for each saved search. For Splunk instances with a high volume of saved searches, this may result in oversized payloads that fail to be processed by BigPanda. Any payload over 6 MB will fail to process with BigPanda.

We recommend reviewing your saved searches to ensure that only actionable, useful information is being sent to BigPanda. See the FAQ How do I make my monitor saved searches actionable? for more information.

👍

View search query at any time

Click any of the alerts on the Action Manager to open up the alert and see configuration settings and the original search query.

Advanced Action Manager

The advanced action manager allows you to enable or disable a subset of alerts using a query filter.

1. Enter a search value into the Filter text box to narrow the list below to only alerts that fit that value
2. Click Send Filtered Alerts to BigPanda to enable all alerts listed to send to BigPanda.
3. Click Stop Sending Filtered Alerts to BigPanda to disable all alerts listed from sending to BigPanda

Search Commands

BigPanda also provides search commands to stop or start sending alerts to BigPanda. The two available search commands are addbigpanda and removebigpanda.

1. In Splunk, Navigate to Search & Reporting
2. In the search bar do a query following this search structure: | rest /services/saved/searches | YOUR FILTER | SEARCH COMMAND
3. This may return saved Report entities in addition to saved Alerts. To ensure that only Alerts are displayed, add | search alert_threshold!="" to your search command.

📘

Search commands in a distributed cluster

When using a distributed cluster, you must be logged into the captain node to be able to successfully use the BigPanda Search Commands. If you are logged into a different node you will receive authentication errors when attempting the BigPanda Search Commands.

The action manager relies in the backend on the BigPanda search commands addbigpanda and removebigpanda.