Managing Correlation Patterns

Correlation patterns can be customized to fit the needs of your organization.

At its core, BigPanda's Algorithmic Correlation relies on pattern recognition. A pre-configured list of patterns is matched against alerts to identify alert clusters in real-time. To classify alerts into incidents, BigPanda looks at information in 4 dimensions:

  • Time
  • Topology (e.g., datacenter, rack, cluster)
  • Context (e.g., criticality, team, customer impact)
  • Alert types (e.g., network, storage, application)

Each correlation pattern defines general properties to correlate: source, timespan, common alert attributes, and a filter.

For example, you can create a pattern to correlate AppDynamics alerts with the same application, starting within 30 minutes of one another, in the production cluster.

You can customize correlation patterns to tailor alert correlation to the specifics of your infrastructure. Correlation patterns are managed from the Correlation Patterns settings page.

Correlation Patterns Settings PageCorrelation Patterns Settings Page

Correlation Patterns Settings Page

Relevant Permissions

Roles with the following permissions can access Custom Tags and Correlation Patterns in the BigPanda Settings:

Custom_tags_Read

Read-only - view existing custom tags in the BigPanda Settings.

Custom_tags_Full_Access

Full access - preview and create new and inactive Extraction and Composition tags in the BigPanda Settings.

Correlations_Read

Read-only - view existing correlation patterns in the BigPanda Settings.

Correlations_Full_Access

Full access - preview and create new correlation patterns in the BigPanda Settings.

To learn more about how BigPanda's permissions work, see the RBAC - Role Based Access Control documentation.

Creating New Correlation Patterns

You can create new correlation patterns at any time. Once activated the correlation patterns will automatically begin correlating new alerts.

❗️

When you create or edit a correlation pattern or activate a previously disabled pattern, only new incidents will be correlated according to the pattern. Existing incidents and alerts will not be impacted. When you disable or delete a pattern, new alerts are no longer correlated according to it. However, existing incidents stay correlated according to the pattern logic for the remaining life cycle of the incident

To create a new correlation pattern:

  1. Navigate to Settings > Correlation Patterns
  2. Click New Pattern
  3. Define the conditions that indicate the alerts are related
Correlation Pattern EditorCorrelation Pattern Editor

Correlation Pattern Editor

Field

Description

Source Systems

One or more integrated monitoring systems for which this pattern applies.

Enable cross source correlation

Option to correlate alerts from different source systems into the same incident. This option applies only if you select more than one source system for the pattern.

  • Select the check box to correlate alerts from different source systems into the same incident, when applicable.
  • Clear the check box to correlate only alerts from the same source into the same incident. The pattern still applies to every alert from every selected source, but will create different incidents for each source.

Correlation tags

Tag names to correlate alerts with matching values. For example, enter ‘cluster’ and ‘check’ to correlate all alerts that come from the same cluster and have the same check.

You can enter up to five tags.

Time Window

Maximum duration between the start time of correlated alerts in minutes.

You can select a time window from 1 minute up to 4320 minutes (3 days).

Query Filter

(Optional) Query that further refines which alerts are correlated. Queries use BigPanda Query Language (BPQL) to filter alerts by alert tag data.

For example, you can specify a tag of datacenter and then enter a query of check=ping to correlate only ping alerts by datacenter.

Note

(Optional) Short description of the pattern. E.g., a note that explains why the pattern is important and how it works.

Create As Inactive

Option to save the pattern definition without affecting your BigPanda instance.

  • Select the check box to create the correlation pattern but not begin correlating alerts according to the pattern.
  • Clear the checkbox to begin correlating new alerts according to the pattern immediately after it is created.

❗️

Correlation Time Window

The correlation time window applies to the first event for a new alert. Alerts are correlated into the same incident only if their first event falls within the same time window (that is, they started around the same time).

  1. Click Create Pattern.

If more than one pattern matches an incident, the incident title is based on the pattern with the largest time window.

The Preview Pane

The Preview pane appears to the right of the Correlation Pattern editor and displays the effects this correlation pattern would have had. The Preview pane uses real historical data in BigPanda without affecting any live data.

The preview displays detailed information for up to 50 incidents that match the correlation pattern within a selected time frame. It correlates alerts from incidents that have been updated in the last 7 days and can be refreshed to update with real-time incoming alerts.

The Preview pane allows you to evaluate the correlation results for:

  • Effectiveness—review the compression rate to see the percentage of alerts that are correlated into incidents. If a pattern is not as effective as it used to be, you may need to optimize the pattern to account for infrastructure changes.
  • Accuracy—review how actual alerts would have been correlated into incidents according to this pattern. Confirm that alerts in each incident are related to the same problem.

Use the Preview pane to refine and adjust correlation patterns as you create or edit them.

Correlation Pattern Preview PaneCorrelation Pattern Preview Pane

Correlation Pattern Preview Pane

As you make adjustments to the correlation pattern settings, click Refresh to update the Preview pane to reflect the changes.

To the right, adjust the date range to see the impact the correlation pattern would have had on older alerts, or in a broader time window.

Edit a Correlation Pattern

Edit a correlation pattern at any time to adjust which alerts will be clustered together.

To edit a correlation pattern:

  1. Navigate to Settings > Correlation Patterns
  2. From the list of correlation patterns, select the pattern you would like to manage
  3. To Edit the pattern, click the Pencil icon or click Edit Correlation Pattern in the details pane
  4. In the Correlation Pattern editor, adjust the properties to fit your needs.
  5. Click Update Pattern to apply the changes.

If the pattern is saved as Active, new alerts will be correlated by the pattern. Open incidents will be correlated by the original pattern that formed the initial incident.

🚧

Correlation Patterns and Custom Tags

Correlation patterns are based off tag names. When editing custom tag names, correlation patterns with the tag must be updated to reflect the change.

Duplicate a Correlation Pattern

Duplicate a pattern to use a previously created pattern as a template for a new pattern. Duplicating a correlation pattern will save time for patterns that are similar, such as two data sources that use most of the same tags, but not all.

To duplicate a correlation pattern:

  1. Navigate to Settings > Correlation Patterns
  2. From the list of correlation patterns, select the pattern you would like to manage.
  3. To Duplicate the pattern, click the Copy icon
  4. Adjust the pattern as necessary to fit the new properties. The system will not allow duplicate patterns and will trigger an error message if you try to save the pattern while it is identical to any other pattern in the system.
  5. Click Duplicate Pattern when you are happy with the changes.

Inactivate a Correlation Pattern

Inactivating a correlation pattern stops BigPanda from using that correlation pattern to cluster alerts together, but still preserves the pattern if you would like to turn it on again. Inactivating a correlation pattern is common when you are researching which adjustments need to be made, or when you are unsure if a correlation pattern is adding meaningful correlation.

To inactivate a correlation pattern:

  1. Navigate to Settings > Correlation Patterns
  2. From the list of correlation patterns, select the pattern you would like to manage.
  3. In the details pane, click the Active toggle.
  4. You will be prompted to confirm the deactivation. Click Deactivate to continue or Cancel to return to the previous screen. The toggle will turn grey.

Reactivate a correlation pattern at any time by clicking the Active toggle. The toggle will turn green and alerts will be correlated by the pattern again.

Delete a Correlation Pattern

Delete a correlation pattern when you are sure that it no longer applies to your infrastructure or process. A deleted correlation pattern cannot be recovered.

To delete a correlation pattern:

  1. Navigate to Settings > Correlation Patterns
  2. From the list of correlation patterns, select the pattern you would like to manage.
  3. To Delete the pattern, click the Trash icon.
  4. Click Delete to confirm, or Cancel to return to the previous page.

New incidents will not be correlated by the deleted pattern, but existing incidents will continue to use the correlation pattern that they were created under.

Searching and Filtering Correlation Patterns

By default, the correlation patterns list is sorted by most recently created or edited pattern. You are able to search and filter the pattern list to make managing your patterns easier.

Enter a term into the search bar to look for correlation patterns with specific names and properties. For example, enter Nagios to see all of the correlation patterns that have Nagios included as a source system.

The filter dropdowns beneath the search allow you to further hone the correlation patterns list. Filter by Correlation Tag, Source, or Status by selecting options from the dropdown lists. To remove your filter settings, click Clear.

Suggested Correlation Patterns

In general, correlation patterns are managed by Administrators and the BigPanda Customer Success team. BigPanda's Machine Learning Engine will also automatically generate correlation pattern suggestions based on historical user data. Upon the integration of a monitoring tool, the review process begins and automatically generated patterns will be suggested in the correlation patterns list.

Suggested Correlation Pattern FlagSuggested Correlation Pattern Flag

Suggested Correlation Pattern Flag

Suggested patterns are highlighted with the purple Suggested label and by default are not active. You are able to edit, activate, duplicate, and delete a suggested correlation pattern like any other correlation pattern, but they will always be marked as system recommended.

To activate a suggested pattern:

  1. Navigate to Settings > Correlation Patterns
  2. From the list of correlation patterns, select the suggested pattern that should be activated
  3. To activate the pattern, click the Active toggle icon in the pattern details pane. The toggle will turn green

New incoming alerts will be clustered by the correlation pattern.