Triage Incidents

Use triage steps to determine priority, establish an assignee, share the incident, and more.

👍

Welcome to the New Docs Site Structure!

BigPanda docs moved to this new organization on September 30th, 2022.

If you're not finding what you're looking for, let us know what's missing in this short survey.

Incidents in BigPanda represent a high-level issue occurring within your infrastructure. BigPanda digests all of the raw data from your integrated monitoring systems and automatically correlates this complex data into incidents.

The first step of incident management is to triage the incident. Triaging the incident allows you to determine priority, establish an assignee who will work on the incident, share the incident with others, merge duplicate incidents, and split incidents that represent multiple issues.

For more information about incidents, see the Incidents in BigPanda and the Incidents Tab documentation pages.

View Incidents in Environments

Environments are global incident filters within the Incidents tab. The All Incidents environment is the default view.

On the left of the Incident tab is the Environments pane. All available environments are listed, with the current Environment highlighted and expanded.

To change which environment you are viewing, select the desired environment’s name from the list.

The Incident Feed will update to show only incidents that are grouped into that environment.

Each environment is sorted into status folders: Active, Unhandled, Shared, Snoozed, Maintenance, and Resolved. Incidents that fit the environment rules will be automatically placed in their respective status folder(s). When selecting an environment, the Active folder will open first. To move to a different folder, select the folder name from the Environments pane.

📘

Incidents will appear in all relevant folders. An incident that has been shared and snoozed will appear in both folders. Resolving an incident will move it to the Resolved folder and remove it from other folders.

The Maintenance folder will only appear if Maintenance Plans have been configured by an admin. See the Maintenance plans documentation for more information.

At the top of the Environments pane, a search bar allows you to filter environments. For organizations that have numerous environments, use the filter feature to quickly isolate a particular environment.

To filter the Environments pane, begin typing the environment name into the Filter search bar. Matching results will appear in real-time.

Clearing the filter search bar will revert the list and show all environments.

👍

Each user is able to Star environments, saving them to a Starred group at the top of the Environments pane. Click the Star beside an environment name, or the Star option in the three dots dropdown to save it for easy access.

Starring and Filtering EnvironmentsStarring and Filtering Environments

Starring and Filtering Environments

Environment Groups

Your environments may be clustered together into environment groups, or categories. Environment groups organize your environments by common functions or properties such as business services, teams, and infrastructure areas. Environment groups aren’t actionable, but can make it easier to navigate the list if you have a large number of environments.

The default General environment group contains all environments not explicitly moved or created under a new group. To avoid duplication, each environment can only be associated with one group.

Each user is able to hide the environments that belong to groups that aren’t relevant to their work.

To hide the environments of a group, hover over the environment group's name and click HIDE. The group will collapse so that only the name appears.

To expand the group at any time, click SHOW.

Prioritize Incidents

Priority tags create a sortable hierarchy to mark the order in which incidents should be addressed. Priority tags make it easier to view the severity and urgency of your incidents at a glance.

26442644

Incident Feed with Priority Tags

Priority tags are visible in the Incident Feed and the Incident Details pane at the top left of the incident, next to incident severity. Priority can be assigned from either of these locations.

📘

The name of priority tags can be edited from within Settings > Incident Enrichment. When the name of a priority tag is changed, the change applies to all existing incidents as well as new incidents.

To learn more about how priority tags work in BigPanda, please see the Priority Tag Customization documentation.

Priority tags can also be used to sort incidents, allowing you to keep your eye on the most important and urgent issues.

📘

Incidents that have not been prioritized will not show the priority box on the ribbon, but hovering over the incident ribbon will bring up the icon to set priority.

Assign Priority

Priority tags can be assigned from both the Incident Feed and the Incident Details pane.

To assign a priority tag:

  1. In the incidents tab, select the incident that you wish to assign a priority.
  2. Select the Priority Tag Marker next to the incident’s alert count in the incident feed or in the incident details pane to open the priority tag dropdown.
  3. Select the priority marker from the options.
10121012

Priority Tag Dropdown

From the incident feed, you can assign priority to multiple incidents at a time by marking the checkboxes next to the relevant incidents and selecting a priority from the dropdown.

Sort by Priority

You can use priority tags to sort the incident feed to bring the highest priority incidents to the top of the feed.

To sort the incident feed by priority:

  1. Navigate to the Incidents tab. Above the Incident Feed, select the Sort icon.
  2. Select Priority.
10101010

Sort by Dropdown

🚧

Incidents that are not assigned a priority will be listed under the prioritized incidents by Last Changed.

Priority tags remain tied to an incident for up to 18 months after it has been resolved.

Manage Priority Options

Priority settings can be customized to better fit the needs of your organization. To learn more about customizing priority tags please see the Manage Incident Enrichment documentation.

Assign an Incident

To ensure effective incident collaboration and management, assign each incident to a responsible party who will see the event through to resolution. If you are responsible for the incident, assigning it to yourself ensures that your teammates know it is being handled so they can focus on other incidents.

Incidents can be filtered and searched by assignee to make it easy to keep focused only on relevant tasks.

To assign an incident:

  1. In the incident feed, hover over the incident which you wish to assign to yourself or another person.
  2. Click the Assign Incident icon on the left-hand side of the incident.
    OR
    Select an incident, and then click Assign Incident at the top right of the incident details pane.
680680

Assign Icon

  1. Select the BigPanda user you want to assign the incident to. Select your name to assign the incident to yourself.

📘

When the list opens, your name appears at the top.

10301030

Assign Dropdown

Once the incident has been assigned:

  • The assignee's profile image appears on the incident
  • The assignment is added to the recent activity feed.
  • If the assignee is currently logged in, a notification appears on their BigPanda UI.
  • When you hover over the assignee's profile image, you can see the assignee's name, who assigned them the incident, and when the incident was assigned to them.
725725

Assignee Information

📘

Assigning Multiple Incidents

You can assign multiple incidents at the same time by using the bulk actions selection boxes.

Reassign Incidents

You can reassign an incident to a different assignee or decide to remove the current incident assignment and leave the incident unassigned.

To reassign an incident:

  1. On the incident, select the current assignee's profile image.
  2. From the dropdown list, select the person you want to reassign the incident to.
  3. To remove the current assignment without assigning it to a new user, select Unassign.
10301030

Reassigning Incidents

📘

Assignments for Resolved and Reopened Incidents

When an incident is resolved, it remains assigned to the same user to ensure that the information is available for searches and historical investigations. If the incident is reopened, the incident is automatically assigned to that same user. For more information about when incidents are reopened, see Incident Life Cycle Logic.

Filter the Incident Feed Using Assignments

In addition to keeping clear tracking of individual incidents, assignments can also be used to filter the incident feed. Filter by your own name to get a clear picture of incidents you are responsible for, or by another team member's name to see their workload.

To filter the incident feed by assignee:

  1. In the Incident Feed, select the Filter by Assignee icon.
846846

Filter By Assignee Icon

  1. In Filter by Assignee, select your name or another assignee’s name from the list. Select Unassigned to see any incidents that don’t have ownership.

📘

You can only search the filter one name at a time.

  1. To refilter the list, clear the filter and select a new assignee from the dropdown list.
298298

Clear filter Option

When a new incident is assigned to you after you have filtered the incident feed, it will not appear on your list of assigned incidents. To display any new incidents assigned to you after filtering, select Refresh below the filter icon. After refreshing the incident feed, new incidents are added to your list of incidents, and these incidents are also updated in real-time.

663663

Refresh Button

📘

Filter by Email

You can also filter the incident feed using an assignee’s email address. In the Search Incidents text box, enter “assignee = <Email_address>.”

The filter displays a list of all incidents assigned to the selected assignee.

Share Incidents

BigPanda incidents can be shared through email or integrated channels to maximize collaboration and streamline resolution workflows. Incident sharing allows you to efficiently notify your team of critical issues, automatically create tickets, or loop in team members who don’t use BigPanda.

Shared incidents include detailed information on the incident including the status, subject, and each active alert in the incident. The share will also include links to the BigPanda incident, and a simplified Incident Preview for easy review.

By default, BigPanda can share incidents through email or SMS. Additional sharing channels can be configured to send BigPanda incidents to your team’s ticketing and collaboration systems. To learn more about setting up sharing channels, see the Manage AutoShare documentation and the Collaboration Integrations list.

Incidents can be shared manually from the Incident Feed at any time. AutoShare rules may also have been configured to automatically share Incidents that meet certain conditions to specific recipients. To learn more about creating AutoShare rules for incidents, see the AutoShare Configuration documentation.

Share an Incident

Incidents can be shared manually at any time, even when in Maintenance or Resolved status. There is no limit to the number of recipients that an incident can be shared with, but incidents can only be shared with each recipient once. Recipients will receive updates on incident changes as long as they are subscribed to the incident.

To share an incident:

  1. Navigate to the Incidents Tab.
  2. In the Incidents Feed, hover over the incident and click the grey Share arrow icon.
    For incidents that have already been shared, the share arrow will be blue and visible even without hovering over the incident.

📘

The share arrow can also be found on the top right of incidents in the incident details pane.

The Share Incident dialog box opens.

10521052

Share Incident Dialog Box

  1. Select the sharing channel from the Share Via dropdown.
    In addition to Email and Text Message (SMS), any sharing channels configured by your administrator will appear in the list. Contact an administrator if you do not see the desired channel.
  2. If you select Email or Text Message (SMS), the Recipients field will appear. Enter recipients’ by email or SMS number.
    As you type, suggested recipients will appear below the box. Click a recipient to add, or finish typing the contact information and hit Enter.
    SMS numbers must be formatted as +(Country Code)(Number)
  3. (Optional) Add a note to the share in the Annotate this share field to give context on why you are sharing the incident. Information you add into the box will be included in the message or ticket created and will appear in the Activity feed for the incident.
  4. Click Share to send the incident, or Cancel to return to the previous screen.

In the bottom left corner of BigPanda a progress dialog box opens. While the share is in progress, you are able to cancel the share by clicking UNDO.

If a problem occurs and the share is unsuccessful, a dialog box will appear in the bottom left with error information.

📘

You can enter any email address in the Recipients field, not just internal BigPanda users. To prevent potentially sensitive data from being sent to the wrong person, it is best practice to select from the suggested users whenever possible.

Incident Preview

When an incident is shared, BigPanda includes a link to an incident preview page where recipients can see the latest incident status without logging in to BigPanda.

23702370

Incident Preview Page

To open an Incident Preview:

  1. Open the email, SMS, ticket, or message where the incident was shared
  2. Click View Updated Incident or click the link listed on the Preview line

👍

The incident preview link is active for 30 days.

The incident preview is updated in real time as the incident evolves and includes key information on the type of events noted, status of the incident, and the source system of the triggering events.

The full list of active alerts in the incident is included in the preview, in a searchable list. Enter alert criteria such as host, check, source, or more to filter the alerts for instant visibility on system impacts and event types.

Incident Updates

Once shared, recipients will be updated when key changes happen. This ensures teams collaborating together externally are always working from the latest BigPanda data. Users can unsubscribe from incidents to stop receiving updates.

BigPanda sends information to the subscribed recipients of a share when:

  • A maintenance window is updated
  • An incident status has changed
  • An acknowledged value has changed
  • An incident enters a snoozed state
  • An incident enters a flapping state
  • A comment has been added (disabled by default)
  • An incident tag is changed
  • An incident changes environments

🚧

To reduce noise levels, BigPanda does not send updates for snoozed or flapping incidents after recipients have been notified the incident has entered that state.

👍

Webhook Update Rules

Integrations that use the Notifications Webhook follow different sharing rules. Notification Webhooks automatically send an update sync every 2.5 minutes with all changes since the last update. This update is sent regardless of flapping status.

View Existing Shares

Incident shares can be viewed from the Incidents Tab.

The blue share icon will appear in the incident feed or top right actions bar for any incidents that have already been shared manually or through AutoShare. Beside the blue arrow, the number of shared recipients will be listed.

Shares will also appear in the Incident Details pane on the Activity tab.

Click the number beside the blue share icon to view sharing details.

15961596

Existing Shares Window

The sharing details popup includes the sharing history and subscriber information for the incident, including:

  • How many times the incident has been shared, broken out into a Manual Shares and an AutoShare section.
  • The recipient of the share.
  • When each share was first sent.
  • If the incident was manually shared, which user shared it.
  • Whether the last attempt to share with the recipient was successful.
  • Whether the recipients are currently subscribed to receive updates on the incident.
  • If the incident was shared with an external ticketing system, the current status of the ticket in the target system and a direct link to it will both be listed.

Unsubscribe From a Shared Incident

When an incident is shared with a recipient, they are automatically subscribed to updates for that incident. You can unsubscribe a recipient from updates for a specific incident from either the incident in BigPanda or through the Incident Preview.

To unsubscribe in BigPanda:

  1. Navigate to the Incidents tab and locate the incident in the incident feed.
  2. Click the Number of Shares icon to open up the Existing Shares window.
  3. Find the recipient's name in the shares list.
  4. Click the Subscribed toggle.

The toggle will turn gray. To resubscribe, click the toggle again.

To unsubscribe through the Incident Preview:

  1. Open the Incident Preview from the shared message.
  2. Click Stop Updates.

The page will note that Email updates are currently disabled. To resubscribe, click Resume Updates.

Once unsubscribed, a recipient will no longer receive update emails, SMS messages, or ticket updates for the incident. The Incident Preview will remain available as long as the link is still active.

📘

You can resubscribe to an incident that was shared with you at any time

Merge Incidents

Merging enables you to add the alerts from source incidents to a designated destination incident to be handled as one singular incident.

When merged, all the alerts in the source incidents (along with their history) are moved to the destination incident. After a merge, source incidents will no longer contain any alerts, resolving the incidents and removing them from the incident feed.

The destination incident maintains its assignee and snooze settings. If the incident is shared, all changes made to the incident are updated and shared in real time.

487487

To merge incidents:

  1. In the Incident Feed, select incidents and click Merge at the top of the incident details pane to open the merge popup.
  1. Optional: If you would like to change the automatically selected destination, manually change the source incident you would like to assign as the destination by clicking Use as Destination.

  1. Click Merge Incidents. The merged incident now appears in the incident feed, marked with a "Manual" badge.

Split Incidents

Splitting is separating a source incident's alerts to form a new destination incident. Incidents may need to be split when:

  • The algorithm grouped together alerts that were not closely related.
  • Multiple factors led to the creation of the incident.
  • Certain alerts within the incident were resolved.

When splitting an incident, selected alerts from “source” incidents are moved (along with their history) to the newly created destination incident. The source incidents maintain their assignee and snooze settings. If the source incident is shared, any further updates to the alerts that remain in the original incident after the split are sent in real time.

The newly created destination incident is subject to the same rules as any other new incident in the system.

To split an incident:

  1. In the Incident Details pane on the Alerts tab, select the alerts you would like to split out. Click Split at the top right of the Alerts table to open the split pop-up.

🚧

You cannot split out all the alerts in the incident.

  1. Click Split Incident. The split incident now appears in the incidents feed. Both source and destination incidents are marked with the "Manual" badge.

📘

After merging or splitting, because incident correlation was manually intervened, all split incidents will be marked “Manual”. Manual incidents will no longer be included in automatic correlation, meaning no new alerts will be added. Existing alerts will still be updated.

Snooze Incidents

Snoozing non-urgent incidents can help keep your team focused on the right issues. For example, a low disk space issue can often wait weeks before it becomes urgent. If you are not planning to do anything about a low-priority issue right now, it can be helpful to get it out of the way. Snoozing an incident removes it temporarily from the incident feed.

Snoozing an IncidentSnoozing an Incident

Snoozing an Incident

To snooze an incident:

  1. Navigate to the incident you would like to snooze, and click the Snooze bell icon.
  2. Configure the snooze:
    • Snooze for—select how long the incident should be removed from view.
    • Annotate this snooze—(optional) enter a comment to appear in the incident activity feed.
    • Cancel snooze on new alerts or critical updates toggle—select the check box to automatically cancel the snooze if:
      • A new alert is added.
      • The severity of an existing alert increases such as when an alert changes from warning to critical.
      • The incident is resolved.

Clear the check box to keep the incident snoozed until the snooze period elapses, regardless of updates to the incident.

  1. Click Snooze.

The incident no longer appears in the incident feed. When the Snooze period ends, the incident again appears in the active feed.

📘

Canceling Actions

Make a mistake? Incident actions that change how incidents are viewed or organized can be canceled for 5 seconds after the action was taken. An Undo button will appear at the bottom left of the screen.

View Snoozed Incidents

To see all the snoozed incidents, click the Snoozed folder in the left pane. To cancel the snooze or to change the snooze period, click the bell icon.

Perform Actions on Multiple Incidents

Bulk ActionsBulk Actions

Bulk Actions

The bulk actions pane appears and shows the number of incidents selected and the actions you can perform.

You can prioritize, merge, assign, resolve, snooze, comment on or share multiple incidents at the same time with the bulk actions pane.

  1. (Optional) Perform a search to show all applicable incidents in the feed.

You can perform bulk actions only on incidents that appear in the feed at the same time. The bulk actions pane covers the search bar, preventing you from doing searches after selecting incidents.

  1. Select the check box beside each relevant incident.
  • To select consecutive incidents, select the first incident, then press Shift and select the last incident. The incidents and all incidents between them are selected.
  • To see the details in the right pane, click an incident in the feed.

️ Clear Selections to Enable Actions on Incidents

To clear all incident check boxes without performing an action, click the Deselect all check box in the bulk actions pane. You must clear all selections to hide the bulk actions pane and enable the actions on individual incidents.

  1. Click the icon for the action you want to perform.
  • Merge - Group incidents together to be handled as one incident.
  • Assign - Assign the incident to an owner who will be responsible for seeing it through to resolution.
  • Resolve - Mark the issue Resolved.
  • Snooze - Snooze active incidents; cancel or change the settings for snoozed incidents.
  • Comment - Add a comment or view previous comments from your colleagues.
  • Share - Share an incident to collaborate with key team members.
  1. Fill in the details and complete the action as you would for a single incident.

Next Steps

Start Remediating Incidents

Learn more about Navigating the Incidents Tab

Dig into The Incident Life Cycle