Manage Planned Maintenance
Maintenance plans reduce unnecessary noise by marking and silencing alerts triggered by system maintenance.
A maintenance plan defines conditions to suppress and silence alerts during scheduled system maintenance. An alert that matches a maintenance plan condition and occurs within the maintenance time window will be marked for maintenance.
Alerts marked for maintenance will still be correlated into an incident if they match the incident's correlation criteria, but follow specific rules to keep these alerts from cluttering your Incident Feed.
Alert level
Maintenance plans function on the alert level. Incidents marked for Maintenance remain in the Maintenance folder only if all alerts in the incident were received and last updated within the maintenance window.
Audit logs
When multiple users are working in the BigPanda platform, it’s vital to see historic information on system configuration changes. Audit logs are available for alert enrichment in BigPanda, allowing admins to keep track of configuration changes.
Users with the Audit Logs permission can view historical configuration changes and actions made related to maintenance plans. See the Use the Audit Log documentation for more information.
Relevant Permissions
Roles with the following permissions can access the Maintenance Plans section, the Alert Filters API and the Maintenance Plans V2 API:
| Role Name | Description |
|---|---|
| Plans | View, create, edit, and delete maintenance plans in BigPanda Settings. |
Permission access levels can be adjusted by selecting either View or Full Access. To learn more about how BigPanda's permissions work, see the Roles Management guide.
Stopping plans
Only users with Full Access are able to stop a plan.
Maintenance Alerts in Incidents
An incident's Status and Last Changed timestamps do not take maintenance alerts into consideration. For example, if one alert within an incident is marked as Warning and another is Critical but is marked for maintenance, the critical alert is silenced for the maintenance period. The incident is classified as Active and the status is Warning.
Matching alerts that begin before the maintenance plan window are marked for maintenance only if they are updated within the maintenance window. Marked alerts remain silenced even after the time window ends unless they are updated with a new active status after the maintenance time window.
Incidents composed exclusively of alerts that are marked for maintenance can be found in the Maintenance folder within BigPanda environments. If an incident has at least one alert that is not marked for maintenance, it appears in the folder relevant to the incident's Active alerts.
Example Maintenance Flows
If a maintenance plan starts at 2 AM and ends at 3 AM:
For an incident that contains only alerts that match the maintenance plan rules:
- If the incident starts at 1:30 AM, it will not enter the Maintenance folder.
- If the incident starts at 2:30 AM, the incident will be flagged for maintenance and be moved to the Maintenance folder. The incident will remain in the Maintenance folder even after 3 AM. After 3 AM, if a new alert enters the incident or an existing alert is updated, the incident will be moved out of the Maintenance folder even if it matches the plan rules.
For an incident that will be correlated with 2 alerts - A) An alert that matches, B) An alert that does not:
- If the incident starts at 1:30 AM, it will not enter the Maintenance folder.
- If the incident starts at 2:30 AM, with only alert A, it will be moved into the Maintenance folder. When alert B is correlated to the incident, the maintenance flag will be removed from the incident and the incident will be moved to the Active folder.
- If the incident starts at 2:30 AM, with only alert B, it will not be moved into the Maintenance folder when alert A is correlated.
Learn more
To learn how to completely suppress alerts during maintenance, check out Manage Alert Filtering
Plans that have been suppressed will not be visible in the BigPanda console.
View Maintenance Plans
Current, future, and past maintenance plans can be viewed within the Planned Maintenance settings screen.
View Maintenance Plans
Maintenance plans have three possible statuses:
- Planned: The plan is scheduled to run at a future time, but is not currently suppressing alerts.
- Running: The plan is currently suppressing matched alerts.
- Done: The plan suppressed matched alerts in the past, but is not scheduled to run again.
You can filter the list of maintenance plans by entering a search term in the field above the list.
Viewing finished plans
Maintenance plans marked "Done" within the past 60 days can be viewed in Planned Maintenance. After 60 days, these plans will no longer be available via the UI or API.
Create a Maintenance Plan
Maintenance Plans are managed in the Planned Maintenance settings screen.
Plan limit
Each organization can only have 3,500 maintenance plans at a time.
To create a new Maintenance Plan:
- Navigate to Settings > Planned Maintenance.
- Click + New Plan.
- In the Create a New Plan window, enter plan details.
- Click Create Plan.
- (Optional) In the right pane, click Preview alerts to see a sample selection of historical alerts that match the plan source and condition.
Incoming alerts that match the plan condition will be automatically marked with the Maintenance tag. Incidents with all alerts in maintenance will be moved to the Maintenance folder in BigPanda environments.
Create a New Plan
| Field | Description | Options |
|---|---|---|
| Plan name | A short and descriptive name for the plan. | |
| Condition | A BPQL condition to define which alerts should be suppressed during the maintenance period. | |
| Define time window | Parameters for the maintenance window. | • Time zone: The time zone for your maintenance plan. This is defined per-plan and is separate from your user time settings. Plans will automatically adjust for daylight savings in this time zone. See supported time zones. • Starts: What time and day the plan should first run. • Ends: What time and day the first time window should end. • Repeat: How often the plan should run. Options are: Does not repeat, Daily, Weekly, Monthly (relative) or Monthly (specific date). • End repeat: When the plan should stop repeating. For plans that should run indefinitely, choose a date in the far future. |
| Add description | An optional note explaining the reason for the maintenance plan. | |
| Once plan ends | What should happen to suppressed alerts when the maintenance window ends. | • Suppress alerts: Leave alerts marked as in maintenance. This is the default. • Resolve alerts: Mark all suppressed alerts as resolved. • Reactivate alerts: Mark all suppressed alerts as active. |
Customizing post-maintenance behavior
When choosing the option to resolve or reactivate alerts after a maintenance window ends, the selected action will take effect within five minutes. This action will appear in Recent Activity for affected incidents.
If the plan suppressed a large number of alerts and these are returned to active, they will be fed gradually back into the pipeline so as not to negatively impact an active critical event.
Note: These additional actions are only available for plans that are configured for up to three days. For maintenance plans longer than three days, you will not be able to automatically resolve or reactivate alerts.
AutoShare Trigger
Suppressed alerts that are moved to active may trigger AutoShares. AutoShares may also trigger if all the alerts in an incident are marked resolved.
A summary of the window schedule will appear below the options, making it easy to ensure the plan is scheduled for the correct period.
Specific Date vs Relative Repetition
For monthly recurring windows with a selected date, the plan will only run in months where that date occurs. When scheduling a window on the 29th, 30th, or 31st, the plan may not run every month.
Relative dates will run on the closest available date, and will run every month. When selecting the "Last" weekday, the plan may run on the 4th or 5th week, depending on the number of times that weekday occurs in that month.
Condition limitations
Maintenance plan conditions cannot be longer than 25,000 characters long.
Not all tags are available for maintenance plan conditions. See the Tag Naming documentation for a list of tags that have limited functionality in BigPanda.
Granular time settings
The selection field in the UI uses half-hour segments to simplify scheduling. If you need to configure a schedule window with a more granular time, use the Maintenance Plans V2 API with the Frequency options added (see below).
End Date Required
Recurring windows must have an end date. To schedule a plan that should run indefinitely, select an end date in the distant future.
Manage Maintenance Plans
You can edit, duplicate, or permanently delete each maintenance plan created.
To manage maintenance plans:
- Navigate to Settings > Planned Maintenance. A list of existing maintenance plans appears.
- Select the maintenance plan you wish to edit, duplicate, or delete.
- Use any of the following options to modify the maintenance plan:
| Option | Description |
|---|---|
| Edit | Any maintenance plan that is In Progress or Planned can be edited. Maintenance plans with the status Done can no longer be edited. a. Click the Pencil icon on the incident tag ribbon, or the Edit Plan button in the incident tag details pane. b. The Edit Plan window opens, allowing you to make changes to the maintenance plan. c. When satisfied with the settings, click Update Plan to save. |
| Duplicate | Duplicate a maintenance plan to use a previously created plan as a template. a. Click the Duplicate Plan icon. b. Adjust the maintenance plan as necessary to fit the new properties. c. Click Duplicate Plan when you are happy with the changes to save. |
| Delete | Delete a maintenance plan when you are sure that it no longer applies to your infrastructure or process. A deleted maintenance plan cannot be recovered. a. Click the Trash icon. b. Click Delete to confirm, or Cancel to return to the previous page. |
You can use the Maintenance Plans V2 API to manage and schedule your Maintenance Plans.
Alert filters
To learn how to entirely prevent alerts from entering BigPanda or correlating to incidents, check out Alert Filters or the Alert Filter Plans API.
Please note that plans that alerts that have been filtered will not be visible in the BigPanda console.
Next Steps
Learn to navigate the BigPanda Settings page
Learn to use the Maintenance Plans V2 API
Learn to completely remove alerts using Alert Filtering
Updated 3 months ago