Nagios Notifications
The notification-based Nagios integration is ideal if you've configured custom filters for Nagios notifications.
Nagios provides monitoring for systems, applications, services, and business processes in your IT infrastructure. Install the notification-based Nagios integration if you've configured custom filters for Nagios notifications that are working well for your needs. BigPanda will correlate problems detected in Nagios into actionable incidents in BigPanda so you can understand and respond faster to critical issues in your infrastructure.
Nagios sends notifications to a BigPanda Contact, which is configured during the installation process. BigPanda then processes and correlates the alerts from Nagios to create incidents in BigPanda. Alert data in BigPanda is updated only when another notification is received, which is based on the notification interval defined for the service.
The BigPanda agent also parses these files:
-
The status.dat file—performs a full sync to send cleared events for open notifications in BigPanda. This sync ensures that BigPanda can recover from connectivity lapses. You can customize the interval or disable the full sync.
-
The objects.cache file—allows BigPanda to enrich alerts with host-groups and service-groups as well as custom information. For custom enrichments, configure custom tags or contact BigPanda support.
When an alert closes in Nagios, the alert is closed in BigPanda.
Scheduling Downtime for Maintenance
You can schedule downtime in Nagios for a maintenance window. During a scheduled downtime, Nagios suppresses normal notifications for the host or service and sends a start event for the downtime.
After receiving the start event for a scheduled downtime, BigPanda handles subsequent alerts from objects in maintenance as follows:
-
Does not show new incidents in the incident feed.
-
Correlates alerts to existing incidents, if applicable, and marks the alerts in Maintenance status (indicated in grey).
-
Does not send AutoShare notifications for incidents where all associated alerts are in maintenance.
After receiving the end event for the scheduled downtime, BigPanda resumes normal processing for subsequent alerts from the objects.
Which alerts are in Maintenance
BigPanda handles scheduled downtime according to the options you select when configuring the downtime. For example, if you configure a host to be in maintenance but not the services on that host, then alerts for those services do not change to maintenance status in BigPanda.
Key Features
-
Integrates BigPanda with Nagios by using notifications.
If you don't use Nagios notifications, use the log-based Nagios Integration. For more information about the differences, see Nagios. -
Processes only the Nagios alerts that generate notifications and correlates them into high-level incidents.
-
Provides insights into the infrastructure that is most important to you.
-
Allows you to independently configure custom tags to send to BigPanda.
System Requirements
The Nagios integration supports 64-bit servers. If your Nagios implementation is running on a 32-bit server, contact BigPanda Support and request a product change.
Nagios Notifications Data Model
BigPanda normalizes alert data from Nagios into tags. You can use tag values to filter the incident feed and to define filter conditions for Environments. The primary and secondary properties are also used during the deduplication and correlation processes.
Standard Tags
Tag | Description | Attributes |
---|---|---|
host | Name of the monitored host. | Primary Property. |
check | Name of the service being checked on the host. | Secondary Property. |
hostgroups | List of groups where the host is configured as a member. To enable this tag, provide access to the objects.cache file, then contact BigPanda support and request a product change. | |
servicegroups | List of groups where the service is configured as a member. To enable this tag, provide access to the objects.cache file, then contact BigPanda support and request a product change. | |
acknowledged | Indicator of whether the alert has been acknowledged in Nagios. | |
flapping | Indicator of whether the monitor is flapping. | |
maintenance | Indicator of whether the monitor is scheduled for downtime. |
Custom Tags
You can independently configure the Nagios Notifications integration to send custom Nagios object variables to BigPanda. These custom tags become available in BigPanda along with the standard tag data to help users resolve problems.
To send custom tags to BigPanda:
- In your Nagios configuration files, find the name and object type for the custom variable.
For example, open the Nagios file /usr/local/nagios/etc/objects/localhost.cfg.
In the following example, a custom variable for service object is named _test_service_tag.
Note: Custom variable names must begin with an underscore (_).
define service{
use local-service
host_name devbox
service_description Swap Usage
check_command check_local_swap!20!10
_test_service_tag Test-Service-Tag ; my custom var
}
-
Edit the BigPanda contact file
bigpanda-contact.cfg.
The default location for the file is:/etc/bigpanda/bigpanda-contact.cfg.
-
Add the argument name and custom variable to the BigPanda notification command.
To see the custom tag in BigPanda, argument names must begin with _SERVICE
for service-based notifications or _HOST
for host-based notifications.
For example:
For a service with a custom variable named _TEST_SERVICE_TAG
, add: _SERVICETEST_SERVICE_TAG="$_SERVICETEST_SERVICE_TAG$"
For a host with a custom variable named custom_var, add: _HOSTTEST_SERVICE_TAG="$_HOSTTEST_SERVICE_TAG$"
define command {
command_name notify-service-by-bigpanda
command_line /usr/bin/bigpanda-notification HOSTOUTPUT="$HOSTOUTPUT$" HOSTSTATE="$HOSTSTATE$"
HOSTNAME="$HOSTNAME$" HOSTGROUPNAMES="$HOSTGROUPNAMES$" LASTSERVICESTATECHANGE="$LASTSERVICECHECK$" LASTHOSTSTATECHANGE="$LASTHOSTCHECK$" LONGSERVICEOUTPUT="$LONGSERVICEOUTPUT$"
LONGHOSTOUTPUT="$LONGHOSTOUTPUT$" NOTIFICATIONTYPE="$NOTIFICATIONTYPE$" SERVICEOUTPUT="$SERVICEOUTPUT$"
SERVICEDESC="$SERVICEDESC$" SERVICESTATE="$SERVICESTATE$" SERVICEGROUPNAMES="$SERVICEGROUPNAMES$"
_SERVICETEST_SERVICE_TAG="$_TEST_SERVICE_TAG$"
}
Tag names may differ from custom variable names
BigPanda automatically normalizes the custom variable names to meet the tag naming requirements, as necessary. For example, if you send a custom variable with the name _Host.name, BigPanda normalizes the tag name to
host_name.
Install the Integration
Administrators can install the integration by following the on-screen instructions in BigPanda.
Customize Nagios Notifications
The BigPanda agent performs a full sync of all Nagios events at regular intervals to ensure that all cleared events are sent. You can configure the query interval or disable the full sync by editing the config file for the agent. You can also change the directories for event files and the dump folder.
Configure the Query Interval
You may want to configure the full sync to reduce the load on your Nagios server, depending on the average volume of events Nagios generates.
-
Open the config file, which is located at
/etc/bigpanda/bigpanda.conf.
-
Locate the settings for the Nagios full sync, named
nagios/statusdat.
-
Change the value of the
refresh_interval
to your preferred interval between full syncs, in seconds.
The default value is3600
(1 hour).
Disable the Full Sync
The full sync ensures that BigPanda can recover from connectivity lapses. Disable this feature only if you need to reduce the load on your Nagios server.
-
Open the config file, which is located at
/etc/bigpanda/bigpanda.conf
-
Locate the settings for the Nagios full sync, named
nagios/statusdat.
-
Change the value of
"enabled"
tofalse.
{
"endpoint": "/data/events/alerts",
"name": "nagios/statusdat",
"enabled": false,
"app_key": "092ut0jwoiqp3t9iwp09843qt",
"action": "POST",
"config": {
"status_dat": {
"filename": "/usr/local/nagios/var/status.dat",
"ok_only": true,
"refresh_interval": 3600
}
}
}
Change the Directories for Event Files and the Dump Folder
Event files contain the alert payload that is sent to BigPanda. The integration temporarily persists this information to disk to ensure delivery if the communication channel is disrupted. Event files are also copied to the dump folder, which is used for debugging purposes and is not processed by the agent. You may want to change these locations; for example, to use a disk with more storage.
-
Ensure your agent is upgraded to version 4.1.0-2 +. You can check the version in the agent configuration file:
/etc/bigpanda/bigpanda.conf.
If your version is older than the required version, upgrade to the latest version of the BigPanda agent. -
Open the
bigpanda-contact
config file, which is located at/etc/bigpanda/bigpanda-contact.cfg.
-
In the BigPanda notification command, add any of the following flags to the
command_line
to change the destination directory settings.
Flag | Description |
---|---|
-queue-dir | Changes the destination (write) directory for event files (default: /var/lib/bigpanda/queue ). You must also change the source (read) directory in the BigPanda agent config file (see step 4). |
-dump-queue-dir | Changes the location of the dump folder (default: /var/lib/bigpanda/queue_dump ). |
-create-dirs | Specifies whether to create the destination directories if they doesn't exist (default: true ). |
define command {
command_name notify-service-by-bigpanda
command_line /usr/bin/bigpanda-notification -queue-dir "/var/lib/MyDirectory/queue" -dump-queue-dir "/var/lib/MyDirectory/queue_dump" -create-dirs "false"
HOSTOUTPUT="$HOSTOUTPUT$" HOSTSTATE="$HOSTSTATE$" HOSTNAME="$HOSTNAME$" HOSTGROUPNAMES="$HOSTGROUPNAMES$" LASTSERVICESTATECHANGE="$LASTSERVICESTATECHANGE$" LASTHOSTSTATECHANGE="$LASTHOSTSTATECHANGE$" LONGSERVICEOUTPUT="$LONGSERVICEOUTPUT$" LONGHOSTOUTPUT="$LONGHOSTOUTPUT$" NOTIFICATIONTYPE="$NOTIFICATIONTYPE$" SERVICEOUTPUT="$SERVICEOUTPUT$" SERVICEDESC="$SERVICEDESC$" SERVICESTATE="$SERVICESTATE$" SERVICEGROUPNAMES="$SERVICEGROUPNAMES$"
}
define command {
command_name notify-host-by-bigpanda
command_line /usr/bin/bigpanda-notification -queue-dir "/var/lib/MyDirectory/queue" -dump-queue-dir "/var/lib/MyDirectory/queue_dump" -create-dirs "false"
HOSTOUTPUT="$HOSTOUTPUT$" HOSTSTATE="$HOSTSTATE$" HOSTNAME="$HOSTNAME$" HOSTGROUPNAMES="$HOSTGROUPNAMES$" LASTSERVICESTATECHANGE="$LASTSERVICESTATECHANGE$" LASTHOSTSTATECHANGE="$LASTHOSTSTATECHANGE$" LONGSERVICEOUTPUT="$LONGSERVICEOUTPUT$" LONGHOSTOUTPUT="$LONGHOSTOUTPUT$" NOTIFICATIONTYPE="$NOTIFICATIONTYPE$" SERVICEOUTPUT="$SERVICEOUTPUT$" SERVICEDESC="$SERVICEDESC$" SERVICESTATE="$SERVICESTATE$" SERVICEGROUPNAMES="$SERVICEGROUPNAMES$"
}
- If you changed the destination (write) directory for event files, change the source (read) directory:
-
Open the BigPanda agent config file, which is located at
etc/bigpanda/bigpanda.conf.
-
Locate the settings for Nagios notifications, named
nagios/notifications.
-
Change the value of the
queue_directory
setting to the same directory that you specified in Step 3.
{
"name": "nagios/notifications",
"enabled": false,
"action": "POST",
"app_key": "092ut0jwoiqp3t9iwp09843qt",
"endpoint": "/data/integrations/nagios/notifications?app_key=092ut0jwoiqp3t9iwp09843qt",
"config": {
"queue_directory": "/var/lib/MyDirectory/queue",
}
}
Uninstall the Integration
Deleting an integration requires changes to both the integrated system and BigPanda. You must uninstall the integration on the integrated system and then delete the integration from BigPanda.
When replacing an existing integration with a new tool or system, we recommend configuring the new integration first to ensure no data is lost.
Stop Sending Data to BigPanda
Manually resolve any open alerts sent from the integration to remove the associated incidents from your incident feed. These incidents will not automatically resolve without an ok
status from the original sending integration.
Remove BigPanda from Nagios Configuration Files
-
Open the Nagios main configuration file.
The default location is:/usr/local/nagios/etc/nagios.cfg.
-
Remove all references to the BigPanda contact and the
bigpanda-contact.cfg file.
-
Restart the Nagios service.
Uninstall the Agent
If the agent supports only the Nagios integration, you can uninstall the agent from the server.
Do not uninstall the BigPanda agent if it is supporting other BigPanda integrations. In this case, remove the Nagios integration from the agent.
Remove the Nagios Integration from the Agent
If the agent is supporting multiple integrations, you can remove only the specific integration without disabling the other BigPanda integrations that the agent supports.
- Stop the BigPanda agent by running the following command.
sudo service bigpanda stop
- Remove the integration from the agent configuration file by running the following command.
Replace the app key with the corresponding value in BigPanda
sudo bigpanda-config -r <INTEGRATION APP KEY>
- Start the BigPanda agent by running the following command.
sudo service bigpanda start
Delete the Integration from BigPanda
- In BigPanda, navigate to the Integrations tab and select the desired integration from the list.
- In the integration details on the right, click Delete Integration. A support message opens, pre-populated with a request to delete the selected integration.
- Press Enter to send the request.
- The BigPanda support team will remove the integration from the UI.
To stop sending Nagios events to BigPanda, you can uninstall the agent from the server or remove the Nagios integration from the agent.
Updated about 1 year ago