Zabbix

Send monitoring events from Zabbix to BigPanda.

Supported VersionsTypeAuthentication Type
2.2, 3.0, 3.2, 4.x, 5.x, 6.xWebhookOrg Bearer Token

Zabbix is an open-source solution for network and application monitoring. Install this integration to correlate noisy trigger alerts from Zabbix into actionable incidents in BigPanda.

👍

The Open Integration Manager is available for use with Zabbix. For more information, see the Open Integration Manager documentation.

📘

If you previously installed the agent-based BigPanda Zabbix integration, you can upgrade at any time to the webhook version.

Key Features

  • Keeps you up-to-date on the state of triggers by correlating the Zabbix events for each trigger into easy-to-track alerts.
  • Correlates alerts for triggers in the PROBLEM state into actionable incidents in BigPanda.

Before You Start

To ensure a successful integration of Zabbix with BigPanda, the following requirements need to be established first.

  • A Zabbix user with Read access to all of the host groups that you want to monitor in BigPanda. Please see the Configure Zabbix Permissions section for more information.
  • The Zabbix server must be able to communicate with the BigPanda server via https (port 443).

Disable Zabbix Triggers

Triggers that are disabled in Zabbix are not monitored by BigPanda. If a trigger is disabled when an alert is open, the alert will not close automatically and the BigPanda incident will remain open. In this case, you must manually resolve the BigPanda incident.

We recommend that Zabbix administrators check that all alerts are resolved in BigPanda before disabling the related trigger.

Zabbix Data Model

BigPanda normalizes alert data from Zabbix into tags. You can use tag values to filter the incident feed and to define filter conditions for Environments.

During alert correlation, BigPanda assigns correlated events an incident identifier or incident_key. By default, the incident_identifier for Zabbix alerts is the triggerid.

The incident_identifier is critical during the alert correlation process and defines deduplication and resolution rules.

Primary and Secondary properties are also important to the correlation process and may impact which alerts are correlated together.

Standard Tags

TagDescriptionAttributes
hostsNames of the Zabbix hosts that the trigger belongs to.Primary Property.
itemsZabbix items that are contained in the trigger. Zabbix items are individual metrics defined for Zabbix hosts. For example:

- net.tcp.service[smtp,,25].last

- ntp.client.offset.max

- disk.io.status[sda].sum
Secondary Property.
item_valuesItems and their corresponding values. The items match the list in the items tag. For example:

- net.tcp.service[smtp,,25].last: 1

- ntp.client.offset.max: 0.5258

- disk.io.status[sda].sum: 86.0000%
triggeridName of the trigger in Zabbix.Incident_Identifier
trigger_descriptionDescription of the trigger in Zabbix.
maintenanceFlag indicating if the host is under maintenance. This tag is included only when the value is true.
expressionTrigger expression configured in Zabbix to calculate the trigger state.
acknowledgedFlag indicating if the latest event for the trigger has been acknowledged in Zabbix.
rum book_urlURL configured in Zabbix as a link associated with the trigger.

Install the Integration

Administrators can install the integration by following the on-screen instructions in BigPanda.

Configure Zabbix Permissions

Connecting BigPanda with Zabbix requires a new Zabbix user with read access to all the host groups that you want to monitor in BigPanda. During configuration, you can choose the level of access for this user—Zabbix Super Admin or Zabbix Admin—and the user is created automatically with the user name Bigpanda in a new user group called BigPanda Service.

Granting the user Zabbix Super Admin permissions is the easiest way to ensure BigPanda has the required access. If granting this level of access is not an option, you can choose Zabbix Admin. The user will be given permissions to all existing host groups at the time of the configuration and you must manually grant permissions to any host groups added after the initial configuration. You can view and change the type of user in Zabbix at any time.

Before You Start

Obtain access to the Zabbix UI with a Zabbix Super Admin user.

Grant BigPanda Access to Host Groups in Zabbix

If the Bigpanda user has Zabbix Super Admin permissions, then BigPanda has access to relevant data for all host groups in Zabbix without additional configuration. If the Bigpanda user has Zabbix Admin permissions, then any access to host groups must be explicitly granted in Zabbix by configuring permissions for the BigPanda Service user group.
To manually grant BigPanda access to host groups in Zabbix:

  1. In Zabbix, go to Administration > Users > BigPanda Service > Permissions. The Zabbix permissions for the BigPanda Service user group are displayed.
  2. Under Composing Permissions > Read only, click Add and then select the host groups.

📘

Host groups that appear under Calculated Permissions > Deny are not accessible to the Bigpanda user, so events for these host groups will not be sent to BigPanda. If you want to send events for these host groups to BigPanda, add them to the Read-Only permissions, as described in step 2, and they will be automatically removed from the Deny list.

  1. At the bottom of the screen, click Update.

Change the BigPanda Access Level in Zabbix

1.In Zabbix, go to Administration > Users.

  1. Under Members, click the Bigpanda user.
  2. Click Permissions.
  3. In the User type field, select Zabbix Super Admin or Zabbix Admin to change the access level.
  4. At the bottom of the screen, click Update.

Deactivate the Integration

If you want to stop sending data to BigPanda but don’t want to delete your integration, you can temporarily deactivate it.

To deactivate an inbound integration:

  1. In BigPanda, navigate to the Integrations tab and select the desired integration from the list. This will open integration details on the right side of the window.
  2. At the top of the integration details, click the Active/Inactive toggle next to the application name to change the status of the integration.

🚧

Alert Resolution for Inactive Integrations

Any active alerts belonging to an inactive integration must be manually resolved or they will stay in the system until the auto-resolve window is reached.

Uninstall the Webhook-Based Zabbix Integration

You may want to remove an integration if you are upgrading to a new version, or if your infrastructure changes. Generally, deleting an integration requires changes to both the integrated system and BigPanda. Follow the steps below to remove an integration from BigPanda and your integrated system.

Before You Start

  • Modify or remove any Environments or AutoShare rules that reference the integration. All active alerts from the integration will be resolved after deletion.
  • (Optional) Delete any shared data that was posted in the integrated system. For example, close any incident tickets opened by BigPanda in the target system.

Uninstall the Integration

  1. On the integrated system, disable any settings that send events to BigPanda or receive shared data from BigPanda.
  2. If the integration is agent-based, uninstall the agent from the server.

Delete the Integration from BigPanda

  1. In BigPanda, navigate to the Integrations tab and select the desired integration from the list.
  2. In the integration details on the right of the page, click the trash icon, then confirm you want to delete the integration. The integration will then be removed immediately.

👍

️Alert Resolution for Deleted Integrations

All active alerts from the integration will be resolved after deletion.

️Data Removal

This procedure does not remove any data from the integrated system.

The Agent-Based Zabbix Integration

🚧

The Agent-based integration is no longer the desired configuration approach but will still be supported for customers who have yet to migrate to the webhook version.

When using the agent-based integration, BigPanda Agent queries for all new Zabbix trigger events in real-time via the Zabbix API. To ensure a full sync of all current alerts, the BigPanda Agent also queries for all active triggers on each monitored host. The query intervals are configurable.

Limitations

  • The agent creates a dedicated user and user group in Zabbix for querying the Zabbix data. The user type affects how BigPanda is granted access to your monitoring data. For more information about how to grant the desired access, see Configure Zabbix Permissions..
  • To stay within the memory limit of the Zabbix API, the agent uses batching during the full sync. For a successful full sync, your Zabbix configuration must remain within these guidelines:
    • Less than 100,000 host groups.
    • Less than 100,000 monitored hosts per host group.
    • Less than 100,000 active triggers per monitored host.

📘

The success of the full sync does not affect the continuous sync, which ensures that all new trigger events are received and correlated accurately in BigPanda.

Configure the Integration

By editing the config file for the agent, you can configure settings for how the BigPanda agent queries Zabbix for events. You can configure the batch size for the incremental sync and the query interval and batch size for the full sync. You can also disable the full sync.

Before You Start

Obtain access to the server where the BigPanda agent is installed.

Configure The Incremental Sync

The agent queries for all new Zabbix trigger events in real-time via the Zabbix API. You may want to configure this incremental sync to reduce the load on your Zabbix server, depending on the volume of events Zabbix generates.

  1. Open the config file, which is located at /etc/bigpanda/bigpanda.conf.
  2. Locate the settings for the Zabbix incremental sync, named zabbix/state.
  3. Change the value of the max_event_batch_size to your preferred batch size.
    The batch size is the maximum number of events that the Zabbix API can return at a time during an incremental sync.
    The default value is 100.

Configure the Full Sync

To ensure a full sync of all current alerts, the BigPanda agent queries for all Zabbix events at regular intervals. You may want to configure the full sync to reduce the load on your Zabbix server, depending on the number of monitored triggers you have configured.

  1. Open the config file, which is located at /etc/bigpanda/bigpanda.conf.
  2. Locate the settings for the Zabbix full sync, named zabbix/fullsync.
  3. Change the following settings to your preferred values.
SettingDescriptionDefault Value
full_sync_intervalInterval between full syncs, in seconds.900 (15 minutes)
max_trigger_batch_sizeMaximum number of triggers that the Zabbix API can return at a time during a full sync.50

Disable the Full Sync

📘

The full sync acts as a backup to the ongoing incremental sync to ensure accuracy. Disable this feature only if you need to reduce the load on your Zabbix server.

  1. Open the config file, which is located at /etc/bigpanda/bigpanda.conf.
  2. Locate the settings for the Zabbix full sync, named zabbix/fullsync
  3. Add the following key and value: "enabled": false.

❗️

Make sure you add the pair outside of the config property.

"name": "zabbix/fullsync", 
"enabled": false, 
"config": {
  "user": "BigPanda",...
  },

Troubleshoot the Agent-Based Zabbix Integration

Follow these steps to diagnose and solve common problems with the Zabbix integration. When troubleshooting, you can use the agent log files, which are located at /var/log/bigpanda/agent.log by default.

Problem: BigPanda agent cannot connect to Zabbix API

Symptoms & Diagnosis
This error appears in the BigPanda agent log:
Error while logging into Zabbix API

Possible Causes
The log may contain further details about the possible cause for this error. Possible causes include:

  • The agent does not have the correct credentials for the BigPanda user in Zabbix (Log file contains: Invalid params., Login name or password is incorrect.)
  • The agent does not have the correct Zabbix host credentials (Log file contains: 404 Client Error: Not Found).

Solutions
Verify that the BigPanda user credentials and the Zabbix host credentials are correct.

  1. Open the agent configuration file, which is located at etc/bigpanda/bigpanda.conf.
  2. Locate the configuration settings for the Zabbix plugin.
  3. Verify that the credentials for the BigPanda user in Zabbix are correct.
  • The user and password values should match the credentials for the BigPanda user in Zabbix. To check the credentials in Zabbix, go to Administration > Users.
  • The default values are user: bigpanda, password: <App Key>.
  1. Verify that the credentials for the Zabbix host are correct.
  • The host should match the Zabbix host address.
  • The default value is host: <http://localhost/zabbix.>

Problem: Specific Zabbix alerts are not showing up in BigPanda

Symptoms & Diagnosis
Zabbix is displaying an event for a trigger in PROBLEM state, and the alert is not showing up in BigPanda.

Possible Causes
Several causes may lead to the issue. Please follow the solutions section below.

Solutions

  1. Check the agent log for errors.
  2. If the log shows the error Error while logging into Zabbix API, try the solutions in Problem: BigPanda agent cannot connect to Zabbix API. If the error cannot be resolved, contact BigPanda support with the recommended troubleshooting information.
  3. If the agent log does not show errors, verify that the BigPanda user in Zabbix has permissions to the host group of the host that is generating the alert. See Configure Zabbix Permissions.
  4. If the problem is still not resolved, contact BigPanda support with the recommended troubleshooting information.

Contact BigPanda Support About the Zabbix Integration

When contacting BigPanda support, please provide the following information:

  1. BigPanda agent.log The default location is: /var/log/bigpanda/agent.log.
  2. Events for the Zabbix triggers: If the issue involves specific Zabbix triggers, please provide all of the events for those Zabbix triggers. Obtain the events by running the following Zabbix API calls.

❗️

Run the first call to obtain an authentication token. In the second call, use the token from the first call to replace <YOUR AUTH>. Also, replace the with the relevant Zabbix trigger.

curl -i -X POST -H 'Content-Type: application/json' -d '{"params": {"password": "zabbix", "user": "Admin"}, "jsonrpc": "2.0", "method": "user.login", "id": 1, "auth": null}' http://localhost/zabbix/api_jsonrpc.php

curl -i -X POST -H 'Content-Type: application/json' -d '{"jsonrpc": "2.0", "params": {"objectids": “<TRIGGER ID>", "output": "extend"},"method": "event.get", "id": 1, "auth": "<YOUR AUTH>"}' http://localhost/zabbix/api_jsonrpc.php
  1. (Recommended) Versions of the following:
  • Zabbix
  • OS and OS Arch (32-bit or 64-bit)
  • Python >= 3.6.8

Uninstall the Agent-Based Zabbix Integration

To stop sending Zabbix events to BigPanda, you can uninstall the agent from the server or remove the Zabbix integration from the agent. Then, delete the BigPanda user and user group from Zabbix.

Before You Start

  • Modify or remove any Environments or AutoShare rules that reference the integration. All active alerts from the integration will be resolved after deletion.
  • Obtain access to the server where the BigPanda agent is installed.
  • Determine if the agent on the server supports only the Zabbix integration or supports multiple BigPanda integrations.
  • Determine the OS on the server.
  • Obtain access to the Zabbix UI with a Zabbix Super Admin user.

🚧

When replacing an existing integration with a new tool or system, we recommend configuring the new integration first to ensure no data is lost.

Uninstall the Agent

If the agent supports only the Zabbix integration, you can uninstall the agent from the server.

📘

Do not uninstall the BigPanda agent if it is supporting other BigPanda integrations. In this case, remove the Zabbix integration from the agent.

Remove the Zabbix Integration from the Agent

If the agent is supporting multiple integrations, you can remove only the specific integration without disabling the other BigPanda integrations that the agent supports.

  1. Stop the BigPanda agent by running the following command.

sudo service bigpanda stop

  1. Remove the integration from the agent configuration file by running the following command.

sudo bigpanda-config -r <INTEGRATION APP KEY>

❗️

Replace the app key with the corresponding value in BigPanda.

  1. Start the BigPanda agent by running the following command.

sudo service bigpanda start

Delete the BigPanda User and User Group from Zabbix

  1. In the Zabbix UI, go to Administration > Users.
  2. Locate the BigPanda Service user group and the Bigpanda user.
  3. Delete the user group and user.

Delete the Integration from BigPanda

  1. In BigPanda, navigate to the Integrations tab and select the desired integration from the list.
  2. In the integration details on the right of the page, click the trash icon, then confirm you want to delete the integration. The integration will then be removed immediately.

👍

️Alert Resolution for Deleted Integrations

All active alerts from the integration will be resolved after deletion.

️Data Removal

This procedure does not remove any data from the integrated system.