Zabbix

Zabbix provides an open-source solution for network and application monitoring of millions of metrics. Install this integration to correlate noisy trigger alerts from Zabbix into actionable incidents in BigPanda so you can find and remediate problems in your infrastructure quicker and easier.

Supported Versions:
2.2, 3.0, 3.2

Type:
Agent

Key Features

  • Keeps you up-to-date on the state of triggers by correlating the Zabbix events for each trigger into easy-to-track alerts.
  • Correlates alerts for triggers in the PROBLEM state into actionable incidents in BigPanda.

Prerequisites

To ensure a successful integration of Zabbix with BigPanda, the follow requirements need to be established first.

  1. A Zabbix user with Read access to all of the host groups that you want to monitor in BigPanda. Please see the Configuring Zabbix Permissions section for more information.
  2. Root or sudo permissions to install the BigPanda agent on a Linux server that is on the same network or can access the Zabbix API.
  3. The Zabbix server must be able to communicate with the BigPanda server via https (port 443).

How It Works

  • The agent queries for all new Zabbix trigger events in real-time via the Zabbix API.
  • To ensure a full sync of all current alerts, the agent also queries for all active triggers on each monitored host. The query intervals are configurable. See Customizing Zabbix..

Disabling Zabbix Triggers

Triggers that are disabled in Zabbix are not monitored by BigPanda. Therefore, if a trigger is disabled when an alert is open, the alert will not close automatically and the BigPanda incident will remain open. In this case, you must manually resolve the BigPanda incident.
A recommended good practice is for Zabbix administrators to check that all alerts are resolved in BigPanda before disabling the related trigger.

Limitations

  • The agent creates a dedicated user and user group in Zabbix for querying the Zabbix data. The user type affects how BigPanda is granted access to your monitoring data. For more information about how to grant the desired access, see Configuring Zabbix Permissions..
  • To stay within the memory limit of the Zabbix API, the agent uses batching during the full sync. For a successful full sync, your Zabbix configuration must remain within these guidelines:
    • Less than 100,000 host groups.
    • Less than 100,000 monitored hosts per host group.
    • Less than 100,000 active triggers per monitored host.

The success of the full sync does not affect the continuous sync, which ensures that all new trigger events are received and correlated accurately in BigPanda.

Installing The Integration

Administrators can install the integration by following the on-screen instructions in BigPanda. For more information, see Installing an Integration.

Zabbix Data Model

BigPanda normalizes alert data from Zabbix into tags. You can use tag values to filter the incident feed and to define filter conditions for Environments. The primary and secondary properties are also used during the correlation process.

Standard Tags

Tag
Description
Attributes

hosts

Names of the Zabbix hosts that the trigger belongs to.

items

Zabbix items that are contained in the trigger. Zabbix items are individual metrics defined for Zabbix hosts. For example:

  • net.tcp.service[smtp,,25].last

  • ntp.client.offset.max

  • disk.io.status[sda].sum

Second Property.

item_values

Items and their corresponding values. The items match the list in the items tag. For example:

  • net.tcp.service[smtp,,25].last: 1

  • ntp.client.offset.max: 0.5258

  • disk.io.status[sda].sum: 86.0000%

trigger

Name of the trigger in Zabbix.

trigger_description

Description of the trigger in Zabbix.

maintenance

Flag indicating if the host is under maintenance. This tag is included only when the value is true.

expression

Trigger expression configured in Zabbix to calculate the trigger state.

acknowledged

Flag indicating if the latest event for the trigger has been acknowledged in Zabbix.

rum book_url

URL configured in Zabbix as a link associated with the trigger.

Configuring Zabbix Permissions

Connecting BigPanda with Zabbix requires a new Zabbix user with read access to all the host groups that you want to monitor in BigPanda. During configuration, you can choose the level of access for this user—Zabbix Super Admin or Zabbix Admin—and the user is created automatically with the user name Bigpanda in a new user group called BigPanda Service.

Granting the user Zabbix Super Admin permissions is the easiest way to ensure BigPanda has the required access. If granting this level of access is not an option, you can choose Zabbix Admin. The user will be given permissions to all existing host groups at the time of the configuration and you must manually grant permissions to any host groups added after the initial configuration. You can view and change the type of user in Zabbix at any time.

Prerequisites

Obtain access to the Zabbix UI with a Zabbix Super Admin user.

Granting BigPanda Access To Host Groups In Zabbix

If the Bigpanda user has Zabbix Super Admin permissions, then BigPanda has access to relevant data for all host groups in Zabbix without additional configuration. If the Bigpanda user has Zabbix Admin permissions, then any access to host groups must be explicitly granted in Zabbix by configuring permissions for the BigPanda Service user group.
To manually grant BigPanda access to host groups in Zabbix:

  1. In Zabbix, go to Administration > Users > BigPanda Service > Permissions. The Zabbix permissions for the BigPanda Service user group are displayed.
  2. Under Composing Permissions > Read only, click Add and then select the host groups.

Host groups that appear under Calculated Permissions > Deny are not accessible to the Bigpanda user, so events for these host groups will not be sent to BigPanda. If you want to send events for these host groups to BigPanda, add them to the Read-Only permissions, as described in step 2, and they will be automatically removed from the Deny list.

  1. At the bottom of the screen, click Update.

Changing the BigPanda Access Level in Zabbix

1.In Zabbix, go to Administration > Users.

  1. Under Members, click the Bigpanda user.
  2. Click Permissions.
  3. In the User type field, select Zabbix Super Admin or Zabbix Admin to change the access level.
  4. At the bottom of the screen, click Update.

Customizing Zabbix

By editing the config file for the agent, you can configure settings for how the BigPanda agent queries Zabbix for events. You can configure the batch size for the incremental sync and the query interval and batch size for the full sync. You can also disable the full sync.

Prerequisites

Obtain access to the server where the BigPanda agent is installed.

Configuring The Incremental Sync

The agent queries for all new Zabbix trigger events in real-time via the Zabbix API. You may want to configure this incremental sync to reduce the load on your Zabbix server, depending on the volume of events Zabbix generates.

  1. Open the config file, which is located at /etc/bigpanda/bigpanda.conf.
  2. Locate the settings for the Zabbix incremental sync, named zabbix/state.
  3. Change the value of the max_event_batch_size to your preferred batch size.
    The batch size is the maximum number of events that the Zabbix API can return at a time during an incremental sync.
    The default value is 100.

Configuring the Full Sync

To ensure a full sync of all current alerts, the BigPanda agent queries for all Zabbix events at regular intervals. You may want to configure the full sync to reduce the load on your Zabbix server, depending on the number of monitored triggers you have configured.

  1. Open the config file, which is located at /etc/bigpanda/bigpanda.conf.
  2. Locate the settings for the Zabbix full sync, named zabbix/fullsync.
  3. Change the following settings to your preferred values.
Setting
Description
Default Value

full_sync_interval

Interval between full syncs, in seconds.

900 (15 minutes)

max_trigger_batch_size

Maximum number of triggers that the Zabbix API can return at a time during a full sync.

50

Disabling The Full Sync

The full sync acts as a backup to the ongoing incremental sync to ensure accuracy. Disable this feature only if you need to reduce the load on your Zabbix server.

  1. Open the config file, which is located at /etc/bigpanda/bigpanda.conf.
  2. Locate the settings for the Zabbix full sync, named zabbix/fullsync
  3. Add the following key and value: "enabled": false.

Make sure you add the pair outside of the config property.

"name": "zabbix/fullsync", 
"enabled": false, 
"config": {
  "user": "BigPanda",...
  },

Uninstalling Zabbix

To stop sending Zabbix events to BigPanda, you can uninstall the agent from the server or remove the Zabbix integration from the agent. Then, delete the BigPanda user and user group from Zabbix.

Prerequisites

  • Manually resolve any open incidents related to the integration to remove them from your incident feed. Optionally, remove any additional data in BigPanda. For example, modify or remove any Environments or AutoShare rules that reference the integration.
  • Obtain access to the server where the BigPanda agent is installed.
  • Determine if the agent on the server supports only the Zabbix integration or supports multiple BigPanda integrations.
  • Determine the OS on the server.
  • Obtain access to the Zabbix UI with a Zabbix Super Admin user.

Uninstalling the Agent

If the agent supports only the Zabbix integration, you can uninstall the agent from the server.

Do not uninstall the BigPanda agent if it is supporting other BigPanda integrations. In this case, remove the Zabbix integration from the agent.

Removing the Zabbix Integration from the Agent

If the agent is supporting multiple integrations, you can remove only the specific integration without disabling the other BigPanda integrations that the agent supports.

  1. Stop the BigPanda agent by running one of the following commands.
    • For RHEL 5 and 7, CentOS 5 and 7, Debian, and Ubuntu:
      sudo service bigpanda stop
    • For RHEL 6 and CentOS 6:
      sudo initctl stop bigpanda
  2. Remove the integration from the agent configuration file by running the following command.
    sudo bigpanda-config -r <INTEGRATION APP KEY>

Replace the app key with the corresponding value in BigPanda.

  1. Start the BigPanda agent by running one of the following commands.
    • For RHEL 5 and 7, CentOS 5 and 7, Debian, and Ubuntu:
      sudo service bigpanda start
      • For RHEL 6 and CentOS 6:
        sudo initctl start bigpanda

Deleting the BigPanda User and User Group from Zabbix

  1. In the Zabbix UI, go to Administration > Users.
  2. Locate the BigPanda Service user group and the Bigpanda user.
  3. Delete the user group and user.

Post-Requisites

Delete the integration in BigPanda to remove the Opsview integration from your UI.

Troubleshooting

Follow these steps to diagnose and solve common problems with the Zabbix integration. When troubleshooting, you can use the agent log files, which are located at /var/log/bigpanda/agent.log by default.

Problem: BigPanda agent cannot connect to Zabbix API

Symptoms & Diagnosis
This error appears in the BigPanda agent log:
Error while logging into Zabbix API

Possible Causes
The log may contain further details about the possible cause for this error. Possible causes include:

  • The agent does not have the correct credentials for the BigPanda user in Zabbix (Log file contains: Invalid params., Login name or password is incorrect.)
  • The agent does not have the correct Zabbix host credentials (Log file contains: 404 Client Error: Not Found).

Solutions
Verify that the BigPanda user credentials and the Zabbix host credentials are correct.

  1. Open the agent configuration file, which is located at etc/bigpanda/bigpanda.conf.
  2. Locate the configuration settings for the Zabbix plugin.
  3. Verify that the credentials for the BigPanda user in Zabbix are correct.
    • The user and password values should match the credentials for the BigPanda user in Zabbix. To check the credentials in Zabbix, go to Administration > Users.
    • The default values are user: bigpanda, password: <App Key>.
  4. Verify that the credentials for the Zabbix host are correct.
    • The host should match the Zabbix host address.
    • The default value is is host: http://localhost/zabbix.

Problem: Specific Zabbix alerts are not showing up in BigPanda

Symptoms & Diagnosis
Zabbix is displaying an event for a trigger in PROBLEM state, and the alert is not showing up in BigPanda.

Possible Causes
Several causes may lead to the issue. Please follow the solutions section below.

Solutions

  1. Check the agent log for errors.
  2. If the log shows the error Error while logging into Zabbix API, try the solutions in Problem: BigPanda agent cannot connect to Zabbix API. If the error cannot be resolved, contact BigPanda support with the recommended troubleshooting information.
  3. If the agent log does not show errors, verify that the BigPanda user in Zabbix has permissions to the host group of the host that is generating the alert. See Configuring Zabbix Permissions.
  4. If the problem is still not resolved, contact BigPanda support with the recommended troubleshooting information.

Contacting BigPanda Support About the Zabbix Integration

When contacting BigPanda support, please provide the following information:

  1. BigPanda agent.log The default location is: /var/log/bigpanda/agent.log.
  2. Events for the Zabbix triggers: If the issue involves specific Zabbix triggers, please provide all of the events for those Zabbix triggers. Obtain the events by running the following Zabbix API calls.

Run the first call to obtain an authentication token. In the second call, use the token from the first call to replace <YOUR AUTH>. Also, replace the <TRIGGER ID> with the relevant Zabbix trigger.

curl -i -X POST -H 'Content-Type: application/json' -d '{"params": {"password": "zabbix", "user": "Admin"}, "jsonrpc": "2.0", "method": "user.login", "id": 1, "auth": null}' http://localhost/zabbix/api_jsonrpc.php

curl -i -X POST -H 'Content-Type: application/json' -d '{"jsonrpc": "2.0", "params": {"objectids": “<TRIGGER ID>", "output": "extend"},"method": "event.get", "id": 1, "auth": "<YOUR AUTH>"}' http://localhost/zabbix/api_jsonrpc.php
  1. (Recommended) Versions of the following:
    • Zabbix
    • OS and OS Arch (23- or 64-bit)
    • Python

Zabbix


Zabbix provides an open-source solution for network and application monitoring of millions of metrics. Install this integration to correlate noisy trigger alerts from Zabbix into actionable incidents in BigPanda so you can find and remediate problems in your infrastructure quicker and easier.

Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.