DocsAPI ReferenceRelease Notes
Docs
BigPanda AppRequest DemoBigPanda UniversityBigPanda Community

BigPanda Agent Advanced Configuration

Troubleshoot, migrate, and configure advanced settings for the BigPanda Agent.

Suggest Edits

The BigPanda agent is a low-footprint utility installed on the master host of your monitoring tool. It consumes alerts from the tool in real-time and then transmits them to BigPanda via TLS-encrypted HTTP calls.

❗️

Python Compatibility

To use the Open Integration Manager for Agent-based integrations, Python version>= 3.8.0 and an Agent version >= 10.2 are required.

Available Configuration Options

The generic BigPanda agent is available for configuration on the below systems:

Integrations that Require the BigPanda Agent

The following integrations leverage the BigPanda Agent:

See the specific integration instructions for details on installation and configuration.

Update the BigPanda Agent

You can upgrade the BigPanda Agent to access the features and fixes available in a newer version.

Process

  1. Stop the BigPanda agent by running the following command.

    sudo service bigpanda stop

  2. Upgrade the agent by running one of the following command.

    sudo yum update
sudo yum list --showduplicates bigpanda-agent | expand
sudo yum install bigpanda-agent-<version-number>

    sudo apt update
sudo apt install bigpanda-agent

  3. Make sure that the BigPanda Agent has started by running the following command.

    sudo service bigpanda start

  4. Verify the BigPanda Agent is loaded by running the following command.

    sudo systemctl status bigpanda

  5. Verify that the Agent version has been upgraded by running the following command.

    sudo grep version /etc/bigpanda/bigpanda.conf

Migrate the BigPanda Agent to v10.x or Higher

The BigPanda Agent has been updated to route alert traffic to the Open Integration Manager. Use the steps below to update from a v9.x release to a v10.x release.

📘

Minimum Versions

To use the Open Integration Manager for Agent-based integrations, Python version >= 3.8.0 and an Agent version >= 10.2 are required.

For RHEL 7/8 based OS, Python version 3.8 is required to run the Agent version >= 10.2, and is installed during the migration process. For RHEL 9, the system version of Python 3.9 is used.

Prepare to Migrate the Agent

Before migrating the Agent, take the following steps:

  • Ensure a backup has been made of the agent config file. This will serve as a copy should you need to roll back the upgrade. 
    sudo cp /etc/bigpanda/agent/bigpanda.conf /etc/bigpanda/agent/bigpanda.conf.tmp
  • Ensure the user doing the upgrade has been added to the sudoers list. Do not upgrade from the root account.
  • If there are multiple versions of python3 installed, install a python3 alternative to use Python 3.8 or later. 
    sudo update-alternatives --install /usr/bin/python3 python3 $(which python3.8) 1

    sudo alternatives --install /usr/bin/python3 python3 $(which python3.8) 1

Migration Process

The migration should happen seamlessly during the update process. Once the new version is installed, review the agent config file to ensure the new information is accurate.

v10.x config example

{
  "root": {
    "version": "10.2.0",
    "agent_id": "<agent_id>",
    "data_encoding": "utf-8",
    "backend": {
      "alert_address": "https://integrations.bigpanda.io",
      "backend_address": "https://api.bigpanda.io",
      "api_token": "<api_token>",
      "timeout": 30,
      "verify_certificate": true
    },
    "plugins": [
      {
        "name": "agent/heartbeat",
        "endpoint": "/agents/<agent_id>/heartbeat",
        "action": "POST",
        "config": {
          "interval": 60
        }
      },
      {
        "name": "rawalertstransfer",
        "app_key": "<app_key>",
        "action": "POST",
        "endpoint": "/oim/api/alerts",
        "batch_size": 500,
        "enabled": true,
        "config": {
          "queue_directory": "/var/lib/bigpanda/queue",
          "dlq_directory": "/var/lib/bigpanda/dlq",
          "app_key": "<app_key>"
        }
      }
    ]
  }
}

The following fields were updated in the v10.x config:

  • alert_address has been added and should point to the Open Integration Manager.
  • backend_address has been added, replacing address, and should point to the BigPanda API backend URL.
  • The endpoint for each plugin has been updated to point to the /oim/api/alerts endpoint as needed.
Using Alternative Python Versions

The BigPanda Agent version 10.x and later requires Python version >= 3.8. The BigPanda Agent installer sideloads Python 3.8 as a Python alternative. This will install the bigpanda module and all Python packages in /usr/lib/python3.8/site-packages/.

To use an alternative version of python3, follow the guide below to install the Agent dependencies in the correct location.

Alternate loading for Python 3.9 as an example:

  1. Switch the alternative Python version back to 3.9.

    sudo alternatives --config python3

There are 3 programs which provide 'python3'.

  Selection    Command
-----------------------------------------------
*  1           /usr/bin/python3.6
   2           /usr/bin/python3.9
 + 3           /usr/bin/python3.8

Enter to keep the current selection[+], or type selection number: 2

  2. Validate the version of Python was switched to Python 3.9.

    python3 --version

  3. Ensure the package requirements are installed in /usr/lib/python3.9/site-packages/. Site locations can be verified with python3 -m site.

    python3 -m pip install -r /etc/bigpanda/agent/requirements.txt

  4. Copy the bigpanda python module to the new /lib location.

    sudo cp -R /usr/lib/python3.8/site-packages/bigpanda/ /usr/lib/python3.9/site-packages/

Once all of the packages are installed in the Python 3.9 /site-packages directory, you can start the bigpanda agent as normal.Once all of the packages are installed in the Python 3.9 /site-packages directory, you can start the bigpanda agent as normal.

Start the BigPanda Agent

Once the BigPanda Agent config has been verified as accurate, you can start the BigPanda agent.

sudo service bigpanda start

Make sure to verify that the agent has started correctly and is receiving heartbeat responses from BigPanda.

tail -50 /var/log/bigpanda/agent.log

Configure the BigPanda Agent with a DMZ

You can set up the BigPanda Agent to communicate with a monitoring server that is behind your corporate Demilitarized Zone (DMZ). First, configure the monitoring server to send alerts to a server in your DMZ. Then, set up SSH keys to allow the monitoring server to communicate with the DMZ server. Finally, configure the DMZ server to integrate with BigPanda.

The following example demonstrates one method for implementing this setup on CentOS 7 using rynsc to transfer and synchronize files between the monitoring and the DMZ servers.

Configuring the BigPanda Agent with a DMZ

Configuring the BigPanda Agent with a DMZ

Before You Start

  • Administrator access to both the monitoring and the DMZ servers.
  • Set up permissions to execute the rsync ssh commands, granted either through RSA keys or in the rsyncd.conf file.
  • Administrator access to BigPanda.

Configure the Monitoring Server

  1. On the monitoring server, install the BigPanda agent by following the install instructions.

  2. Configure a scheduled task to run the following rsync command every minute.

    rsync -a -e "ssh" --remove-source-files /var/lib/bigpanda/queue/ $DMZ_servername:/var/lib/bigpanda/queue

📘

Adjust Code to Fit

Be sure to replace $DMZ_servername with the network name of your DMZ server. The /var/lib/bigpanda/queue directory will be created when you install the BigPanda agent on the DMZ server.

You can use Windows Scheduled Task facility or a cron job. For example, follow these steps to create a cron job that runs as the root user.

  1. Open a crontab file by running the following command.

    cron -u root -e

  2. Add the following line in the crontab file.

    ***** root /usr/bin/rsync -a -e "ssh" --remove-source-files /var/lib/bigpanda/queue/ $DMZ_servername:/var/lib/bigpanda/queue

Set up SSH Keys

  1. On the monitoring server, generate a public SSH key with no password by running the following commands.

    ssh-keygen -f ~/.ssh/id_rsa -q -P ""
cat ~/.ssh/id_rsa.pub

  2. Copy the key to your clipboard and log in to your DMZ server.

  3. If the SSH folder does not exist on the DMZ server, create it manually by running the following commands.

    mkdir ~/.ssh
chmod 0700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 0644 ~/.ssh/authorized_keys

  4. In the ~/.ssh/authorized_keys file, add the SSH key that you copied in Step 2.

Configure the DMZ Server

  1. On the DMZ server, install the BigPanda agent by following the install instructions.

Debug the Proxy Configuration

Connectivity issues are often manifested as connection errors inside the agent log file. You are able to debug the root cause of connectivity issues when configuring the agent behind a proxy.

Before you start

Ensure that you have:

  • Agent version 8.3.0 and above.
  • Python version 3.6.8 and above.
  • python3-pip.

❗️

Additional Requirements for <=10.2.0

Python 3.8 is required for Agent version 10.2.0 and above

Process

  1. SSH to the server on which the agent is installed.

  2. Run the following command and save the output to a file.

    bigpanda-check-proxy --proxy <proxy_url>

  3. Run the following command and save the output to another file. curl -v --proxy <proxy_url> "https://api.bigpanda.io/health/data"

    curl -v --proxy <proxy_url> "<https://api.bigpanda.io/health/data">

  4. Send the output from both commands to BigPanda support. We will work with you to troubleshoot common connectivity issues.

Uninstall the BigPanda Agent

To fully remove an agent-based integration, you must remove the BigPanda Agent from the associated server.

Uninstall the Agent from RedHat Enterprise Linux (RHEL) and CentOS

  1. Connect to the server where you want to uninstall the BigPanda Agent package. You must have root privileges.

  2. Run the following commands.

    sudo yum remove bigpanda-agent
sudo rm /etc/yum.repos.d/bigpanda.repo

Uninstall the Agent from Debian and Ubuntu

  1. Connect to the server where you want to uninstall the BigPanda Agent package. You must have root privileges.

  2. Run the following commands:

    sudo apt-get purge bigpanda-agent
sudo rm /etc/apt/sources.list.d/bigpanda.list
sudo apt-get update

Uninstall The Agent Manually

❗️

Does Not Remove from Package Manager's Database

This method does not remove the package from your package manager's database. Consider using this information to verify that the Agent files are completely removed.

  1. Stop the BigPanda service by running the following command:

    sudo service bigpanda stop

  2. Remove the BigPanda python module from the Python3 library. This library is usually located in /usr/lib/python<VERSION>/site-packages/bigpanda or /usr/lib/python<VERSION>/dist-packages/bigpanda.

    You can verify site package locations with the following commands:

    $ python3 -m site
$ pip3 list -v

  3. Remove the following files:

    • /usr/bin/bigpanda-agent/
    • usr/bin/bigpanda-config
    • etc/init/bigpanda.conf
    • etc/init.d/bigpanda
    • /etc/bigpanda
    • var/log/bigpanda
    • var/lib/bigpanda
    • var/run/bigpanda

    📘

    Find Command

    Use the find command to help locate files: $ sudo find / -name bigpanda*

  4. Remove the BigPanda user and group by running the following commands:

    sudo userdel -r bigpanda
sudo groupdel bigpandav

FAQs

Where can I find the agent log files?

By default, the log files are located at /var/log/bigpanda/. BigPanda log files are rotated when they reach 10MB and only 3 versions are stored at any given time. To change these settings, see Configure the BigPanda Agent Logs.

Is the BigPanda agent secure?

Yes, we’ve designed the agent to be secure and non-intrusive. The installer uses signed deb/rpm packages. The agent itself runs as a custom non-root user and it communicates with BigPanda via SSL.

How do I install multiple BigPanda agents?

In most cases, you need only one agent per master server of your integration. If you have multiple masters, you need to create a separate integration, using different app keys, for each master.

What happens if my internet connection is down?

BigPanda does not miss alerts, even if your connection is down. The BigPanda agent intelligently retries sending your alerts, until it’s successful. Once the connection is revived, you will see the events in BigPanda.

Can I use the same agent on multiple integrations?

No, each agent is assigned a unique "agent_id" value (found in the /etc/bigpanda/bigpanda.config file) and can only support one integration.

Next Steps

Find details about the BigPanda Agent Advanced Configuration

Learn to Configure the BigPanda Agent Log

Learn how to Integrate with BigPanda

Updated 7 days ago