Health Check Temporal Worker

BigPanda's Health Check Worker is a lightweight Docker agent that runs on your infrastructure and automates device health checks against your monitoring platform. When BigPanda's triage agent detects a device-related incident, the worker queries your monitoring system and reports the device status back to BigPanda.

The worker connects securely to BigPanda using mutual TLS (mTLS) and only makes outbound connections — no inbound ports need to be opened.

Key features

Automates device health verification during incident triage
Runs securely on your infrastructure with mTLS encryption
Requires no inbound network access
Supports Docker and Kubernetes deployments

Before you start

Make sure you have the following ready:

Docker installed on the target machine
mTLS certificates from BigPanda (provided during onboarding)
An API key for your monitoring platform with read access to device health status

Deploy the worker

Download and load the image

BigPanda provides a presigned URL for the Docker image. Download and load it:

curl -L '<PRESIGNED-URL>' -o health-check-worker.tar.gz
docker load < health-check-worker.tar.gz

Install certificates

BigPanda provides your organization's mTLS certificates during onboarding. Place them in acerts/directory:

certs/
├── client.pem       # Client certificate (identifies your organization)
├── client-key.pem   # Client private key
└── ca.cert          # BigPanda CA certificate

Certificate security

Keep client-key.pem secure. It authenticates your worker to BigPanda. If compromised, contact BigPanda immediately to rotate the certificate.

(Optional) Stop the existing worker

If you are upgrading an existing worker, stop and remove the current container:

docker stop adr-temporal-worker && docker rm adr-temporal-worker

Create a configuration file

Create an .env file with the following variables:

# --- Connection (provided by BigPanda) ---
TEMPORAL_ADDRESS=temporal.prod.bigpanda.io:7233
TEMPORAL_TLS_CERT_PATH=/certs/client.pem
TEMPORAL_TLS_KEY_PATH=/certs/client-key.pem
TEMPORAL_TLS_CA_PATH=/certs/ca.cert

# --- Secrets (configured by you) ---
DEVICE_API_KEY=<your-monitoring-platform-api-key>

Monitoring platform authentication

DEVICE_API_KEY must have read access to device health status on your monitoring platform. The worker sends it as a Bearer token in the Authorization header. If your monitoring platform does not require authentication, you can omit this variable.

Run the worker

docker run -d \
  --name adr-temporal-worker \
  --restart unless-stopped \
  --log-opt max-size=50m \
  --log-opt max-file=3 \
  -p 8080:8080 \
  -v $(pwd)/certs:/certs:ro \
  --env-file .env \
  adr-temporal-worker:latest

Verify the deployment

Check the worker logs to confirm a successful start:

docker logs -f adr-temporal-worker

You should see output similar to:

Starting Temporal Worker
Worker state changed { state: 'RUNNING' }
Worker started
Health server listening

Then verify the health endpoints:

curl http://localhost:8080/health    # Returns: {"status":"ok","startedAt":"..."}
curl http://localhost:8080/ready     # Returns: {"status":"ready"}

Health Check process

The worker connects outbound to BigPanda and waits for workflow instructions. When BigPanda's triage agent detects a device-related incident, it triggers the health check workflow for your worker.

BigPanda Cloud                        Your Network
──────────────                        ────────────

Triage Agent detects                  Worker (this service)
"device unreachable"                      │
        │                                 │ Connects to BigPanda
        ▼                                 │ via mTLS (outbound only)
Temporal Server                           │
        │                                 │
        └── Routes workflow ─────────────►│
            to your worker                │
                                          ▼
                                   Calls your monitoring API
                                   GET {healthCheckUrl}/{deviceName}
                                   Auth: Bearer <DEVICE_API_KEY>
                                          │
                                          ▼
                                   Returns result to BigPanda
                                   (UP / DOWN / NOT_SUPPORTED)

You don't need to configure or schedule workflows. BigPanda manages this automatically. The worker just needs to be running and connected.

Environment variables

Connection configuration

Primary connection configuration is provided by the Bigpanda team.

Variable	Description	Example
`TEMPORAL_ADDRESS`	BigPanda server address	`temporal.prod.bigpanda.io:7233`
`TEMPORAL_TLS_CERT_PATH`	Path to mTLS client certificate	`/certs/client.pem`
`TEMPORAL_TLS_KEY_PATH`	Path to mTLS client private key	`/certs/client-key.pem`
`TEMPORAL_TLS_CA_PATH`	Path to BigPanda CA certificate	`/certs/ca.cert`

Secrets

Secrets are configured by you, following the procedure for the connected monitoring platform.

Variable	Description	Required
`DEVICE_API_KEY`	API key for your monitoring platform. Sent as a `Bearer` token in the `Authorization` header.	Only if your monitoring platform requires authentication

Optional configuration

Variable	Default	Description
`METRICS_PORT`	`8080`	Port for health endpoints
`LOG_LEVEL`	`info`	Log level (`debug`,`info`,`warn`,`error`)

Health and Monitoring endpoints

Endpoint	Response	Purpose
`GET /health`	`{"status":"ok","startedAt":"..."}`	Health check — returns 200 if the worker is running
`GET /ready`	`{"status":"ready"}`	Readiness check — returns 200 if the worker is initialized

Use these endpoints for container orchestration health probes such as Docker healthcheck or Kubernetes liveness and readiness probes.

Network Requirements

Direction	Destination	Port	Protocol	Purpose
Outbound	`temporal.prod.bigpanda.io`	`7233`	gRPC + mTLS	BigPanda server connection
Outbound	Your monitoring platform	Varies	HTTPS	Device health checks
Inbound	None required	—	—	—

Firewall configuration

No inbound ports need to be opened. The worker only makes outbound connections.

Kubernetes Deployment

If you are deploying to Kubernetes, use the following manifests as a starting point.

Create secrets for your certificates and monitoring API key:

apiVersion: v1
kind: Secret
metadata:
  name: adr-temporal-worker-secrets
  namespace: bigpanda
type: Opaque
stringData:
  DEVICE_API_KEY: "<your-monitoring-platform-api-key>"
---
apiVersion: v1
kind: Secret
metadata:
  name: adr-temporal-worker-certs
  namespace: bigpanda
type: Opaque
data:
  client.pem: <base64-encoded-cert>
  client-key.pem: <base64-encoded-key>
  ca.cert: <base64-encoded-ca>

Then create the deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: adr-temporal-worker
  namespace: bigpanda
spec:
  replicas: 1
  selector:
    matchLabels:
      app: adr-temporal-worker
  template:
    metadata:
      labels:
        app: adr-temporal-worker
    spec:
      containers:
        - name: worker
          image: adr-temporal-worker:latest
          ports:
            - containerPort: 8080
          envFrom:
            - secretRef:
                name: adr-temporal-worker-secrets
          env:
            - name: TEMPORAL_ADDRESS
              value: "temporal.prod.bigpanda.io:7233"
            - name: TEMPORAL_TLS_CERT_PATH
              value: "/certs/client.pem"
            - name: TEMPORAL_TLS_KEY_PATH
              value: "/certs/client-key.pem"
            - name: TEMPORAL_TLS_CA_PATH
              value: "/certs/ca.cert"
          volumeMounts:
            - name: certs
              mountPath: /certs
              readOnly: true
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 30
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 15
      volumes:
        - name: certs
          secret:
            secretName: adr-temporal-worker-certs

Troubleshooting

Symptom	Likely Cause	Solution
Worker fails to start with an ENOENT error	Certificate files not found	Verify that cert files exist at the paths specified in `TEMPORAL_TLS_CERT_PATH`, `TEMPORAL_TLS_KEY_PATH`, and `TEMPORAL_TLS_CA_PATH`.
TLS handshake failed in logs	Invalid, expired, or incorrectly signed certificates	Contact BigPanda to verify or rotate your certificates.
Worker crashes with DEADLINE_EXCEEDED or a connection error	Cannot reach BigPanda	Verify that your firewall or proxy allows outbound traffic to the address specified in `TEMPORAL_ADDRESS`.
Health endpoint not responding	Port not exposed or blocked	Verify that port 8080 is exposed (`-p 8080:8080`) and not blocked by a local firewall.
Device check returns DOWN for all devices	Invalid or missing monitoring API key	Verify that `DEVICE_API_KEY` has read access to device health status. Test the API manually: `curl -H "Authorization: Bearer <key>" https://your-monitor/api/device-health/<device>`
Device check times out (60 seconds)	Monitoring platform unreachable from worker	Verify that the worker can reach your monitoring platform. Check DNS resolution and firewall rules.
Container exits immediately	Missing required environment variables	Verify that all `TEMPORAL_*` variables are set in your `.env` file. Run `docker logs adr-temporal-worker` to identify the specific missing variable.
Container restarts repeatedly	Persistent connection or authentication failure	Run `docker logs adr-temporal-worker` to identify the error, then refer to the relevant row above.

Useful Commands

docker logs -f adr-temporal-worker          # View live logs
docker logs --tail 100 adr-temporal-worker  # View last 100 lines
docker restart adr-temporal-worker          # Restart the worker
docker stop adr-temporal-worker             # Stop the worker
docker rm adr-temporal-worker               # Remove the worker
curl http://localhost:8080/health           # Check health status
curl http://localhost:8080/ready            # Check readiness status

Security

All communication with BigPanda uses mTLS with per-organization certificates.
No inbound network access is required. The worker only makes outbound connections.
Your monitoring platform credentials (DEVICE_API_KEY) stay on your infrastructure and are never sent to BigPanda.
The worker only executes registered workflows. No arbitrary code execution is possible.

Next Steps

If you run into any issues or need help with your deployment, contact BigPanda support at [email protected] with:

Your worker version (docker images | grep adr-temporal-worker)
Relevant logs (docker logs --tail 200 adr-temporal-worker)
Steps to reproduce the issue

In this section: