Agent Check: celery

Overview

This check monitors Celery through the Datadog Agent. Celery is a distributed task queue system that enables asynchronous task processing in Python applications.

The Celery integration provides valuable insights into your task queue system by:

• Monitoring worker health, status, and task execution metrics
• Tracking task processing rates, runtime, and prefetch times
• Providing visibility into worker performance and task distribution
• Helping identify bottlenecks and optimize task processing efficiency

Setup

Follow the instructions below to install and configure this check for an Agent running on a host. For containerized environments, see the Autodiscovery Integration Templates for guidance on applying these instructions.

Installation

The Celery check is included in the Datadog Agent package. No additional installation is needed on your server.

Prerequisites

1. Install and configure Celery Flower, the real-time web monitor and administration tool for Celery.

Configuration

1. Edit the celery.d/conf.yaml file in the conf.d/ folder at the root of your Agent's configuration directory to start collecting your Celery performance data. See the sample celery.d/conf.yaml for all available configuration options.

   init_config:
   
   instances:
     ## @param openmetrics_endpoint - string - required
     ## Endpoint exposing the Celery Flower's Prometheus metrics
     #
     - openmetrics_endpoint: http://localhost:5555/metrics

2. Restart the Agent.

Validation

Run the Agent's status subcommand and look for celery under the Checks section.

Data Collected

Metrics

See metadata.csv for a complete list of metrics provided by this integration.

Events

The Celery integration does not include any events.

Service Checks

The Celery integration includes the following service check:

See service_checks.json for a list of service checks provided by this integration.

Troubleshooting

Need help? Contact Datadog support.