Celery

功能

• Distributed task execution and scheduling
• Asynchronous background job processing
• Complex task workflows and orchestration
• Robust error handling with retries and backoff
• Integration with Python web frameworks (Django, Flask, FastAPI)

使用场景

• Offloading long-running operations like email sending or report generation
• Implementing scheduled tasks similar to cron jobs
• Distributing computation across multiple worker nodes
• Building complex asynchronous workflows with task dependencies

非目标

• Replacing simple asyncio for in-process async I/O
• Providing real-time request/response handling
• Serving as a solution for minimal infrastructure needs

工作流

1. Install Celery and necessary dependencies.
2. Configure the Celery application, including broker and backend.
3. Define tasks using decorators or Task classes.
4. Execute tasks asynchronously using `.delay()` or `.apply_async()`.
5. Monitor task execution via workers, Flower, or inspection commands.
6. Implement advanced features like periodic tasks, workflows, and error handling.

实践

• Task design
• Error handling
• Performance optimization
• Monitoring
• Security

先决条件

• Python environment
• Celery package (`pip install celery`)
• A message broker (e.g., Redis, RabbitMQ)
• An optional result backend (e.g., Redis, database)