Skip to content

Tasks

Lime Task provides the functionality to run asynchronous tasks, declare/create and start a task as well as asking for its status.

In order to gain the best performance it's a good practice to base your solutions on tasks. Any action that takes longer than a normal REST request should be realized as a task. Same good use cases are e.g. bulk updates, heavy processing, reporting or integrations.

In comparison to a REST request the execution is handled asynchronously. That means, that whenever a task is started, it's put in a queue and waits for an available worker to be executed. For that reason Lime Task can't promise any execution times just that the task is eventually taken care of. During that time a task owner can ask for the current status, which might be PENDING, STARTED, SUCCESS or FAILURE. Whenever the execution of a task is completed, the status object can also contain a result.

How Task Execution Works

Tasks Are Enqueued, Not Executed Directly

When you call send_task(), the task is only added to a queue - it does not execute immediately. Execution happens later when an available worker picks up the task from the queue.

Lime Task uses a message queue architecture built on Celery and RabbitMQ. Understanding this architecture is crucial for working effectively with tasks:

graph LR
    A[Your Code] -->|send_task| B[RabbitMQ Queue]
    B --> C{Available Worker?}
    C -->|Yes| D[Worker Executes Task]
    C -->|No| E[Task Waits in Queue]
    E --> C
    D --> F[Task Complete]

Key Concepts

Enqueuing vs Execution

When you send a task, you're placing an order - not cooking the meal:

  • send_task() adds the task to a queue and returns immediately
  • Execution happens when a worker picks up the task from the queue
  • There is no guarantee about when execution will start

Worker Pool

Tasks are executed by a pool of workers (the Task Handler service):

  • Workers continuously monitor queues for tasks to execute
  • When a worker is available, it picks up the next task from the queue
  • If all workers are busy, new tasks wait in the queue
  • The number of workers affects how quickly tasks can be processed

No Guarantees About When Execution Starts

When a task will begin executing depends on several factors, including, but not limited to:

  • Queue length: How many tasks are waiting ahead of yours
  • Worker availability: Whether workers are free or busy with other tasks
  • Task duration: How long currently executing tasks take to complete
  • Worker capacity: Total number of workers configured for your environment

A task might execute immediately if workers are idle, or wait minutes/hours if workers are saturated with long-running tasks.

Task Status Lifecycle

Understanding task status helps you monitor execution:

  • PENDING: Task has been sent but not yet received by any worker
  • RECEIVED: A worker has received the task from the queue
  • STARTED: The worker has begun executing the task code
  • SUCCESS: Task completed successfully
  • FAILURE: Task failed during execution
  • REVOKED: Task was cancelled before execution
  • RETRY: Task is being retried after a failure

The time between send_task() and STARTED status is entirely dependent on worker availability - there's no way to predict or guarantee when a task will begin execution.

Generate the tasks module

To add the tasks module to your package, change directory to your package directory and run this command:

lime-project generate tasks

If you're working on Linux, make sure your package is installed on the taskhandler container.

Local Development: Use Threads Mode

When running the Task Handler locally, use lime-task-handler --pool=threads to better mimic production behavior with concurrent task execution. The default solo mode runs tasks sequentially and should only be used for debugging. See Task Handler - Execution Pool Modes for details.

Declare a task

The task decorator provides you with a lime application based on the application name you use in the send_task method. So you have to be aware of the fact, that the first argument of your task function is ALWAYS a lime application

from lime_application import LimeApplication
from lime_task import task


@task
def get_lime_company(
    application: LimeApplication, company_id: int, properties: dict
) -> dict:
    company = _get_company_object(application, company_id)
    description = _concat_description(company, properties)

    return {
        "description": description,
    }

Task QoS Categories

Tasks can be decorated with a qos (Quality of Service) parameter to categorize them by priority or execution requirements. This helps route tasks to appropriate queues for optimal performance:

from lime_task import task


@task(qos="asap")
def urgent_processing_task(app):
    """High-priority task that should execute as soon as possible"""
    pass


@task(qos="background")
def slow_report_generation(app):
    """Low-priority task that can run in the background"""
    pass

Common QoS categories used in Lime CRM:

  1. asap: High-priority tasks that should be executed as soon as possible
  2. background: Lower-priority tasks with no latency requirement
  3. scheduled: Tasks that run on a schedule (automatically applied by the scheduler)

You can define custom QoS categories for your package (e.g., "email" or "integrations") to further isolate or prioritize specific workloads. The actual queue routing is configured in the service configuration.

QoS Best Practices

  • Use qos="asap" for user-triggered actions that need quick feedback
  • Use qos="background" for heavy processing, bulk operations, or tasks that can wait
  • Create custom QoS categories when you need to isolate or scale specific workload types independently

As mentioned before a task status can contain a result after the execution is done. This is whatever this task function returns. The result should be rather considered as a summary of the task than the affected data.

Example of a good result

{
    "result": "updated the sales representative of 50 deal entities to coworker with id 1234",
    "id": "generated-id",
    "status": "SUCCESS"
}

Example of a bad result

{
    "result": {
        "deals": [
            {
                "id": 432,
                "salesRepresentative": 1234
            },
            {
                "id": 321,
                "salesRepresentative": 1234
            }
        ]
    },
    "id": "generated-id",
    "status": "SUCCESS"
}

Send a task

Use the send_task method from the lime_task package in order to start a task. You can either do that in a separate module in your package like this:

from lime_task import send_task

task_name = "my_package.tasks.lime.get_lime_company"  # task function name including the full module path
task_status = send_task(task_name, my_limeapp, 1001)

print("status:", task_status.status)
print("result:", task_status.result)

...or adjust the provided POST request in the generated task_endpoint.py. If you're working on Linux and want to use the endpoint, remember to also install the package on the appserver container.

Get status

You can get the current status of a task if you know the task id.

Note

Make sure the code is bootstrapped, otherwise the service locator and task status service will not be available.

Running this in the context of an endpoint, event handler or a task handler works since they are bootstrapped. The task status service is accessible as a function argument or class attribute of the type TaskStatusService.

Deprecated Method

The older lime_task.get_status(task_id, app_name) function is deprecated. Use TaskStatusService instead for new code.

Version Requirement

TaskStatusService was introduced in lime-core version 25.315.0 (September 2025), available in lime-crm version 2.1135.0 and later. For earlier versions, you must use the deprecated lime_task.get_status() function.

import lime_task
import lime_webserver.webserver


class TaskStatusResource(lime_webserver.webserver.LimeResource):
    task_status_service: lime_task.TaskStatusService

    def get(self, task_id: str):
        try:
            task_status = self.task_status_service.get(task_id)
        except lime_task.NotFoundError:
            return f"Task with id {task_id!r} was not found", 404
        except lime_task.TaskExecutionError as e:
            return {
                "id": task_id,
                "status": "FAILURE",
                "result": f"Task execution error: {str(e)}"
            }, 500

        return {
            "status": task_status.to_dict(),
        }

Task Status Response Format

The task_status.to_dict() method returns a standardized response:

{
  "id": "task-uuid-here",
  "status": "SUCCESS",
  "result": "Task completed: processed 42 companies"
}

Available Status Values

Tasks can have the following status values: - PENDING - Task is waiting to be processed - STARTED - Task is currently being executed
- SUCCESS - Task completed successfully - FAILURE - Task failed during execution - REVOKED - Task was cancelled - RETRY - Task is being retried after failure

Task events

After a task has run an event is dispatched on the message queue.

The following routing keys are used:

  • core.task.{task_name}.success.v1 if the task successfully ran
  • core.task.{task_name}.failure.v1 if the task failed

Hint

Build an event handler that listens to errors in your scheduled tasks. That will help you get notified and react properly e.g. by sending out an email when the task couldn't be completed.

Acknowledge a task

When working with tasks in Lime CRM, you have the option to choose between two types of acknowledgments: early acknowledgment (early ack) and late acknowledgment (late ack). These acknowledgments affect how the task execution is handled and how the task's state is managed. Early acknowledgment acknowledges the task as soon as it is received, while late acknowledgment waits for the worker to confirm task completion before acknowledging it.

The default behavior for a task in in Lime CRM is to do early acknowledgement. You can change that by writing acks_late=True in the task decorator:

import logging

from lime_task import task

logger = logging.getLogger(__name__)


@task(acks_late=True)
def my_task(app, *args, **kwargs):
    logger.info("Executing my_task with args: %s, kwargs: %s", args, kwargs)

You can also combine acks_late with other task decorator parameters such as qos:

@task(qos="background", acks_late=True)
def idempotent_background_task(app):
    logger.info("Executing idempotent background task")

When using late acknowledgment in task processing, it is important to ensure that the task is idempotent. Idempotence means that executing the task multiple times will have the same outcome as executing it once. Please beware, that writing an idempotent task can be challenging, especially when dealing with external systems and potential failures. It's essential to handle various scenarios appropriately to ensure the task behaves correctly and doesn't negatively impact the overall performance of your solution. When an external system is down or inaccessible, it's important to implement appropriate error handling mechanisms, otherwise there is a high risk that your task will be re-run forever and block other tasks in the system from running.

We prefer early ack

We recommend using early acknowledgment for easier task creation, as late acknowledgment tasks can run indefinitely and potentially block other tasks in the system.

Frequently Asked Questions

Does send_task() execute my task immediately?

No. send_task() only enqueues the task to RabbitMQ. The task executes later when an available worker picks it up from the queue. There are no guarantees about when execution will start - it depends on worker availability and queue length.

Does the Task Scheduler execute scheduled tasks at the exact scheduled time?

No. The Task Scheduler only sends the task to the queue at the scheduled time. Execution depends on worker availability, just like manually sent tasks. See Scheduled Tasks for details.

How can I guarantee a task executes immediately?

You can't. Lime Task uses a queue-based architecture where when execution starts depends on worker availability. To improve how quickly tasks start executing:

  • Ensure enough workers are running
  • Use QoS categories to route urgent tasks to dedicated workers
  • Keep tasks short to avoid blocking workers
  • Monitor worker saturation and queue depths

Can I check if a task is still waiting in the queue?

Yes. A task with PENDING status hasn't been received by any worker yet. Once it shows RECEIVED status, a worker has picked it up from the queue. STARTED status means the worker is actively executing the task code. However, you cannot predict how long a task will wait before execution - that depends entirely on worker availability.

Why do I see two RECEIVED events for the same task?

Task acknowledgment failure. This happens when a task fails to be acknowledged by the first worker, causing RabbitMQ to redeliver it to another worker. Both workers receive the task and log a RECEIVED event.

Important: Tasks in Lime CRM do not guarantee "at-most-once" execution. In failure scenarios, a task may be picked up by multiple workers. This is why it's important to design tasks to be idempotent when using acks_late=True, or to use early acknowledgment (the default) to prevent redelivery.

Why is my scheduled task running late?

Worker saturation. Even though the scheduler sent the task on time, if all workers are busy with long-running tasks, your scheduled task waits in the queue. See Scheduled Tasks - Improving On-Time Execution for strategies to address this.

Can each task use a different timezone for scheduling?

No. Celery (and therefore Lime Task) supports only one timezone per solution. The timezone is configured in service configuration and applies to all applications and all tasks. Individual tasks cannot use separate timezones. See Scheduled Tasks - Timezone Support for details.

How long are task results stored?

30 days by default. Task results are stored in Redis or Elasticsearch for the duration specified by the result_expires configuration parameter (default: 2592000 seconds = 30 days). After this time, results are automatically deleted. See Task Handler configuration for details.

What happens if a task fails?

A task event is dispatched. When a task fails, an event with routing key core.task.{task_name}.failure.v1 is published to RabbitMQ. You can create an event handler to react to failures (e.g., send notifications, log errors, trigger retries). See Task events for details.