Tasks¶

Lime Task provides the functionality to run asynchronous tasks, declare/create and start a task as well as asking for its status.

In order to gain the best performance it's a good practice to base your solutions on tasks. Any action that takes longer than a normal REST request should be realized as a task. Same good use cases are e.g. bulk updates, heavy processing, reporting or integrations.

In comparison to a REST request the execution is handled asynchronously. That means, that whenever a task is started, it's put in a queue and waits for an available worker to be executed. For that reason Lime Task can't promise any execution times just that the task is eventually taken care of. During that time a task owner can ask for the current status, which might be PENDING, STARTED, SUCCESS or FAILURE. Whenever the execution of a task is completed, the status object can also contain a result.

Generate the tasks module¶

To add the tasks module to your package, change directory to your package directory and run this command:

lime-project generate tasks

If you're working on Linux, make sure your package is installed on the taskhandler container.

Declare a task¶

The task decorator provides you with a lime application based on the application name you use in the send_task method. So you have to be aware of the fact, that the first argument of your task function is ALWAYS a lime application

from lime_task import task

@task
def get_lime_company(app, id, props):
    company = _get_company_object(app, id)
    description = _concat_description(company, props)

    return dict(description=description)

As mentioned before a task status can contain a result after the execution is done. This is whatever this task function returns. The result should be rather considered as a summary of the task than the affected data.

Example of a good result¶

{
    "result": "updated the sales representative of 50 deal entities to coworker with id 1234",
    "id": "generated-id",
    "status": "SUCCESS"
}

Example of a bad result¶

{
    "result": {"deals": [{"id": 432, "salesRepresentative": 1234},{"id": 321, "salesRepresentative": 1234},] },
    "id": "generated-id",
    "status": "SUCCESS"
}

Send a task¶

Use the send_task method from the lime_task package in order to start a task. You can either do that in a separate module in your package like this:

from lime_task import send_task

task_name = 'my_package.tasks.lime.get_lime_company' #task function name including the full module path
task_status = send_task(task_name, my_limeapp, 1001)

# Update status
print('status:', task_status.status)
print('result:', task_status.result)

or adjust the provided POST request in the generated task_endpoint.py. If you're working on Linux and want to use the endpoint, remember to also install the package on the appserver container.

Get status¶

You can also get the current status by just knowing the task id and the application name. For data security reason you are just allowed to ask for a task status with the same application that started the task in the first place. Otherwise the method raises a NotFoundError.

import lime_task
import logging

logger = logging.getLogger(__name__)

task_id = 'generated-id'
app_name = 'my-app-name'
try:
    task_status = lime_task.get_status(task_id, app_name)
    while not task_status.is_completed()):
        time.sleep(30)
        task_status = lime_task.get_status(task_id, app_name)

    print('status:', task_status.status)
    print('result:', task_status.result)
except lime_task.NotFoundError:
    logger.error('Couldn't find task {} for app {}.format(task_id, app_name))

Task events¶

After a task has run an event is dispatched on the message queue.

The following routing keys are used:

core.task.{task_name}.success.v1 if the task succesfully ran.
core.task.{task_name}.failure.v1 if the task failed.

Hint

Build an eventhandler that listens to errors in your scheduled tasks. That will help you get notified and react properly e.g. by sending out an email when the task couldn't be completed.

Acknowledge a task¶

When working with tasks in Lime CRM, you have the option to choose between two types of acknowledgments: early acknowledgment (early ack) and late acknowledgment (late ack). These acknowledgments affect how the task execution is handled and how the task's state is managed. Early acknowledgment acknowledges the task as soon as it is received, while late acknowledgment waits for the worker to confirm task completion before acknowledging it.

The default behavior for a task in in Lime CRM is to do early acknowledgement. You can change that by writing acks_late=True in the task decorator:

@task(acks_late=True)
def my_task(app, *args, **kwargs):
    ...

When using late acknowledgment in task processing, it is important to ensure that the task is idempotent. Idempotence means that executing the task multiple times will have the same outcome as executing it once. Please beware, that writing an idempotent task can be challenging, especially when dealing with external systems and potential failures. It's essential to handle various scenarios appropriately to ensure the task behaves correctly and doesn't negatively impact the overall performance of your solution. When an external system is down or inaccessible, it's important to implement appropriate error handling mechanisms, otherwise there is a high risk that your task will be re-run forever and block other tasks in the system from running.

We prefer early ack

We recommend using early acknowledgment for easier task creation, as late acknowledgment tasks can run indefinitely and potentially block other tasks in the system.