Errors related to memory

Appizap Workflow Builder allows for flexibility in the amount of data that can be retrieved and manipulated by each node within a workflow. This unrestricted access provides users with freedom to create complex workflows without limitations. However, this openness can sometimes result in issues, particularly when the execution of workflows exceeds the available memory capacity. This guide aims to address this challenge by offering insights on recognizing and preventing such errors from occurring.

Identifying "out of memory situations"

Appizap Workflow Builder offers error notifications to caution users about potential memory-related issues. These notifications include messages like "Execution halted at this node due to a possible memory depletion in Appizap Workflow Builder". The occurrence of error messages such as "Problem running workflow," "Connection Lost," or "503 Service Temporarily Unavailable" indicates that an Appizap Workflow Builder instance is no longer accessible.

When hosting Appizap Workflow Builder on your own server, you may encounter error messages like "Allocation failed - JavaScript heap out of memory" in the server logs. When utilizing Appizap Workflow Builder Cloud or the Appizap Workflow Builder Docker image, Appizap Workflow Builder will automatically restart upon encountering this issue. Conversely, if Appizap Workflow Builder is operated using npm, manual restart may be necessary.

Typical causes

Issues like this arise when a workflow process in Appizap Workflow Builder requires a larger amount of memory than what is currently allocated. Several factors can contribute to the increased memory usage in workflow executions, leading to performance bottlenecks and potential failures. Some of these factors include:

• Amount of JSON data.

• Size of binary data.

• Number of nodes in a workflow.

• Some nodes are memory-heavy: the Code node and the older Function node can increase memory consumption significantly.

• Manual or automatic workflow executions: manual executions increase memory consumption as Appizap Workflow Builder makes a copy of the data for the frontend.

• Additional workflows running at the same time.

Avoiding out of memory situations

When faced with a lack of memory issue, there are two possible courses of action that can be taken in order to address the problem within the Appizap Workflow Builder platform. One option is to allocate more memory resources to Appizap Workflow Builder, thereby enabling it to handle larger volumes of data and more complex workflows.

Alternatively, the memory consumption of Appizap Workflow Builder can be optimized by identifying and eliminating unnecessary processes or functions that may be draining resources unnecessarily. By implementing either of these solutions, users can ensure that Appizap Workflow Builder operates efficiently and effectively without running into memory constraints.

Increase available memory

When setting up Appizap Workflow Builder on your own hosting environment, the process of enhancing the memory capacity for Appizap Workflow Builder involves allocating a greater amount of memory for your Appizap Workflow Builder instance. This could potentially lead to supplementary expenses through your selected hosting service.

Reduce memory consumption

This method is more complicated and involves restructuring workflows. Here are some tips to decrease memory usage. Not all suggestions work for every workflow.

• Split the data processed into smaller chunks. For example, instead of fetching 10,000 rows with each execution, process 200 rows with each execution.

• Avoid using the Code node where possible.

• Avoid manual executions when processing larger amounts of data.

• Split the workflow up into sub-workflows and ensure each sub-workflow returns a limited amount of data to its parent workflow.

Splitting the workflow can initially seem counter-intuitive. It involves adding two nodes: Loop Over Items and Execute Workflow.

If the sub-workflow completes most of the work for each batch and sends a small result set back to the main workflow, it helps reduce memory usage. The sub-workflow only needs to store the data for the current batch in memory.

Increase old memory

When using self-hosted Appizap Workflow Builder, there may be instances where you come across JavaScript heap out of memory errors. In such cases, it can be helpful to assign more memory to the old memory section of the V8 JavaScript engine. This can be achieved by configuring the V8 option --max-old-space-size=SIZE either via the command line interface (CLI) or the NODE_OPTIONS environment variable.