Batch Processing

Learn how to perform batch processing.

Overview

Batch processing is a method of handling data by dividing it into smaller groups (batches) and processing each group separately. This is useful when working with large volumes of information such as customer records, invoices, or files as it improves efficiency and error handling.

How It Works

When you create batches, the items are divided into groups of the specified batch size. Each batch becomes a separate collection that you can process independently.

Example: Instead of sending 1,000 customer emails all at once, you can create batches of 50 emails each, resulting in 20 manageable batches to process one at a time.

Note: The last batch may contain fewer items if the total count doesn't divide evenly. For example, if you have 7 total items and specify a batch size of 3:

• Batch 1: Items 1, 2, 3

• Batch 2: Items 4, 5, 6

• Batch 3: Item 7

Syntax

Below is a line-by-line overview of the automation syntax. Expand each line to learn more.

create batches from the collection

What does it do?

Instructs the system to create batches from a collection or dataset.

Where does it go?

This phrase should be written on a new line.

Is it required?

✅ Yes — This phrase is required in the syntax.

Does it require data?

✅ Yes — Replace the collection with a reference to the dataset or objects you want to batch.

Example

create batches from the items

the batch size is x

What does it do?

Specifies how many items should be included in each batch.

Where does it go?

Indented beneath create batches from the collection.

Is it required?

✅ Yes — This phrase is required to define the batch size.

Does it require data?

✅ Yes — Replace x with an integer value representing the number of items per batch.

Example

the batch size is 3

After creating batches, process each one individually using:

process each batch as follows

Within the processing logic, reference the items in the current batch using the batch's <collection>. For example:

the batch's users

the batch's invoices

the batch's documents

Examples

1. Basic Batch Creation

The following example shows how to create batches from a simple list of numbers:

Automation

the items are 1, 2, 3, 4, 5, 6, 7
create batches from the items
    the batch size is 3

Results

• Batch 1: [1, 2, 3]

• Batch 2: [4, 5, 6]

• Batch 3: [7]

2. Processing Customer Records

Here, the customers are processed in batches of 10.

the customers are the above
create batches from the customers
    the batch size is 10
process each batch as follows
    send an email to "[email protected]" where
        the subject is "Currently processing 10 customers"
        the message is "The users are {the batch's customers}"

3. Invoice Processing

In this example, the table's rows are broken up in batches. Then, each batch is processed in parallel using run parallelization.

get the invoice table's rows
get the above as the invoices
create batches from the invoices
  the batch size is 20
process each batch as follows
  the results are the batch's invoices
  start a run where
    the procedure is "to process the invoices"
    the results
wait for the runs