The Collect operator is a powerful and flexible tool for efficiently gathering data from loop iterations into a cohesive array. It can be used to handle data in complex workflows more simply and allow subsequent steps to reference the loop's results.
The Collect operator must be used within a loop.
The operator collects a maximum of 20 MB of data. If this limit is exceeded, the step will fail with an error.
How to use
Initiate a loop: Start by adding a Loop operator to your workflow.
Add the Collect operator: Drag and drop the Collect operator into the loop.
Configure the collection: In the Collect operator's property panel, specify the path to the data you want to gather from each loop iteration.
Flatten results
In the Collect operator property panel, the Flatten results parameter allows you to combine all iteration inputs into a single array or store inputs as multiple appended arrays. By default, each iteration's results are combined and stored as a single array, simplifying data management for scenarios like paginated results.
Nested loops
In a nested loop, each iteration of the parent loop clears previously collected nested loop results. To maintain all results in a single array, use additional Collect operators in the parent loop.
Multiple collections
To segment the collected data, you can add more Collect operators within the same loop, each capturing different sets of information from the incoming data.
Use cases
See an example of how the Collect operator works in the Simple loops with Torq template.
Pagination to extract a complete data set
In the example below, the Activity Log is retrieved in pages of 10 entries. The workflow iterates through each data page, collecting results for each iteration, until all data is extracted. See more details about this same example in the Loop operator article.
Retrieve the activity log in pages of 10 entries, and if pagination is required, add an Until break loop type.
Each loop iteration retrieves the next page of results. The workflow processes the results for each page and before proceeding to the next page.
Use the Collect operator in the loop to compile the specified results for each iteration into a single combined array, by appending each iteration's results to those that have already been gathered.
Use the Break operator to break the loop once there are no additional pages to retrieve.
Subsequent steps in the workflow can access the data gathered by the Collect operator.
Combining outputs into a single array
The Collect operator can also gather results from multiple previous steps into a single array. This allows you to aggregate related data into one place, which can be accessed by subsequent steps in the workflow. By mapping fields from these steps into the Collect operator, you can build a custom JSON object.
This example collects both the user name and email from two different steps in the loop into a combined array.
Add the Collect operator into a loop, after the workflow steps you want to combine the results from.
Map the outputs of those steps into the input parameter of the Collect operator.
On each iteration, the Collect appends the incoming results from both steps into a single array.