Loops enable the repeated execution of steps, based on the selected loop type. Torq offers three loop types, each with distinct applications (see use case examples below):
Items: Iterate through each item in a dataset. For example, go over a list of IP addresses and enrich each one.
Range: Perform a set of steps a specified number of times. For example, checking a resource for availability at regular intervals and stopping after several attempts to report an error (polling).
Until Break: Repeat until a condition is met and the Break operator is triggered. For example, navigating through pages of results and stopping when no additional pages are available (pagination).
How to use
Add the operator: Drag and drop the Loop operator onto the canvas.
Select the loop type: Choose one of the following options to define how the loop behaves:
Items: Iterate through a dataset and process each item.
Use the Items field to specify the path to the dataset you want to loop over.
Range: Execute the set of steps a specified number of times.
Specify the range start, end, and increment size for each iteration.
Until Break: Continue looping until a specific condition is met.
Set up a condition using the If operator and include the Break operator to stop the loop.
Set the execution mode: Specify if iterations should be executed sequentially or in parallel. Parallel executions can speed up processing where applicable. Learn more about running loop iterations in parallel.
Until Break loops do not support parallel execution of iterations.
Add steps for loop logic: Include steps within the loop to define the actions performed during each iteration.
Access the loop index and value: Use these editable references within the operator's Execution Options in step inputs:
Iteration Index: Use
{{ $.loop_index }}to access the current iteration index.Iterated Data: Use
{{ $.loop_value }}to access the data being processed in the current iteration.
Gather loop iteration results (optional): Use the Collect operator to collect the results from each loop iteration.
Click the Loop operator to open the properties panel. Under Suggested operators, drag and drop the Collect operator to add it to the workflow. Alternatively, drag and drop the operator from the Builderbox.
I put: Specify the data to collect for each loop iteration (string, number, JSON object, or array).
Flatten Results: If working with arrays, toggle to Yes to combine all inputs into a single array (default). Toggle to No to keep them separate as multiple appended arrays.
End a loop early (optional): Use the Break operator to exit the loop based on certain conditions.
Click the Loop operator to open the properties panel. Under Suggested operators, drag and drop the Break operator to add it to the workflow.
Once the loop ends, the workflow continues to execute the subsequent steps.
There are limits to the number of iterations that will be performed. If you hit these limits, the loop stops with an error message:
Sequential loops (In and Range) support a maximum of 50,000 iterations
Until Break loops support a maximum of 1,000 iterations
Parallel loops support a maximum of 10,000 iterations
For additional iteration requirements, contact Torq support.
Parallel loops
Parallel loop executions run in batches of up to 10. The execution of each new batch starts only after the previous one is completed. A loop may appear stuck on an iteration if one of the iterations in the current batch takes longer than the others to complete.
Use cases
Items: Loop on a dataset
Use the Items type Loop operator to iterate through a list of data and perform an action on each item.
Enrich each IP address in a list
This workflow iterates through a list of IP addresses, enriching each one.
A list of unique IPv4 addresses is extracted from data received from RecordedFuture.
The workflow then loops over the list of unique IPs by providing the IP addresses array as input for the Loop operator.
For example,
{{ $.extract_all_ipv4_addresses.results }}.
Each IP address is enriched using VirusTotal.
Each iteration results in a separate output entry in the Execution Log.
Range: Polling
Use the Range type Loop operator to perform polling, which involves waiting for a process to complete before continuing with the execution.
Multiple attempts to retrieve URL scan analysis
This workflow scans URLs and issues a report once the analysis results are available. The workflow iterates a predefined number of times before exiting with a URL scan analysis report or with an error if no results are found.
The workflow scans a URL from a suspicious email.
The analysis of the URL scan and reporting can take some time to complete, so you need to implement a loop waiting for the results.
Use the Range type loop to get analysis results from the Virustotal scan. The loop runs a maximum of nine times (unless the results are retrieved sooner).
Inside the loop, the If operator is used to add conditions to the flow:
If the analysis is complete, Virustotal will return an analysis report, and the workflow exits.
If the analysis is not completed, the Wait operator pauses the workflow for 20seconds and then continues the next iteration to check if the analysis is complete.
If the analysis is incomplete after the nine iterations, the workflow exits with an error.
Until break: Loop for pagination
Use the Until break loop type to divide data into pages to keep large volumes of data manageable. If you have more data than one page can show, you'll get a next page token that points to the following page's data. Use this token in the next run to keep getting more pages. If this token is empty or not there, it means you have extracted all the data.
The Unti Break loop type requires If and Break operators to exit the loop when conditions are met.
Pagination using Next page token
In the example below, the activity log is retrieved in pages, and the workflow iterates through each page of data until all data is extracted, using the next page token.
Create an Until break type loop to gather activity logs.
Inside the loop, use the Torq List Activity Logs step to retrieve the list of activity logs.
In the step property panel, configure the optional Next page token parameter to reference the content in its own execution log.
On the first iteration of the loop, the Next page token field will return empty because the Execution Log has not been populated yet.
Subsequent iterations of the loop will return Activity Log data from the now-populated Execution Log.
Use the If operator to check whether an Activity Log exists. If not, use the Break operator to exit the loop.
If an Activity Log exists, collect the log data and check if pagination is required:
Use the Collect operator to accumulate the output for each iteration. The Collect operator collects the specified results, and the results from each iteration are compiled into a single combined array.
Use an additional If operator to check if there are more pages to retrieve. Configure the condition to reference the next page token from the List Activity Logs step. If the next page token is empty or null, break the loop. Otherwise, continue to the top of the loop for the next iteration.
The condition input
{{ $.list:activity_logs.api_object.next_page_token }}is populated by the previous execution of the loop.Once all the activity log data is received, the loop is broken by the inner If operator in step 4 above.
Items: Loop on a dataset using nested loops
Use nested Loops to handle data at multiple levels, like alerts and the observables within them.
Each separate loop must have a unique name, and unique value and key names. Distinct names ensure you can access the right level of data at each point in the workflow.
Off-board a user from all groups
The workflow extracts all the user groups associated with an email address, removing the user from each group.
The outside loop (named Loop over users) iterates over each user, extracting the user group information associated with each user email.
The nested loop (named Loop over groups) then iterates over each of these groups and removes the user.
Once the user has been removed from all associated groups, the workflow iterates through the outside loop to the following user.
Data manipulation before a loop
Manipulating data allows you to define and filter your data before it enters the loop, reducing the complexity of your loop logic and the number of iterations needed.
You can achieve this with various utility steps or the Transform Data operator.
Transform data before a loop
The workflow receives a JSON feed with multiple alerts, each containing user, severity, and source info.
Use the Transform Data operator to tidy the data entering the loop. For example, use the prompt: "Extract alerts with severity 'high' and only keep
user,source_ip, andalert_id." The output is a SON array of high-severity alerts with only the fields needed.Use the Loop operator to loop over each alert item, enriching each
source_ipand creating incident records or notifications using theuserandalert_id.
When data has been pre-filtered and cleaned up, loops are simpler, faster, and less error-prone.
You can manipulate arrays of JSON objects using a loop. However, best practice favours using alternatives such as the Transform data operator or Utility steps.
Learn more by taking the To loop or transform module of the Torq Automation Expert course in the Torq Academy.