Loops enable the repeated execution of steps based on the selected loop type.
Torq offers three loop types, each with distinct applications (see examples below):
In: Iterate through each item in a dataset, for example, going over a list of IP addresses and enriching each one.
Range: Perform a set of steps a specified number of times, for example, checking a resource for availability at regular intervals and stopping after several attempts to report an error (polling).
Until Break: Repeat until a condition is met, for example, navigating through pages of results and stopping when no additional pages are available (pagination).
How to Use
Add the Operator: Drag the Loop operator onto the canvas.
Select the Loop Type: Choose one of the following options to define how the loop behaves:
In: Iterate through a dataset and process each item.
Use the In field to specify the path to the dataset you want to loop over.
Range: Execute the set of steps a specified number of times.
Specify the range start, end, and increment size for each iteration.
Until Break: Continue looping until a specific condition is met.
Set up a condition using the If operator and include the Break operator to stop the loop.
Set the Execution Mode: Specify if iterations should be executed sequentially or in parallel. Parallel execution can speed up processing where applicable. Read more about running loop iterations in parallel.
Until Break loops do not support parallel execution of iterations.
Add Steps for Loop Logic: Include steps within the loop to define the actions performed during each iteration.
Access the Loop Index and Value: Use these editable references within the operator's Execution Options in step inputs:
Iteration Index: Use
{{ $.loop_index }}to access the current iteration index.Iterated Data: Use
{{ $.loop_value }}to access the data being processed in the current iteration.
Gather Loop Iteration Results (Optional): Add the Collect operator (available when hovering over the Loop operator).
Input: Specify the data to collect for each loop iteration (string, number, JSON object, or array).
Flatten Results: If working with arrays, choose Yes to combine all arrays into one or No to keep them separate.
End a Loop Early (Optional): Use the Break operator (available when hovering over the Loop operator) to exit the loop based on certain conditions. Once the loop ends, the workflow continues executing the subsequent steps.
In and Range loops support up to 50,000 iterations, while Until Break loops go up to 1,000. If you hit these limits, the loop stops with an error message.
For additional iteration requirements, contact Torq support.
Using Loops: Examples
In: Loop on a dataset (Basic)
This workflow is triggered by a scheduled task that pulls threat intel data from Recorded Future daily. You can substitute Recorded Future with your threat intel service.
Loop Over IP Addresses
After extracting and de-duplicating the IP addresses retrieved from Recorded Future, loop over the list by providing it as input for the Loop operator:
{{ $.unique_ips.result }}
Enrich Each IP Address with VirusTotal
The action performed on each IP address is IP enrichment using VirusTotal. Access the IP address by using {{ $.loop_value }}. Each iteration results in a separate output entry in the execution log.
In: Loop on a Dataset: Loop Within a Loop (Advanced)
Implementing a loop within a loop will require you to manipulate the execution options of the Loop operator so you can access your data and act on it.
This example will show how you can cross-check the IP addresses that came back benign with another platform and check if it's associated with any problematic domains.
Get information on the domain associated with the benign IP
The IP address that came back as not malicious can be run through an additional engine, in this case, AbuseIPDB. In the IP field, you can use {{ $.loop_value }} as that contains the IP address from the initial loop, which came back as not malicious at first glance.
AbuseIPDB provides the domain information for the IP address, which can be passed on to VirusTotal for a domain lookup.
Loop over the DNS records of a domain
VirusTotal returns all of the information about the domain, including the DNS records for the domain. You can loop over those records.
It’s important to note that you can change the loop name and Execution Options to help identify each loop and make it easy to use each loop's Key and Value in context.
Change the Key name to key_dns_record.
Change the Value name to value_dns_record.
Check if the domain is associated with other IPs
To act on the object you looped over, reference $.value_dns_record in subsequent steps.
You can now create a condition whereby if the type under $.value_dns_record is A, the IP address of that record is checked. If the IP address for the Type A record is the same as the initial IP address that was already checked by VirusTotal, do nothing, as there is no point in checking the same IP address twice.
However, if the IP address is different from the initial address, the IP address in the type A DNS record should be checked by VirusTotal.
After handling the type A DNS record, you can break the loop since, for the purposes of this tutorial, the assumption is there is only one type A record.
Range: Polling
The following example shows how to use Torq to perform polling, i.e. waiting for a process to complete before moving on with your workflow. It is based on extracting and analyzing URLs from a suspected phishing email.
Since the analysis and the respective report can take some time to complete, you will need to implement a loop waiting for the results.
Submit extracted URLs for analysis
After extracting URLs from a suspicious email, you can loop over them. In the input for the loop, you should enter the results of the extraction utility step.
The step you want to execute for each of these URLs is scanning with VirusTotal. Enter the value from the key:value pair of the loop step as the URL to scan. In this example, that is: {{ $.loop_2_value }}
Wait for analysis results retrieval to complete
Create another loop within this loop using the Range type. In this example, the loop runs a maximum of 20 times (or less if the results are retrieved sooner).
You will then pass the ID of the URL scan to a Get Analysis Results step. This step attempts to retrieve the report for the URL that was submitted earlier.
If the analysis has been completed, you can check the results to determine if the URL is malicious. Otherwise, configure the workflow to wait a few seconds before checking again. Once completed, you can return to the outer loop and check the next URL.
Until Break: Loop for Pagination
When you're getting large volumes of data and can divide it into pages, use the page size option to keep the data amounts manageable. If you have more data than one page can show, you'll get a next page token. This token points to the following page's data. Use this token in the next run to keep getting more pages. If this token is empty or not there, it means you've got all the data.
In the example below, the activity log is retrieved in pages of 10 entries.
Retrieve the first page of results and check whether pagination is required.
If pagination is required, add a loop and set the loop type to Until Break.
The action performed in each loop iteration is to retrieve the next page of results by specifying the next page token provided by the previous execution of the same step.
Use the Collect operator to accumulate the output of each execution of the step responsible for retrieving the relevant data.
After retrieving each page of results, check if there are more pages to retrieve.
If the next page token is empty or null, it means there are no additional pages to retrieve, and the loop should be broken (by using the Break operator). Otherwise, continue with the loop to retrieve the next page of results.
