By default, loop iterations are executed sequentially. This means an iteration on an item runs to completion before another starts.
When loop iterations are not dependent on one another, running the loop in Parallel mode is a good idea. Running loops in parallel significantly speeds up loop execution time and is, therefore, more efficient.
Note that loop iterations have a limit of 50,000 (fifty thousand). This is split into 10,000 (ten thousand) per loop in parallel loops.
When to Use Parallel Mode
These are some common cases for running loops in Parallel mode:
Query multiple threat intel vendors and process the queries simultaneously
Send question messages to multiple users while investigating an incident and process the input simultaneously
Scan vulnerabilities across multiple networks and devices simultaneously
Important Notes
Running a loop in Parallel mode will result in simultaneous calls to the same vendor (from the steps in the loop's body). This means the vendor's rate limit might be reached, causing steps to fail. Check the vendor's documentation for rate limits and implement a retry mechanism to resolve this situation.
If a loop requires over 50 iterations at once, it will split into batches of 50 iterations at a time (ex: 100 iterations split into two batches of 50, 80 interactions split into a batch of 50, and a batch of 30). This will not impact the speed at which the steps are run.
Having a loop inside of a parallel loop will multiply the number of iterations run, and you may hit the limit (5000) faster. If you are looping through data, use Chunk array to split the array into smaller batches and avoid hitting the limit of iterations run.
If a step run by a private runner inside the body of a loop fails, check the runner host's available resources and make sure they are sufficient to support the parallel loops.
Exit and Break operators cannot be placed inside a parallel loop.
When to Use Sequential Mode
Run loops in Sequential mode when the order in which the loop iterations are executed is important. Here are a few examples:
Pagination requests
Retry on failure
Escalation flow
How to Use Parallel Mode
The execution order of the loop iterations is unknown. In the example below, the loop input is a list of IP addresses to investigate.
Multiple IP queries are submitted to VirusTotal simultaneously so you can collect results as they are returned. You don't have to wait for a specific query's result before submitting the next one.
Drag the Loop operator onto the designer and switch the Mode toggle to Parallel.
Select the Type and define the data to loop over.
In: the array to loop over
Range: Define the Start, End, and Step size.
(Optional) Use the Collect operator to collect the loop results into a single array.
