Use these utility steps to extract information that can then be accessed in subsequent workflow steps, for example, extracting IOCs from events. See below for examples of each step's input and output.
The steps are grouped into the following categories based on their primary functionality.
Category | Description |
General IOC steps | Checking and extracting all IOCs |
IP address steps | Checking and extracting IP addresses |
Domain, URL, and email steps | Extracting domains, URLs, and email addresses |
Hash & CVE steps | Extracting hashes and IOCs |
Regex steps | Extracts expressions that match the given regex |
JSON objects | Flattens JSON objects |
General IOC steps
Extract IOCs
Extract IOCs from a given input.
Input
A text potentially containing one or more IOCs, like 192.168.10.45, http://malicious.example.com/download, badguy@example.net, e99a18c428cb38d5f260853678922e03
Output
"results": [
"192.168.10.45",
"malicious.example.com",
"http://malicious.example.com/download",
"badguy@example.net",
"e99a18c428cb38d5f260853678922e03"
]
IP address steps
Extract all IPv4 addresses
Returns a list of all IPv4 addresses extracted from the given text.
Input
A text potentially containing one or more IPv4 addresses, like 127.0.0.1 and 10.0.3.11.
Output
"results": [
"127.0.0.1",
"10.0.3.11"
]
Extract all IPv6 addresses
Returns a list of all extracted IPv6 addresses from the given text.
Input
A text potentially containing one or more IPv6 addresses, like 2001:0db8:85a3:0000:0000:8a2e:0370:7334 and fe80:3::1ff:fe23:4567:890a.
Output
"results": [
"2001:0db8:85a3:0000:0000:8a2e:0370:7334",
"fe80:3::1ff:fe23:4567:890a"
]
Check if IP Address is private
Check if the given IP address is only for use within a private network.
Input
8.8.8.8
Output
"api_object": {
"is_private": "False",
"network_address": "8.8.8.8/32",
"version": "4"
Extract the first IPv4 address
Returns the first IPv4 address extracted from the given text.
Input
A text potentially containing one or more IPv4 addresses, like 127.0.0.1 and 10.0.3.11.
Output
"result": "127.0.0.1"
Extract the first IPv6 address
Returns the first IPv6 address extracted from the given text.
Input
A text potentially containing one or more IPv6 addresses, like 2001:0db8:85a3:0000:0000:8a2e:0370:7334 and fe80:3::1ff:fe23:4567:890a.
Output
"result": "2001:0db8:85a3:0000:0000:8a2e:0370:7334"
Domain, URL and email steps
Extract all domains
Returns a list of all domains (including, but not limited to, FQDNs) extracted from the given text.
Input
A text potentially containing one or more domain names, like en.wikipedia.com and torq.io.
Output
"results": [
"en.wikipedia.com",
"torq.io"
]
Extract all email addresses
Returns a list of all email addresses extracted from the given text.
Input
A text potentially containing one or more email addresses, like mary@gmail.com and tom@hey.com.
Output
"results": [
"mary@gmail.com",
"tom@hey.com"
]
Extract all URLs
Returns a list of all URLs extracted from the given text. Only URLs which start with 'http://', 'https://' or 'www' are returned by default.
Input
A text potentially containing one or more URL addresses, like https://google.com and https://facebook.com.
Output
"results": [
"https://google.com",
"https://facebook.com"
]
Extract domain from email
Extracts domain from an email address.
Input
mary@gmail.com
Output
gmail.com
Extract first domain
Returns the first domain (including, but not limited to, FQDNs) extracted from the given text.
Input
A text potentially containing one or more domain names, like en.wikipedia.com and torq.io.
Output
"result": "en.wikipedia.com"
Extract first email
Returns the first email address extracted from the given text.
Input
A text potentially containing one or more email addresses, like mary@gmail.com and tom@hey.com.
Output
"result": "mary@gmail.com"
Extract the domain from a URL
Returns the domain extracted from the given URL.
Input
http://www.<domain name>.com
Output
<domain name>.com
Extract the first URL
Returns the first URL extracted from the given text.
Input
A text potentially containing one or more URL addresses, like https://google.com and https://facebook.com.
Output
"result": "https://google.com"
Extract URL parts
Extracts the parts of a URL address
Input
https://linkprotect.cudasvc.com/url?a=https%3A%2F%2Fpressetunisie-1316850103.cos.ap-chongqing.myqcloud.com%2Fpressetunisie.html&c=E,1,XIgB83y2NPwfKHSIr6j2_O8VyZ_CqeEi1q-K84aHVWE-XIHK_NdnRs1bkGCuaLi3nsIYk6nbnqN3PXAOJ2zYJ1i-lwlmvAiJNnHq78Z_Bg,,&typo=1
Output
"result": {
"firstLevelDomain": "cudasvc.com",
"fragments": "",
"fullyQualifiedDomain": "linkprotect.cudasvc.com",
"host": "linkprotect.cudasvc.com",
"password": "",
"path": "/url",
"port": "",
"queryParameters": {
"a": "https://pressetunisie-1316850103.cos.ap-chongqing.myqcloud.com/pressetunisie.html",
"c": "E,1,XIgB83y2NPwfKHSIr6j2_O8VyZ_CqeEi1q-K84aHVWE-XIHK_NdnRs1bkGCuaLi3nsIYk6nbnqN3PXAOJ2zYJ1i-lwlmvAiJNnHq78Z_Bg,,",
"typo": "1"
},
"scheme": "https",
"username": ""
}
Expand shortened URL
Expand shortened URLs to their original form.
Input
https://bit.ly/3YKy87s
Output
"results": [
"https://google.com"
]
Hash & CVE steps
Extract all CVEs
Returns all of the CVEs extracted from the given text.
Input
A text potentially containing one or more CVEs, like CVE-1999-0001 or CVE-2022-32768.
Output
"results": [
"CVE-1999-0001",
"CVE-2022-32768"
]
Extract all MD5 hashes
Returns a list of all MD5 hashes extracted from the given text.
Input
A text potentially containing one or more MD5 hashes, like 99b6814ccfb074ad6acb28ae47e5db1a and 77fe4f88c159068eced74629bbfef099.
Output
"results": [
"99b6814ccfb074ad6acb28ae47e5db1a",
"77fe4f88c159068eced74629bbfef099"
]
Extract all SHA1 hashes
Returns a list of all extracted SHA1 hashes from the given text.
Input
A text potentially containing one or more SHA1 hashes, like 1c5007684acc8f01285dd51cd05136c00b257329 and a4f82a0285ca3ed87162069244fc1618f7046513.
Output
"results": [
"1c5007684acc8f01285dd51cd05136c00b257329",
"a4f82a0285ca3ed87162069244fc1618f7046513"
]
Extract all SHA256 hashes
Returns a list of all extracted SHA256 hashes extracted from the given text.
Input
A text potentially containing one or more SHA256 hashes, like 4bac27393bdd9777ce02453256c5577cd02275510b2227f473d03f533924f877 and ed2456914e48c1e17b7bd922177291ef8b7f553edf1b1f66b6fc1a076524b22f.
Output
"results": [
"4bac27393bdd9777ce02453256c5577cd02275510b2227f473d03f533924f877",
"ed2456914e48c1e17b7bd922177291ef8b7f553edf1b1f66b6fc1a076524b22f"
]
Extract First CVE
Returns the first CVE extracted from the given text.
Input
A text potentially containing one or more CVEs, like CVE-1999-0001 or CVE-2022-32768.
Output
"result": "CVE-1999-0001"
Extract the first MD5 hash
Returns the first MD5 hash extracted from the given text.
Input
A text potentially containing one or more MD5 hashes, like 99b6814ccfb074ad6acb28ae47e5db1a and 77fe4f88c159068eced74629bbfef099.
Output
"result": "99b6814ccfb074ad6acb28ae47e5db1a"
Extract the first SHA1 hash
Returns the first SHA1 hash extracted from the given text.
Input
A text potentially containing one or more SHA1 hashes, like 1c5007684acc8f01285dd51cd05136c00b257329 and a4f82a0285ca3ed87162069244fc1618f7046513.
Output
"result": "1c5007684acc8f01285dd51cd05136c00b257329"
Extract the first SHA256 hash
Returns the first SHA256 hash extracted from the given text.
Input
A text potentially containing one or more SHA256 hashes, like 4bac27393bdd9777ce02453256c5577cd02275510b2227f473d03f533924f877 and ed2456914e48c1e17b7bd922177291ef8b7f553edf1b1f66b6fc1a076524b22f.
Output
"result": "4bac27393bdd9777ce02453256c5577cd02275510b2227f473d03f533924f877"
Regex steps
Regex steps use the Go (Golang) regular expression syntax.
Extract all using a regex group
Returns a list of all expressions that match the given regex groups (parentheses) pattern using RE2/golang syntax, returning the full match and the groups.
Input
mary5@gmail.com
Regex
mary([a-zA-Z0-9]*)@([a-zA-Z0-9]*).com
Output
{
"results": [
{
"match": "mary5@gmail.com",
"groups": ["5", "gmail"]
}
],
"count": 1,
"found": true
}
Extract all using a regex pattern
Returns a list of all expressions that match the given regex pattern using RE2/golang syntax, returning the full match.
Input
mary5@gmail.com
Regex
mary.*@.*.com
Output
[
"mary5@gmail.com"
]
Extract first expression
Returns the first expression that matches the given regex pattern using RE2/golang syntax.
Input
mary5@gmail.com maryAlice@yahoo.com mary123@outlook.com
Regex
mary.*@.*.com
Output
[
"mary5@gmail.com"
]
Steps for JSON objects
Flatten JSON Object
Given a JSON object (e.g., record or list of records) and field names, this step extracts a list of all inner-most values for the provided field names. If no field names are provided, the entire object will be flattened, and the names of the new keys will match the JSON paths of each of the object's values.
Input
[
{
"name": "John Doe",
"age": 37,
"mother": {
"name": "Jennifer Eod",
"age": 73,
"mother": {
"name": "Nature",
"age": -1
}
}
},
{
"name": "Jane Doe",
"age": 92,
"mother": {
"name": "Lilly McDonald",
"age": 125
}
}
]
Output
{
"result": {
"0.age": 37,
"0.mother.age": 73,
"0.mother.mother.age": -1,
"0.mother.mother.name": "Nature",
"0.mother.name": "Jennifer Eod",
"0.name": "John Doe",
"1.age": 92,
"1.mother.age": 125,
"1.mother.name": "Lilly McDonald",
"1.name": "Jane Doe"
}
Use Case: Extract all IPv4 addresses
This workflow demonstrates how to use the Extract all IPv4 addresses utility step to extract IP addresses from an incoming event, and then enrich the associated case with those IPs as observables.
Create a case: Create a case from the trigger event using the Create a Case step.
In the step Property panel, configure the parameters by giving the case a title and assigning an SLA and severity level.
Extract IP addresses: Extract all IPv4 patterns from the incoming event data.
The util step converts the raw event data into structured, usable values.
The results are stored in
$.extract_all_ipv4_addresses.results, to be used in subsequent steps.
Loop over extracted IPs:
Use a Loop operator to iterate through the list of IPs.
Each extracted IP is then added to the case as an observable using the Add Observable to a Case step.
