The Splunk integration enables you to trigger workflows with Splunk alerts to perform searches and create alerts as part of a workflow.

Did you know? Splunk has a dedicated App for Torq Log Insights!

Use Splunk to Trigger Torq Workflows

To ingest Splunk Enterprise alerts as events in Torq, you must create a generic Webhook integration and configure an alert in Splunk using the generated webhook URL (Torq endpoint). The alert data is sent via the webhook to Torq as a trigger event.

Image showing how logs from various service to Splunk and how Torq ingests them as events.

Configure a Webhook Integration in Torq

Go to Build > Integrations > Triggers > Webhook and click Add.
Type a meaningful name for the integration instance, for example, Splunk-Receiver, and click Add.
Locate the integration and copy the URL link. You will need this when you create the alert in Splunk.
With Splunk Enterprise v9.0 and Splunk Cloud v8.2.2203, Splunk implemented additional security controls requiring the configuration of authorized URL endpoints to which a webhook alert action may be sent. To send an alert action webhook to Torq, ensure that the webhook allow list is updated;
1. Splunk Enterprise customers must adjust the [webhook stanza of alert_actions.conf while Splunk Cloud customers may use Splunk Web to make the necessary changes.
2. Splunk performs a regular expression match against URLs that appear in the allow list. If there is a string match, then an alert is sent to the specified webhook URL. Splunk recommends that when adding a URL to the webhook allow list, define the URL as completely as possible to achieve the most restrictive match.
3. Configure https:\/\/(.*\.|)torq.io\/?.*\/ to handle webhooks generated anywhere within the Torq ecosystem.
4. Specify your environment, https:\/\/(.*\.|)hooks.torq.io\/?.*\/ (for US deployments) or
5. https:\/\/(.*\.|)hooks.eu.torq.io\/?.*\/ for European deployments
6. Or choose to allow only the precise webhook URL defined within Torq.
7. For more information on this feature and how to configure it, consult your applicable Splunk Enterprise or Splunk Cloud documentation.

Create an Alert in Splunk

To create an alert, you'll define and run a Splunk query and save that query as an alert. This alert will be sent via webhook to Torq as a trigger event, either in real-time or on a schedule.

By default, real-time alerts in Splunk Cloud are disabled. Configure the alert as appropriate and supported in your environment. Splunk Cloud also supports sending alerts on a schedule. Refer to Splunk's documentation (for Splunk Enterprise and Splunk Cloud) for more information.

Go to your Splunk homepage and click Search & Reporting.
Enter a search query in the search bar and run the search. For this example, we'll use the search query source="udp:514" sourcetype="syslog".
Save the search as an alert.
Fill in the save alert form.
Enter a meaningful title, for example, Send alerts to Torq.
In the Trigger Actions section, click Add Actions and select Webhook.
Enter the Splunk webhook URL (Torq endpoint) you created earlier and click Save.

Use Splunk Steps in a Torq Workflow

To use Splunk steps in Torq workflows, you must create a Splunk steps integration, which requires a Splunk Enterprise API token.

After you create the token, it will appear in the Token field. Ensure you copy it because it will not be accessible after you close the window.

All Splunk integration steps will also work when using a Splunk Cloud integration, but the Splunk Cloud steps are unique to the Splunk Cloud integration.

Splunk Cloud requires the configuration of its ACLs to allow access to the API port; adding Torq IPs to the Splunk Cloud IP allows rules for the search heads (SHC and ES Search head) via port 8089 (Splunk API).
Historically, this meant opening a ticket with Splunk or using Splunk's Admin Control Service (ACS) API. Recently (as of August 2024), Splunk simplified the configuration with a Splunk Web-based option for self-service configuration of IP-allow rules.
Here's an additional Splunk article on configuring the ACLs for Splunk Cloud: Configure IP allow lists for Splunk Cloud Platform - Splunk DocumentationFor performing search operations on Splunk Cloud:
- Search head API access (port 8089) to the applicable search head: https://[vanity URL].splunkcloud.com:8089
- You may configure multiple integrations, one for each search head you wish to query, such as Enterprise Security, ITSI, and your search head cluster. To send data to Splunk Cloud (such as Torq audit and activity logs):
- HEC access (port 443) to the HEC endpoint: https://http-inputs-[vanity URL].splunkcloud.com
  - This requires the configuration of an HEC token separate from the API token used for performing search operations.

Create a Splunk API Token

For more information about Splunk tokens, see the Splunk documentation.

Sign in to your Splunk tenant.
Click Settings > Tokens. If this is your first time using tokens, you might have to enable token authentication.
Click New Token, configure the token parameters, and then click Create. In our example, the token will expire in 30 days.

Create a Splunk Enterprise Integration in Torq

The following integration can be used for Splunk Enterprise and Splunk Cloud steps.

Go to Build > Integrations > Steps > Splunk and click Add.
Type a meaningful name for the integration instance. This cannot be changed later.
Enter the API token that you generated in your Splunk tenant.
Enter the URL of your Splunk tenant (including port).
Click Add.