Skip to main content

Deploy self-hosted Step Runners on AWS Elastic Kubernetes Service (EKS)

Learn how to deploy Torq Step Runners on an AWS EKS cluster.

Updated over 3 weeks ago

AWS Elastic Kubernetes Service (EKS) manages your hosted Kubernetes environment, making it quick and easy to deploy and manage containerized applications without container orchestration expertise. It also eliminates the burden of ongoing operations and maintenance by provisioning, upgrading, and scaling resources on demand without taking your applications offline.

If you don’t have an AWS EKS Kubernetes cluster deployed, you’ll need to create one. If you already have one set up, you can skip to the section on deploying Step Runners on the cluster.

Create Elastic Kubernetes Service (EKS) clusters

To create an AWS EKS cluster:

  1. Open AWS EKS: Sign in to the AWS Management Console and go to EKS.

  2. Create a cluster: Navigate to Clusters and click Create cluster.

    1. Enter a meaningful name.

    2. Select a Kubernetes version.

    3. Select a cluster IAM role with sufficient permissions.

    4. Select a node IAM role with sufficient permissions.

    5. Assign the cluster to a VPC.

    6. Select multiple subnets. Exclude e and f availability zones as they typically do not support EKS deployments.

    7. Click Create. The process will take around 5 minutes to complete.

Add compute nodes to EKS clusters

It is recommended to allocate at least 2 GB RAM and 2 vCPUs to nodes for Step Runners.

  1. Navigate to node groups: When the cluster's status changes to Active, go to the cluster's Compute tab and find the Node groups section.

  2. Add a node group: Click Add node group.

  3. Configure the node group:

    1. Enter a meaningful name.

    2. Select a node IAM role with sufficient permissions.

    3. Click Next.

    4. (Optional) Modify the node group's default properties.

    5. Click Next.

    6. Select the compute subnets.

    7. Click Next.

    8. Review the node group's configurations and click Create. The process will take around 5 minutes to complete.

Deploy Torq Step Runners on EKS clusters

Deploying a Step Runner in AWS EKS requires using the kubectl utility to connect to the cluster. This can either be done via the AWS CloudShell console in your browser, or through your local CLI by installing kubectl and awscli. It is recommended to use CloudShell as it already has awscli and kubectl installed with the latest versions.

If you are using an AWS user or identity profile that did not create the cluster, you will need to modify the aws-auth ConfigMap to prevent CLI login errors and allow the user to connect and use kubectl.

To create and deploy a Step Runner in EKS:

  1. Navigate to the Step Runner: In Torq, go to Integrations > Step Runner and click Add Instance.

  2. Configure the Step Runner:

    1. Enter a name that reflects the Runner's type and the deployment environment.

    2. Enter a meaningful description to provide context.

    3. Select Kubernetes.

  3. Copy the install command: Click Add and copy the install command that is generated to save it for later.

  4. Open AWS CloudShell: In the AWS Management Console, go to CloudShell.

  5. Configure kubectl to connect to the cluster:

    1. Run aws sts get-caller-identity to gather the CLI caller identity if necessary.

    2. Run aws eks update-kubeconfig --region <region code> --name <cluster name> to update the kubeconfig file for EKS access.

    3. Run kubectl get svc to test communication with the cluster. If communication is not successful, see how to modify the aws-auth ConfigMap.

  6. Execute the install command: Once communication with the cluster is successful, paste the Runner install command from Torq. This deploys the Step Runner using the generated YAML deployment configuration file that was automatically downloaded to your device.

    1. (Optional) For customized deployments, see advanced deployment settings.

  7. Confirm the installation: Return to the Step Runner page in Torq and verify that the Runner's status has changed from Pending to Healthy.

Automate EKS deployment via eksctl

The eksctl CLI tool can be used to automate the deployment and configuration of EKS clusters and node groups. This tool can be used in conjunction with awscli (for authentication) and kubectl to automate the full deployment. eksctl will also create a kubeconfig file with the information required to use kubectl with the new cluster. Like kubectl and awscli, eksctl is already installed on AWS CloudShell.

To create a cluster with eksctl:

  1. Open AWS CloudShell: Sign in to the AWS Management Console and go to CloudShell.

  2. Create a cluster: Run the command eksctl create cluster --name=<cluster name> --nodes=<number of nodes> --auto-kubeconfig --region <region code>
    The kubeconfig file's path will be listed in the output.

Related Articles
Deploy self-hosted Step Runners
Troubleshoot self-hosted Step Runners
Deploy self-hosted Step Runners on Microsoft Azure Kubernetes Service (AKS)
Customize self-hosted Step Runners: Advanced deployment settings
Did this answer your question?