AWS Deployment

This document describes the deployment of the required micro services into AWS. It is intended to be followed in linear order, skipping any steps not relevant to the particular deployment.

Architecture

arch-aws

Deployment Prerequisites

In order to get started, your Atolio support team will do the following on your behalf:

  1. Grant your AWS account access to the Client ECR repos (for pulling Docker images).
  2. Add your Deployment Engineer as a collaborator to the Atolio GitHub repository (lumen-infra), which contains:
  • Deployment documentation
  • Terraform for the Atolio stack infrastructure
  • Configuration files for Atolio services
  • Maintenance scripts

The following deployment prerequisites will help streamline your deployment process.

Determine AWS account

You can either choose to deploy Atolio into an existing AWS account or a new account. Atolio also supports deploying to your own AWS Virtual Private Cloud (VPC). When the account is available, share the AWS account number with your Atolio support team.

We recommend:

  • Ensuring that Service Quotas within your AWS account allow for a minimum of 64 vCPU for On-Demand Standard instances.
  • Raising any other organizational AWS policies / restrictions (e.g. networking, containers) with your Atolio support team ahead of the deployment call.

Determine Atolio DNS name

Before the deployment call, you may want to decide on your desired Atolio web location. Create a AWS Route 53 hosted zone in the AWS account for hosting the Atolio stack (e.g. search.yourdomain.com.): this will be the DNS name (without the trailing dot) for the Atolio web application (e.g. https://search.yourdomain.com).

This hosted zone allows the deployment (i.e. the External DNS controller) to add records to link host names (e.g. search.yourdomain.com, feed.search.yourdomain.comand relay.search.yourdomain.com) to the load balancer as created by the AWS ALB controller.

For the remainder of this document, we will use https://search.yourdomain.com in the examples, but it is expected for you to replace with your own DNS name.

Creation of or using existing VPC

By default, Atolio’s Terraform code will create a VPC. However, you may choose to use an existing VPC and subnets within your AWS account. In this case, set create_vpc to false.

Then, configure vpc_id and provide the subnets using vpc_private_subnet_ids and vpc_public_subnet_ids:

# Uncomment these lines and update the values in case you want to deploy in a
# pre-existing VPC (by default a new VPC will be created).
#
# Note that automatic subnet discovery for the ALB controller will only work
# if the subnets are tagged correctly as documented here:
# https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.7/deploy/subnet_discovery/
# vpc_id                 = "vpc-000"
# vpc_cidr_block         = "10.0.0.0/16"
# vpc_private_subnet_ids = ["subnet-1111", "subnet-2222"]
# vpc_public_subnet_ids  = ["subnet-3333", "subnet-4444"]

Delegate responsibility for Atolio subdomain

The customer’s parent domain (e.g. yourdomain.com) needs to delegate traffic to the new Atolio subdomain (search.yourdomain.com). This is achieved by adding an NS record to the parent domain with the 4 name servers copied from the new subdomain (similar to what is described here). Note that this can only be done after deployment, when the hosted zone for the subdomain is created.

Setup authentication

Atolio supports single sign-on (SSO) authentication through Okta, Microsoft Entra ID, and Google using the OpenID Connect (OIDC) protocol.

Refer to Configuring Authentication for more details on the steps to complete in your desired SSO provider in order to obtain the necessary OIDC configuration values.

Local environment setup

Finally, ensure you have the following utilities installed:

Create Cloud Infrastructure

The Terraform configuration requires an external (S3) bucket to store state. A script is available to automate the whole process (including running Terraform). Before running the script, create a config.hcl file based on the provided config.hcl.template:

cd deploy/terraform/aws
cp ./config.hcl.template config.hcl

Update the copied file with appropriate values. At a minimum, it should look something like this:

# Domain name for Atolio stack (same as hosted zone name without trailing ".")
lumen_domain_name = "search.yourdomain.com"

Then copy the Helm template and update the values with the appropriate OIDC settings. You will also likely modify lumenImageTag to specify the version of Atolio you’d like to deploy. Note: the OIDC settings are necessary for the Helm release to succeed (the Marvin service is dependent on these settings for validating authentication).

cp ./templates/values-admin.yaml values-lumen.yaml
lumenImageTag: "4.9.0"

# Path to your company logo to be shown in the Atolio UI
searchUi:
  publicLogoPath: "https://search.yourdomain.com/yourLogo.svg"

jwtSecretKey: "256-bit-secret-key-for-sign-jwts"

# See also scripts/config-oidc.sh helper script to obtain some of the values below
oidc:
  provider: "add-your-provider-here"
  endpoint: "add-your-endpoint-here"
  clientId: "add-your-id-here"
  clientSecret: "add-your-secret-here"

For the jwt_secret_key any 256 bit (32 character) string can be used. It is used to sign JWT tokens used by the web application and atolioctl tool. It should be a well guarded secret that is unique to the deployment.

You should have all variables within the OIDC block configured. Now you can create the infrastructure and deploy the k8s cluster. From the ’terraform/aws’ directory:

./scripts/create-infra.sh --name=deployment-name

This will create the infrastructure in the us-west-2 AWS region. If you want to deploy in another region parameter (e.g. us-east-1) an additional parameter can be provided:

./scripts/create-infra.sh --name=deployment-name --region=us-east-1

The deployment-name argument is used to generate a deployment name for e.g. tagging resources and naming e.g. the kubernetes cluster and S3 buckets. So make sure it is unique across all deployments. (i.e. using a globally unique deployment name). Typically this is named after the customer for which the Atolio app is deployed or a particular deployment flavour (e.g. acmecorp or engtest).

The script automates the following steps (parameterized based on the provided deployment name):

  1. Create an S3 bucket to store Terraform state
  2. Create a terraform.tfvars file for Terraform
  3. Run terraform init
  4. Run terraform apply (using input variables in generated terraform.tfvars)

With the infrastructure created, you’ll want to update-kubeconfig so an updated context can be added to your local configuration:

aws --profile {atolio profile} eks update-kubeconfig --region us-west-2 --name lumen-{deployment-name}

At this point you should be able to interact with the kubernetes cluster, e.g.

kubectl get po -n atolio-svc

Note, Atolio specific services run on the following namespaces:

  • atolio-svc (Services)
  • atolio-db (Database)

When you have validated that the infrastructure is available, the next step is to configure sources.