Configuring AI Services

Overview

Atolio lets users ask natural-language questions against your organization’s knowledge using retrieval‑augmented generation (RAG). Atolio supports multiple LLM providers, starting with OpenAI and Azure OpenAI.

Supported providers

Prerequisites

  • A healthy Atolio deployment
  • kubectl access to the cluster
  • atolioctl downloaded from the latest release
  • jq installed (for extracting values from Kubernetes secrets)

High‑level configuration workflow

1) Port‑forward the Feeder service

kubectl -n atolio-db port-forward service/feeder 8889

Leave this running in a separate terminal while you configure keys.

2) Generate a JWT token for KV operations

# Get the JWT Secret (used to sign the token)
export jwtSecret=$(kubectl -n atolio-svc get secret lumen-secrets -o json | jq -r .data.jwtSecretKey | base64 -d)

# Change this to reflect your Atolio deployment's domain name
export domainName="https://search.example.com"

export JWT_TOKEN=$(atolioctl connector create-jwt --raw \
  --jwt-audience-sdk=${domainName} \
  --jwt-issuer-sdk=${domainName} \
  --jwt-secret-sdk=${jwtSecret} \
  "atolio:*:*:*")

Confirm JWT_TOKEN is set to a JWT value.

3) Configure the provider

Follow your provider guide to set the model slug and credentials in the KV store.

  • OpenAI: set /lumen/system/ask_model_slug to openai-gpt-4o and store the OpenAI API key.
  • Azure OpenAI: set /lumen/system/ask_model_slug to azure-openai and store the API key, endpoint, and deployment name.

4) Restart the Marvin service

kubectl -n atolio-svc delete pod -l app=marvin

Kubernetes will recreate the pod with the new configuration.

Troubleshooting

  • Startup errors: re-check the stored keys and endpoint URLs.
  • Rate limits (Azure): increase Tokens per Minute (TPM) on the deployment.
  • Connection issues: ensure port‑forwarding is active during KV operations.
OpenAI

Azure OpenAI