Connectors

• 1: Atlassian Bitbucket Connector
• 2: Atlassian Confluence Connector
• 3: Atlassian Connector for Jira & Confluence Data Center
• 4: Atlassian Jira Connector
• 5: Box Connector
• 6: Datadog Connector
• 7: GitHub Connector
• 8: GitLab Connector
• 9: Gong Connector
• 10: Google Connectors
• 10.1: Gmail Connector Setup
• 10.2: Google Chat Connector Setup
• 10.3: Google Drive Connector Setup
• 10.4: Google Sites Connector Setup
• 11: HubSpot Connector
• 12: Keycloak Connector
• 13: Linear Connector
• 14: Microsoft Connectors
• 14.1: OneDrive Connector Setup
• 14.2: Outlook Connector Setup
• 14.3: SharePoint Connector Setup
• 14.4: Teams Connector Setup
• 15: Notion Connector
• 16: Okta Connector
• 17: Salesforce Connector
• 18: ServiceNow Connector
• 19: Slack Connector
• 20: SolarWinds Service Desk Connector
• 21: Staffbase Connector
• 22: Web Connector
• 23: Zendesk Connector
• 24: Zoom Connector
• 25: Zulip Connector

Note: Configuration of sources is regularly evolving. This document covers responsibilities shared by both the Deployment Engineer and Admin. As of v4.24, Admins can configure sources directly through the Admin console without requiring Deployment Engineer intervention for most configurations.

Atolio can be configured to index content across a variety of sources. This document describes the steps required in both the source systems and within Atolio:

1. First, follow the steps required in the source system (links provided below).
2. Then, configure the Atolio connector in your environment using one of the methods outlined in Atolio Configuration.

Each connector guide ends with a Required Configuration Values section that lists every value needed to configure that connector, whether you enter them yourself in the Admin console or send them to Atolio Support.

Atolio Configuration

As of v4.24, there are two methods available for configuring sources in Atolio:

Method 1: Admin Console (Recommended for v4.24+)

Admins can now configure sources directly through the Atolio Admin console without requiring access to the Kubernetes cluster. This browser-based interface provides an intuitive way to manage all available sources.

To configure sources via the Admin console:

1. Navigate to the Admin panel in Atolio
2. Select “Connectors” or “Sources” from the menu
3. Choose the source you want to configure
4. Fill in the required credentials and configuration settings through the UI
5. The configuration is immediately applied to your deployment

Advanced Configuration Options:

The Admin console provides flexible configuration management:

• YAML Editor: Edit configurations directly in an embedded YAML editor for fine-grained control
• Upload: Import existing YAML configuration files (exported from atolioctl or previously downloaded)
• Download: Export your current connector configurations as YAML files for backup, documentation, or migration to other environments

This approach allows you to seamlessly work with the same YAML configuration format that atolioctl uses, whether you prefer the form-based UI or direct YAML editing.

Method 2: atolioctl (Traditional - For Deployment Engineers)

For users who prefer command-line configuration or need advanced options, atolioctl remains available. This terminal-based UI (TUI) allows your Deployment Engineer to configure all available sources from the command line.

Installing atolioctl

Atolio provides an installer script within the lumen-infra repository that will automatically grab the latest version of atolioctl and install it locally.

Note: The installer script requires that you have curl or wget installed:

• curl
• wget

1. Navigate to the lumen-infra repository and into the scripts directory

cd lumen-infra/deploy/terraform/scripts

2. Run the installer script

./get-atolioctl.sh

3. From there, you should be able to run the tool: atolioctl --help

You can re-run this script to download the latest version of atolioctl or run the script with the --version flag to install a specific version of atolioctl, ex:

./get-atolioctl.sh --version 4.10.2

Deploying Connectors

Prior to configuring a source, the connector (ie the deployment in which the source will run on) must be deployed. Atolio uses the concept of instances to support one or more instances of a source (eg multiple tenants), and the creation of these deployments is managed by modifying values-lumen.yaml. Specify connectors like so:

connectors:
  microsoft:
    default:
      enabled: true
  okta:
    default:
      enabled: true
  jiracloud:
    default:
      enabled: true

Connectors are deployed under a default instance. This is an identifier which can be changed as desired, and multiple connectors can be deployed under different instances. Once the instance is set, it should be treated immutable. Modifying this value will result in loss of data and a need to re-index under the new instance. This is due to the instance being part of every resource identifier ingested by Atolio.

  jiracloud:
    internal:
      enabled: true
    external:
      enabled: true

Customizing Connector Deployments

The above YAML examples are the minimal definitions required to deploy a connector to your Atolio cluster. There are additional properties that can be set in your values-lumen.yaml file as directed by your Atolio support team to apply different options. For example:

connectors:
  google:
    default:
      enabled: true
      lumenImageTag: 4.13.0-alpha1
      log:
        level: "debug"
      resources:
        requests:
          cpu: 500m
          memory: 2Gi
        limits:
          cpu: 1000m
          memory: 4Gi

To provision the updates into the cluster, you’d upgrade the lumen Helm chart by running create-infra.sh (recommended) or:

helm upgrade lumen-chart ../../k8s/charts/lumen --values values-lumen.yaml --values values-lumen-templated.yaml -n atolio-svc

Configuring Sources via atolioctl

This section details how to configure sources using atolioctl, including how to obtain the necessary JWT. atolioctl can be obtained through the Releases page.

Tip: Starting in v4.24, most users should use the Admin console method above instead of atolioctl. The Admin console provides a user-friendly browser interface without requiring command-line access.

First an API token needs to be created for use with atolioctl. This token can be generated as follows using the config.hcl file that was used to run the create-infra.sh script:

export JWT_TOKEN=$(atolioctl connector create-jwt --raw --config-dir . "atolio:*:*:*")

Feeder exposed on Load Balancer

To run configuration, provide the configure command to atolioctl. Make sure to use the correct API host name for your deployment. Once loaded, you should see a collection of configuration options and sources in your terminal:

atolioctl configure --jwt-token-sdk ${JWT_TOKEN} --feeder-address feed.search.example.com:443

Port Forwarding (Feeder not exposed on Load Balancer)

To run configuration, port-forward 8889 from service/feeder and provide the configure command to atolioctl. Ensure the correct AWS profile is in context. Once loaded, you should see a collection of configuration options and sources in your terminal:

kubectl port-forward -n atolio-svc service/feeder 8889
atolioctl configure --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Note that for the time being all atolioctl commands below (unless noted otherwise) will require the port forwarding command.

Configuration Tips

To obtain source-specific information, head to source configuration and find the source you’re interested in configuring. Some tips when running atolioctl:

• Run the application in a maximized window to ensure full visibility of all information.
• Press ? to toggle the help menu / key map. You will typically use up/down arrows and <TAB> to navigate the menus.
• No data is persisted until Review & Save is selected and validation has completed. A report will be shown detailing which sources were successfully configured and which not.

YAML Configuration Format

Atolio uses YAML format for source configurations. This format is used consistently across both the Admin console and atolioctl, allowing you to easily switch between methods or export/import configurations.

YAML export/import with atolioctl

As a reminder, the stack is not yet running at this point. Configuration of most sources does not require an operational stack as the validation is simply making a call to the source API and there is no expectation of the source’s connector running yet. There are some exceptions (in particular Jira, Confluence, and BitBucket)

Once desired sources have been configured and the stack deployed, you can import the pre-configured file using atolioctl. Note at this point it’s expected Feeder is operational and port-forwarding to the service address can occur:

atolioctl configure import atolio.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889
Writing key: /lumen/connectors/github/default/github
Writing key: /lumen/connectors/linear/default/linear
Writing key: /lumen/connectors/notion/default/notion
Writing key: /lumen/connectors/slack/default/slack
Writing key: /lumen/connectors/web/default/web
Writing key: /lumen/connectors/bitbucket/default/bitbucket

Similarly, source configuration of the stack can be exported to a local file. This can be useful if wishing to view the entire configuration in a human-readable YAML file, or perhaps store a backup of the configuration in a vault like 1Password. This file will be in the same format as the one imported in the prior step, allowing a loop of export, modify, import:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 atolio.yaml

Both export and import commands can also combined into a single edit command that exports the current source configuration into a text editor (specified by the EDITOR environment variable) allowing manual edits. Upon saving/closing the editor, the configuration will be imported. This command is invoked a follows:

atolioctl configure edit --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Common Fields: Resources

Atolio supports the ability to avoid certain content from being indexed at all. The example below shows how to specifically exclude a Jira project (with the project key “SP”).

When configuring the Jira source, enter the following in the Resources field:

project:
    excluded:
        - "SP" # The Secret Project

The excluded keyword contains a list of Jira project keys to be excluded from indexing. Similarly you can also use the included keyword to achieve the opposite (in that case only those resources that are specifically listed will be indexed, nothing else). Note that the list entries (e.g. SP in the example above) are case sensitive.

In the absence of both included (opt-in) or excluded (opt-out), everything that the credentials (e.g. API key) have access to will be indexed. Note that this is a layer on top of the permissions model provided to the source. Anything added to included will only be indexed if the underlying credentials grant the indexing job access.

This feature is supported by a majority of Atolio connectors, see each connectors respective documentation for details and scope of inclusions / exclusions.

Source Configuration

1 - Atlassian Bitbucket Connector

The Atlassian Bitbucket connector indexes repositories, projects, pull requests, commits, and comments across your Bitbucket instance.

The connector supports Atlassian Bitbucket Cloud.

Installing the Bitbucket App Descriptor

In order to target your specific Atolio deployment, the Atlassian Bitbucket connector requires a private listing to be installed instead of the Atlassian marketplace version. This can be disabled again after the connect app is installed.

In your Bitbucket instance:

1. Under Settings -> Workplace settings, navigate to Installed apps under the APPS AND FEATURES section.
2. After clicking Enable development mode, click the link for Install app from URL.

3. Specify the URL provided by your Deployment Engineer. It will be similar to the following with your chosen DNS name:

• https://relay.search.example.com/connectors/bitbucketcloud/default/install/atlassian-connect.json
• Note: if you have multiple Bitbucket instances, please make sure to provide the correct instance name in the URL above (i.e. default in the example).

Once the Atolio Connect app has been installed, the configuration will be stored securely in your Atolio deployment. You can now disable the development mode as needed.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the app descriptor install URL (https://relay.search.example.com/connectors/bitbucketcloud/default/install/atlassian-connect.json), used for installing the Atolio app from URL in step 3.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  bitbucket:
    default:
      bitbucket:
        connector: bitbucket
        instance: default
        source: bitbucket
        common:
          enabled: true
          identity-provider: false
        secrets: {}
        configuration:
          workspace: "<WORKSPACE_SLUG>"

2 - Atlassian Confluence Connector

The Atlassian Confluence connector indexes spaces, pages, and comments across your Confluence instance.

The connector supports Atlassian Confluence Cloud.

Installing the Confluence app

Atolio connects to Confluence Cloud through an Atlassian Forge app that is unique to your deployment. A site administrator installs it once with the Forge CLI; afterwards Atolio manages the connection and its API credentials automatically.

Prerequisites

• Administrator access to your Confluence Cloud site.
• Node.js and the Forge CLI: npm install -g @forge/cli.
• Your deployment’s connector URL, provided by your Deployment Engineer (for example https://relay.search.example.com/connectors/confluencecloud/default).
• If you set up this connector on an Atolio version earlier than 4.24 as a “Connect Module”, uninstall the previous Atolio Confluence app first (in Confluence: Apps → Manage your apps).

Steps

1. Confirm with your Deployment Engineer that the connector is deployed and reachable at the URL above. Empty/partial configuration is okay at this point.

2. Create a new empty folder for the app, and download your deployment’s pre-filled manifest into it as manifest.yml.

   mkdir atolio-confluence-app && cd atolio-confluence-app
   curl -o manifest.yml https://relay.search.example.com/connectors/confluencecloud/default/forge-manifest.yml
   

   If you run more than one Confluence instance, use the matching instance name in the URL (default above).

3. Log in:

   forge login
   

   It will prompt you to provide an email, and then a token. If no token was created before, visit the link displayed by the CLI, press “Create API token with scopes” button, select “Forge” app, and continue.

4. Register the Forge app:

   forge register "Atolio Confluence Connector"
   

   It will prompt you to select a “Developer Space” to host the app. If no developer space was created before, provide a name such as “Your-Company Developer Space” to create a new one, and continue.

   Note: This step should be executed only once, otherwise a duplicate with the same name would be registered. See Troubleshooting for instructions to delete accidental duplicates.

5. Deploy and install it on your site:

   forge deploy -e production
   forge install -e production --site your-site.atlassian.net --product confluence
   

Once installed, the app registers itself with your Atolio deployment and indexing begins automatically. The connection is stored securely in Atolio — there is nothing further to configure.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the connector URL (https://relay.search.example.com/connectors/confluencecloud/default) and the pre-filled manifest URL (https://relay.search.example.com/connectors/confluencecloud/default/forge-manifest.yml), used for installing the Confluence app.

To limit which Confluence spaces are indexed, see (Optional) Scope Confluence indexing.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  confluencecloud:
    default:
      confluencecloud:
        connector: confluencecloud
        instance: default
        source: confluencecloud
        common:
          enabled: true
          identity-provider: false
        secrets: {}
        configuration:
          baseurl: "<SITE_URL>"

(Optional) Scope Confluence indexing

If you want fine-grained control over which Confluence spaces should be indexed, provide a list of inclusions or exclusions to your Deployment Engineer: they can use the includes or excludes feature in Atolio configuration to control the relevant spaces that should be indexed.

Troubleshooting

Multiple Atolio Confluence apps are displayed on app management page

This likely means forge register step above is executed more than once. To fix it:

1. Open the developer console. It lists every app you have registered, along with each app’s ID.

2. Ask your Deployment Engineer which app your connector deployment is bound to.

3. Keep that app. For each of the others, uninstall it from your site:

   forge uninstall --app-id-override <app-id>
   

   Use the app ID as shown in the developer console, not the full ari:cloud:ecosystem::app/... value.

4. Once an app has no installations left, delete it from the developer console.

3 - Atlassian Connector for Jira & Confluence Data Center

The Atlassian Connector for Jira & Confluence Data Center indexes:

• Projects, issues, comments, and attachments across your Jira instance.
• Spaces, pages, blog posts, comments, and attachments across your Confluence instance.
• Users and groups in Jira & Confluence for the purpose of understanding permissions.

The connectors authenticate to Jira and Confluence as a dedicated service user using a Personal Access Token (PAT). They run as part of your Atolio deployment — no plugin or app needs to be installed in Jira or Confluence.

Note: Earlier releases of this connector shipped as Atlassian SDK plugins (jira-dc-connector-*.jar and confluence-dc-connector-*.jar) installed via Manage apps. Clients are encouraged to migrate to the new connectors described below.

Create a Service User

In each of Jira DC and Confluence DC, create a dedicated user for Atolio to authenticate as (for example, atolio-connector-user).

You have two options for the permission scope you grant this user:

Option 1 — Administrator access (recommended for easy setup). Grant the service user administrator membership:

• In Jira, add the user to the Jira Administrators group (or an equivalent admin role).
• In Confluence, add the user to the Confluence Administrators group (or grant System Administrator).

With admin access, the connector automatically registers the webhook it needs for live updates on startup. No further setup is required in Jira or Confluence.

Option 2 — Read-only access. Grant the service user only the permissions required to read the content you want indexed:

• In Jira, grant Browse Projects on every project to be indexed (the easiest way is to add the user to the default permission scheme).
• In Confluence, grant View permission on every space to be indexed (the easiest way is to add the user to the default space permission scheme).

Read-only access means the connector cannot register its own webhook. You can either:

• Skip webhook registration entirely — the connector will still backfill and periodically resync, but changes in Jira and Confluence will not be reflected live, or
• Register the webhook manually (see Manual Webhook Setup below).

Create a Personal Access Token

For each of Jira and Confluence:

1. Log in as the service user.
2. Go to Profile → Personal Access Tokens → Create token.
3. Give the token a name (e.g. atolio-connector-token) and set an expiry if required by your policy.
4. Copy the token. You’ll provide it as described in Required Configuration Values.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provide one set of values for each connector instance (Jira and Confluence are configured separately).

You can optionally restrict indexing to specific projects (Jira) or spaces (Confluence) via the resources field — see Common Fields: Resources.

Provided by Atolio: the callbackBaseURL, used for Manual Webhook Setup.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  jiradc:  # Only if indexing Jira
    default:
      jiradc:
        connector: jiradc
        instance: default
        source: jiradc
        common:
          enabled: true
          identity-provider: false
        secrets:
          personal_access_token:
            value: "<JIRA_PERSONAL_ACCESS_TOKEN>"
        configuration:
          base-url: "<JIRA_BASE_URL>"
          disable-webhook-registration: true  # Only if the service user is read-only or you manage the webhook yourself
  confluencedc:  # Only if indexing Confluence
    default:
      confluencedc:
        connector: confluencedc
        instance: default
        source: confluencedc
        common:
          enabled: true
          identity-provider: false
        secrets:
          personal_access_token:
            value: "<CONFLUENCE_PERSONAL_ACCESS_TOKEN>"
        configuration:
          base-url: "<CONFLUENCE_BASE_URL>"
          disable-webhook-registration: true  # Only if the service user is read-only or you manage the webhook yourself

If you remove the jiradc or confluencedc line, remove everything indented under it too.

Manual Webhook Setup

If the service user does not have admin permissions, or if you prefer to manage webhooks yourself, register the webhook manually after the connector is running. Set disable-webhook-registration: true in the connector configuration to prevent the connector from attempting to register one.

Jira

1. In Jira, go to Administration → System → WebHooks → Create a WebHook.
2. Set the URL to {callbackBaseURL}/connectors/jiradc/{instance}/callback, where {instance} is the connector instance name (e.g. default). Your Deployment Engineer can provide the correct callbackBaseURL.
3. Enable the following events:
   • jira:issue_created, jira:issue_updated, jira:issue_deleted
   • comment_created, comment_updated, comment_deleted
   • project_created, project_updated, project_deleted

Confluence

1. In Confluence, go to Administration → General Configuration → Webhook → Create webhook.
2. Set the URL to {callbackBaseURL}/connectors/confluencedc/{instance}/callback, where {instance} is the connector instance name (e.g. default). Your Deployment Engineer can provide the correct callbackBaseURL.
3. Enable the following events:
   • page_created, page_updated, page_removed, page_trashed, page_restored
   • blog_created, blog_updated, blog_removed, blog_trashed, blog_restored
   • comment_created, comment_updated, comment_removed
   • space_created, space_updated, space_removed

4 - Atlassian Jira Connector

The Atlassian Jira connector indexes projects, issues, and comments across your Jira instance.

The connector supports Atlassian Jira Cloud.

Installing the Jira app

Atolio connects to Jira Cloud through an Atlassian Forge app that is unique to your deployment. A site administrator installs it once with the Forge CLI; afterwards Atolio manages the connection and its API credentials automatically.

Prerequisites

• Administrator access to your Jira Cloud site.
• Node.js and the Forge CLI: npm install -g @forge/cli.
• Your deployment’s connector URL, provided by your Deployment Engineer (for example https://relay.search.example.com/connectors/jiracloud/default).
• If you set up this connector on an Atolio version earlier than 4.24 as a “Connect Module”, uninstall the previous Atolio Jira app first (in Jira: Apps → Manage your apps).

Steps

1. Confirm with your Deployment Engineer that the connector is deployed and reachable at the URL above. Empty/partial configuration is okay at this point.

2. Create a new empty folder for the app, and download your deployment’s pre-filled manifest into it as manifest.yml.

   mkdir atolio-jira-app && cd atolio-jira-app
   curl -o manifest.yml https://relay.search.example.com/connectors/jiracloud/default/forge-manifest.yml
   

   If you run more than one Jira instance, use the matching instance name in the URL (default above).

3. Log in:

   forge login
   

   It will prompt you to provide an email, and then a token. If no token was created before, visit the link displayed by the CLI, press “Create API token with scopes” button, select “Forge” app, and continue.

4. Register the Forge app:

   forge register "Atolio Jira Connector"
   

   It will prompt you to select a “Developer Space” to host the app. If no developer space was created before, provide a name such as “Your-Company Developer Space” to create a new one, and continue.

   Note: This step should be executed only once, otherwise a duplicate app with the same name would be registered. See Troubleshooting for instructions to delete accidental duplicates.

5. Deploy and install it on your site:

   forge deploy -e production
   forge install -e production --site your-site.atlassian.net --product jira
   

Once installed, the app registers itself with your Atolio deployment and indexing begins automatically. The connection is stored securely in Atolio — there is nothing further to configure.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the connector URL (https://relay.search.example.com/connectors/jiracloud/default) and the pre-filled manifest URL (https://relay.search.example.com/connectors/jiracloud/default/forge-manifest.yml), used for installing the Jira app.

To limit which Jira projects are indexed, see (Optional) Scope Jira indexing.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  jiracloud:
    default:
      jiracloud:
        connector: jiracloud
        instance: default
        source: jiracloud
        common:
          enabled: true
          identity-provider: false
        secrets: {}
        configuration:
          baseurl: "<SITE_URL>"

(Optional) Scope Jira indexing

If you want fine-grained control over which Jira projects should be indexed, provide a list of inclusions or exclusions to your Deployment Engineer: they can use the includes or excludes feature in Atolio configuration to control the relevant projects that should be indexed.

Troubleshooting

Multiple Atolio Jira apps are displayed on app management page

This likely means forge register step above is executed more than once. To fix it:

1. Open the developer console. It lists every app you have registered, along with each app’s ID.

2. Ask your Deployment Engineer which app your connector deployment is bound to.

3. Keep that app. For each of the others, uninstall it from your site:

   forge uninstall --app-id-override <app-id>
   

   Use the app ID as shown in the developer console, not the full ari:cloud:ecosystem::app/... value.

4. Once an app has no installations left, delete it from the developer console.

5 - Box Connector

The Box connector indexes files, folders, comments, tasks, and web links in Box, along with the users and groups that define who can access them.

Setup takes three steps:

1. Create the Box app.
2. Authorize it for your enterprise.
3. Provide the connector its credentials (see Required Configuration Values).

Create the Box Application

Create a Custom App using Server Authentication (Client Credentials Grant) in the Box Developer Console (admin or co-admin account). In the app’s Configuration tab:

• App Access Level: App + Enterprise Access — required to enumerate and impersonate managed users and groups.
• Application Scopes: Read all files and folders stored in Box; Manage users; Manage groups; Manage enterprise properties.
• Advanced Features: Generate user access tokens; Make API calls using the as-user header.

Save changes.

Authorize the Application

A CCG app cannot call the API until an admin authorizes it: in the Box Admin Console → Apps → Custom Apps Manager, add/authorize the app by its Client ID and confirm the scopes. Until then the connection test fails with an authorization error.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  box:
    default:
      box:
        connector: box
        instance: default
        source: box
        common:
          enabled: true
          identity-provider: false
        secrets:
          client_id:
            value: "<CLIENT_ID>"
          client_secret:
            value: "<CLIENT_SECRET>"
          enterprise_id:
            value: "<ENTERPRISE_ID>"
        configuration: {}

Reference: Configuration Parameters

Only the three credentials above are required. Every other field has a safe default. The three credentials go under secrets:; the other parameters go under configuration:.

Note: Leave the rate-limit, concurrency, cache, and event-polling knobs at their defaults unless tuning throughput, memory, event latency, or responding to 429 pressure.

Note: The backfill window only scopes what backfill emits — it does not limit Stream(), which always processes new events as they arrive.

Filtering Indexed Resources

By default everything reachable is indexed. To narrow scope, map a resource key to an included (allowlist) and/or excluded (blocklist).

configuration:
  # ... credentials ...
  resources:
    user:
      excluded:
        - svc-indexer@example.com
    acl_group:
      included:
        - "engineering"
        - "admins"

Semantics: no filters → index all; included → only listed entries; excluded → all but listed entries; both → included minus excluded (exclude wins). Match values are compared for exact equality.

Match values are the native Box identifiers (not Atolio xIDs), since filtering happens during enumeration before any xID is assigned:

Filters are enforced for the identity resources (user, acl_group), which are backfilled first because the rest of the connector’s ACLs depend on them.

Indexed Resources

6 - Datadog Connector

The Datadog connector enables the indexing of Datadog dashboards and notebooks.

Indexed dashboards and notebooks respect Datadog’s granular access controls: if a dashboard or notebook is restricted, it is only searchable in Atolio by the users, teams, roles or organizations that are granted viewer or editor access to it in Datadog. Unrestricted dashboards and notebooks are searchable by all Datadog users in your account.

What Is Indexed

Create API and Application Keys

The connector authenticates with a Datadog API key and application key.

1. In Datadog, navigate to Organization Settings → API Keys and create a new API key, e.g. Atolio. Copy the key value.
2. Navigate to Organization Settings → Application Keys and create a new application key, e.g. Atolio. Copy the key value; it is only shown once.

An application key acts with the permissions of the user who created it. Ideally, create it as a service user who can read users, teams, roles, dashboards, notebooks and their restriction policies, such as a user with the Datadog Admin role.

If you scope the application key, it must still allow read access to all of the above. Leaving the application key unscoped is the simplest option.

Identify Your Datadog Site

Datadog accounts are hosted on one of several Datadog sites. You can identify yours from the URL you use to log in to Datadog:

If your account is on the default datadoghq.com site, no site needs to be configured.

Provide Configuration

Provide the following values to your Deployment Engineer for them to configure in Atolio:

When the configuration is saved, Atolio validates the API and application keys against the Datadog API.

Limitations

• A dashboard or notebook is skipped if its creator’s Datadog user cannot be found (for example, the creator has been removed from the account), or if its restriction policy cannot be read.
• Monitors, logs, metrics, SLOs and other Datadog resources are not currently indexed.

7 - GitHub Connector

The GitHub connector enables the indexing of pull requests, commits, comments, and markdown in your GitHub repositories.

Create Access Token

To install the GitHub connector, use a GitHub account with admin access to the organization (requires read access to teams). Using this account, create a personal access token (currently Classic only) according to these instructions. You will need to ensure repo and user scopes are set (as shown below):

Note: for GitHub Enterprise Server installations, you would need to enable the site_admin scope too.

(Optional) Create GitHub App

Note: GitHub App flow does not work with GitHub Enterprise Server installations. You would need to use a personal access token like above

Instead of using a personal access token, a GitHub app can be created to install the GitHub connector. This provide better rate limits, especially if your org is on Enterprise Cloud. You can create a GitHub app according to these instructions.

You will need to ensure the following app permissions are set:

Repository Permissions:

Organization Permissions:

Setup Webhooks

If live indexing of GitHub content is desired, you will want to setup Webhooks. Go to Settings->Webhooks->Add Webhook. After authentication, you can now setup the webhook with the following configs:

The following events are indexed at the repository level:

• Collaborator add, remove, or changed
• Commit comments
• Issues
• Issue comments
• Pull requests
• Pull requests reviews
• Pull requests reviews comments
• Pushes
• Repositories

The following events are indexed at the organization level:

• Team

Please manually check the boxes for each of the events above.

Warning: Setting up webhooks at the organization level will send webhook events for all repositories within the organization. If this is not desireable, please make changes to the configuration to only process and index specific repositories.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Use either a personal access token or a GitHub App. GitHub Enterprise Server supports only the personal access token.

Provided by Atolio: the webhook Payload URL (https://search.example.com/callback/github/, with your own DNS name), used for Setup Webhooks.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  github:
    default:
      github:
        connector: github
        instance: default
        source: github
        common:
          enabled: true
          identity-provider: false
        secrets:
          token:  # Required field; with a GitHub App setup, ask Atolio Support what to enter
            value: "<PERSONAL_ACCESS_TOKEN>"
          app_private_key:  # Only if using a GitHub App
            value: "<GITHUB_APP_PRIVATE_KEY>"
          webhook_secret:  # Only if using webhooks
            value: "<WEBHOOK_SECRET>"
        configuration:
          org-name: "<ORGANIZATION_NAME>"
          application-id: <GITHUB_APP_ID>  # Only if using a GitHub App
          installation-id: <INSTALLATION_ID>  # Only if using a GitHub App
          saml-authentication-level: "<SAML_AUTHENTICATION_LEVEL>"  # Only if SAML SSO is enabled
          enterprise-slug: "<ENTERPRISE_SLUG>"  # Only if the SAML authentication level is enterprise
          enterprise-base-url: "<ENTERPRISE_BASE_URL>"  # Only for GitHub Enterprise Server
          username-map:  # Only on github.com without SAML
            "<GITHUB_USER_ID>": "<USER_EMAIL>"

application-id and installation-id are numbers, so enter them without quotes.

(Optional) Filter Repositories and Files

Note: If you have many GitHub repositories, it may be worth using the includes or excludes feature in Atolio configuration to control the relevant repositories that should be indexed.

The GitHub connector provides two types of inclusion/exclusions in the resources field of the configuration as illustrated in the example below:

resources:
    repository:
        included:
            - lumen-infra
    file:
        included:
            - .md

The repository name lumen-infra in this example is the repo part in the repository’s URL (which has the following generic format: https://github.com/repos/{owner}/{repo}). The example above will only index the lumen-infra repository.

File filtering is done of the basis of file extension names (including the preceding dot .). The example above will configure the connector to index all markdown files. Note that by default file filtering is completely disabled as it is a relatively time consuming process.

(Optional) Configure GitHub user mappings

If employees have brought pre-existing GitHub accounts to their company’s GitHub organization, the identity resolver cannot automatically map them to company user identities. For this reason, the GitHub configuration contains a user mapping table. For example, the entries below map two GitHub user IDs to the user’s username (i.e. email address) in the organization:

12345678:  user1@example.com # "user1"
87654321:  user2@example.com # "user2"

The numerical GitHub ID can easily be obtained using the following: https://api.github.com/users/{USERNAME} in a browser where {USERNAME} is replaced with the GitHub username which can be found in the user’s profile (under the full name printed below the user’s avatar). The ID is in the id field of the JSON response. Adding an entry like this for every user will allow Atolio to link the GitHub user IDs to other sources.

8 - GitLab Connector

The GitLab connector enables indexing of merge requests, commits, and comments across your GitLab projects. Atolio supports GitLab Server (minimum version v16.5-ee) and GitLab Cloud.

Create Access Token

As an admin, perform the following steps:

• Login as admin user
• Go to https://gitlab.company.com/-/user_settings/personal_access_tokens
• Click “Add New Token”
• Give it a name and expiration date, set scope as read_api
• Copy generated token for Required Configuration Values

Setup Webhooks

If live indexing of GitLab content is desired, you will want to setup Webhooks. If you have GitLab Premium you can set up webhooks for groups. If you don’t have GitLab Premium, you need to set up a webhook for each project separately.

• Go to Settings/Webhooks
• Set URL to https://relay.{search.company.com}/connectors/gitlab/{instance}/callback/gitlab
• Set a strong “secret token”
• Check “Comments”, “Issue Events” and “Merge Request Events” under Triggers
• Keep “Enable SSL verification” checked

You also need to set up system hooks for the GitLab instance. This is done by an admin user and is required for getting live project updates.

• Go to Admin Area/System Hooks
• Set URL to the same as the webhook URL
• Set secret token as the same as the webhook secret token
• Keep “Enable SSL verification” checked

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the webhook and system hook URL (https://relay.{search.company.com}/connectors/gitlab/{instance}/callback/gitlab), used for Setup Webhooks.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  gitlab:
    default:
      gitlab:
        connector: gitlab
        instance: default
        source: gitlab
        common:
          enabled: true
          identity-provider: false
        secrets:
          token:
            value: "<ACCESS_TOKEN>"
          webhook_secret:  # Only if using live indexing
            value: "<WEBHOOK_SECRET>"
        configuration:
          base-url: "<BASE_URL>"  # Only for self-hosted GitLab
          resources:  # Only for gitlab.com
            group:
              included:
                - "<GROUP_TO_INDEX>"
          username-map:  # Only for gitlab.com
            "<GITLAB_USER_ID>": "<USER_EMAIL>"

(Optional) Filter Groups

Note: If you have many GitLab projects, it may be worth using the includes or excludes feature in Atolio configuration to control the relevant projects that should be indexed.

An example inclusion/exclusion configuration will look like the following:

resources:
  group:
    included:
      - examplegroup # Index "examplegroup" from GitLab

GitLab SaaS (gitlab.com) specific configuration

If you’re using the SaaS GitLab product hosted at gitlab.com instead of a self-hosted installation you’ll need to follow the above steps to provide an inclusion filter for at least one group you’d like to index. This is necessary to prevent the connector from attempting to index the entirety of the public SaaS GitLab deployment.

Configuring GitLab user mappings

If people are using their company emails in GitLab, the identity resolver cannot automatically map them to company user identities. For this reason the GitLab configuration contains a user mapping table. For example, the entries:

'12345678':  user1@example.com
'87654321':  user2@example.com

Map two GitLab user IDs to the user’s username (i.e. email address) in the organization. The numerical GitLab ID can easily be obtained using the following URL in a browser where USERNAME is replaced with the GitLab username. The ID is in the id field of the JSON response. On the command this can be obtained by using curl and jq as follows (using the user ID USERNAME in this example):

curl 'https://gitlab.company.com/api/v4/users?access_token=ACCESS_TOKEN&search=USERNAME' | jq '.[0].id'

Adding an entry like this for every user will allow Atolio to link the GitLab user IDs to other sources.

9 - Gong Connector

The Gong connector indexes calls, call transcripts, and keyword trackers in Gong, along with the users and permission profiles that define who can see each call.

Setup takes 2 steps:

1. Create a Gong API key.
2. Provide the connector its credentials and API base URL (see Required Configuration Values).

Create the API Key

Only a Gong user with the Tech admin role can manage API keys.

1. In Gong, select Admin center from the left sidebar.
2. On the Settings tab, under Ecosystem, select API.
3. Select + Get API key.
4. Enter a Key name, for example Atolio. Optionally, set:
   • TTL (days): when the key expires. Leave it empty for a key that does not expire. An expired key stops the connector, so if you set one, plan to rotate the key before it lapses.
   • Trusted IPs: the addresses allowed to use the key. If you set this, include the egress IP addresses of your Atolio deployment, or the connector’s requests are rejected.
5. Copy the Access Key and Access Key Secret. Gong shows the secret only once, and it cannot be retrieved after you close the dialog.

The same page shows your tenant’s API base URL (for example https://us-12345.api.gong.io). The connector needs this value because each Gong tenant is served from its own regional host.

Note: The API page also offers Create Integration, for OAuth apps. The connector does not use OAuth; use + Get API key. If you do not see + Get API key, check that your Gong user has the Tech admin role.

Note: The connection test lists workspaces and fails if none are returned, or if a configured workspace is not found. If it fails with an authorization error, check that the access key is still active and that base-url matches the host shown on the API page.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  gong:
    default:
      gong:
        connector: gong
        instance: default
        source: gong
        common:
          enabled: true
          identity-provider: false
          cron-spec: "*/30 * * * *"
        secrets:
          access-key:
            value: "<ACCESS_KEY>"
          secret-access-key:
            value: "<ACCESS_KEY_SECRET>"
        configuration:
          base-url: "<API_BASE_URL>"
          workspaces:  # Only if indexing some workspaces
            - "<WORKSPACE_NAME>"

The cron-spec sets how often the connector syncs. See Scheduling Syncs.

Reference: Configuration Parameters

Only the two credentials and the base URL are required. Every other field has a safe default. The two credentials go under secrets:; the other parameters go under configuration:.

Unset or 0 values use the default. processing-delay-mins must be shorter than lookback-mins, and lookback-mins shorter than cache-ttl-hours.

Note: Leave the optional settings at their defaults unless you are tuning freshness or API usage.

Scheduling Syncs

Gong does not provide change events, so the connector runs as a scheduled backfill. Set common.cron-spec to control how often it runs, e.g. */30 * * * * to sync every 30 minutes. Without a cron-spec, the connector runs a single backfill at startup and does not pick up new calls afterwards.

The first run indexes calls from the last fill-from-days. Each later run picks up new calls, and re-indexes recent calls that changed in Gong. Transcripts are indexed together with their calls.

Limiting Workspaces

Gong workspaces separate teams or business units within one Gong tenant, and each has its own calls, trackers, and permission profiles. By default the connector indexes every workspace. To index only some of them, list their names under workspaces:

configuration:
  # ... credentials ...
  workspaces:
    - "North America"
    - "EMEA"

Names are matched case-insensitively; workspace IDs are also accepted. An unknown name stops the connector, and the error lists the available workspaces.

Note: The Gong connector does not currently support per-resource included/excluded filters. Use workspaces to narrow scope.

Permissions

Atolio mirrors Gong’s call visibility rules:

• Private calls are visible only to the call’s owner and its internal participants.
• Other calls are visible to the owner and participants, plus anyone whose Gong permission profile grants access to them: all calls, their own calls, calls of their reports, or calls of their manager’s team.

Indexed Resources

10 - Google Connectors

The Google connector enables the indexing of:

• Documents, Sheets, Slides, and other Files in Google Drive
• Email in Gmail
• Pages in Google Sites
• Spaces and messages in Google Chat
• Users and Groups as metadata in your Google Workspace to enable user mapping / permissions

Content and updates are streamed as changes are detected and processed in your Google Workspace.

Step 0: Create Terraform Service Account for Google

Note: Depending on installation being BYOK, Managed, or otherwise this step may be skipped. Consult with an Atolio Support Engineer for any clarification.

Terraform will need a service account to make changes to your GCP project. We will grant it permissions which will essentially allow it to create, read, and delete all project resources. As such, you should keep the credentials for this account safe and not reuse them. The Terraform scripts create a much more restricted service account for normal operation.

1. Visit the IAM Service Accounts page https://console.cloud.google.com/iam-admin/serviceaccounts and select the project created previously to set up the Terraform service account.
2. Click CREATE SERVICE ACCOUNT
3. Choose a name. While it can be anything, the name “Atolio Terraform” is recommended.
4. Select CREATE AND CONTINUE.
5. Grant the following roles: “Owner”, “Security Center Admin”, “Project IAM Admin” (without conditions).
6. Click DONE. Skip step 5, you will not need to grant other users access to this service account.

Now the service account is created we can create API keys:

1. Navigate to https://console.cloud.google.com/iam-admin/serviceaccounts and select the project created previously.
2. Click the service user that was created in the previous step.
3. Select the KEYS tab click the ADD KEY pull-down menu
4. Select Create new key and use JSON for key type. Click CREATE.

This will create another JSON file with the API key and related information. This file will be needed by your Deployment Engineer (in their deploy/terraform directory).

In order for Terraform to programmatically make modifications, you need to manually enable the Google Cloud Resource Manager API.

1. Navigate to https://console.cloud.google.com/apis/library
2. Search for “Cloud Resource Manager API”
3. Select “Cloud Resource Manager API”. Click ENABLE.

The project will use additional APIs, but now that this API is available to Terraform, the Atolio deployment scripts will use Terraform to enable those APIs automatically.

Step 1: Grant Service Account Permissions

The Google Workspace domain needs to grant permission to the integration’s default service account to perform operations. Unfortunately Terraform can’t automate this, so it has to be performed manually.

Your Deployment Engineer will need to make note of the unique ID for the default App Engine service account and the list of scopes. Be aware that this ID is of the default service account, typically named App Engine default service account. It is not the Service Account you’ve previously created.

And the list of scopes are:

https://www.googleapis.com/auth/gmail.readonly,https://www.googleapis.com/auth/calendar.readonly,https://www.googleapis.com/auth/calendar.acls.readonly,https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/admin.directory.group.readonly,https://www.googleapis.com/auth/admin.directory.domain.readonly,https://www.googleapis.com/auth/drive.readonly,https://www.googleapis.com/auth/drive.activity.readonly,https://www.googleapis.com/auth/chat.spaces.readonly,https://www.googleapis.com/auth/chat.messages.readonly,https://www.googleapis.com/auth/chat.memberships.readonly,https://www.googleapis.com/auth/ediscovery,https://www.googleapis.com/auth/devstorage.read_only

Grant every scope in that list. Each source requests only the scopes it needs, in its own token exchange, and Google refuses that exchange outright when the grant is missing any one of them — so a gap takes down every source sharing the exchange, not only the feature the scope belongs to. Gmail asks for calendar.readonly and calendar.acls.readonly alongside gmail.readonly, for example, so an ungranted calendar scope stops mail indexing too.

The scopes cover all five Google sources: mail and calendar, Drive, Chat, Sites, and the directory. Granting the full list keeps a later source from failing when you enable it.

We now need to set up Domain-wide Delegation:

1. Navigate to the Google Admin console: https://admin.google.com/ac/owl. An account with admin access is required.
2. Scroll down and click “MANAGE DOMAIN WIDE DELEGATION”.
3. Click Add new and enter your service account client ID (obtained from your Deployment Engineer in the previous step). No overwrite.
4. Enter the value for google_service_account_id as obtained from your Deployment Engineer.
5. Enter the list of scopes as listed above (as a single comma-delimited string).
6. Click AUTHORIZE.

Now Atolio is authorized to retrieve Google Workspace data.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the client ID of the App Engine default service account and the google_service_account_id, used for setting up domain-wide delegation in Step 1. The connector service account key (service_account_key) is generated from Terraform output by your Deployment Engineer, used for the source configuration in Step 3.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  google:
    default:
      google:
        connector: google
        instance: default
        source: google
        common:
          enabled: true
          identity-provider: true    # Set to true when Google is your identity provider
          cron-spec: "0 7 * * *"     # Re-sync users on this schedule
        secrets:
          service_account_key:
            value: "<SERVICE_ACCOUNT_KEY_FROM_TERRAFORM>"
        configuration:
          project-id: "<GCP_PROJECT_ID>"
          service-user: "<WORKSPACE_ADMIN_EMAIL>"
          audiences:                 # Only if you share content with target audiences
            "<TARGET_AUDIENCE_ID>":
              - "<GROUP_OR_USER_EMAIL>"

Step 2: Deploy the Connector

Before the source can be configured, the connector pod must be running in the cluster. Connectors are deployed by enabling them in values-lumen.yaml. See Deploying Connectors for the full details, including how instances work.

connectors:
  google:
    default:
      enabled: true

default is the instance identifier. Treat it as immutable once set, because the instance is part of every resource identifier the connector ingests. Apply the change by upgrading the lumen Helm chart with create-infra.sh (recommended) or helm upgrade as described in the sources overview.

Step 3: Configure the Source

Source configuration is applied with atolioctl using a YAML file. If you have not already installed atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address the feeder server’s gRPC endpoint, both described next.

atolioctl Prerequisites

JWT_TOKEN holds the API token atolioctl uses to authenticate to the feeder. Generate one from the deployment’s Helm values directory, which holds the values-lumen.yaml and values-lumen-templated.yaml files produced by create-infra.sh. The token is signed with the deployment’s jwtSecretKey, and the atolio:*:*:* match pattern grants access across all connectors, instances, and resources:

export JWT_TOKEN=$(atolioctl connector create-jwt --raw --config-dir . "atolio:*:*:*")

--feeder-address is the gRPC address of the feeder server that is used to read and write source configuration through. The standard path is to port-forward the in-cluster service/feeder and point atolioctl at it on localhost:

# Run K8s port forward to feeder as background to re-use same shell
kubectl port-forward -n atolio-svc service/feeder 8889 &

With that port-forward running in current terminal or another, the examples below use --feeder-address :8889. The empty host means localhost, and because the port is not 443 the connection is made in plaintext, which is what the port-forward exposes. If the feeder is instead published on a load balancer, pass that address with port 443, for example --feeder-address feed.search.example.com:443, which connects over TLS.

Edit and Apply the Configuration

Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.

Export the current configuration to a local file:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml

Add the Google entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.

Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.

Import the edited file to write the configuration back into the stack:

atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Google. The connector validates the credentials and connection itself when it starts and picks up the configuration, and it reads users after that. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.

Target Audiences

A target audience is a group of people your Workspace administrator defines so that content can be shared with them without naming each person. Google Drive and Google Chat can grant access this way.

Map each audience id to the groups and users it grants access to:

        configuration:
          audiences:
            00zu0gcz154hwnv: # target audience id
              - engineering@example.com
              - all-contractors@example.com
              - jane.doe@example.com

Each entry is a group address or an individual’s address.

Finding target audiences

In the Google Admin console, go to Directory > Target audiences and select the audience. Its id is the last segment of the page’s address:

https://admin.google.com/ac/targetaudiences/00zu0gcz154hwnv
                                            ^^^^^^^^^^^^^^^

The same page lists the groups and individuals the audience contains, which are the values to put under the id.

This section applies only if your organization shares content with target audiences. If it does not, you can omit audiences entirely.

Configuration Schema

In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Google source sits at connectors.google.default.google.

GoogleSourceConfiguration

Google Workspace connector configuration.

Common

Common settings shared across all connectors.

Secrets

Sensitive configuration values for authentication.

SecretValue

A reference to a secret value.

Configuration

Google Workspace connector-specific configuration.

10.1 - Gmail Connector Setup

Installing the Gmail Connector

Note: For this step, your deployment engineer will be required.

The Gmail and Google Drive connectors depend on a shared Google Workspace configuration section. Google Workspace requires three properties to be set during configuration:

• ProjectID must reference the project created in GCP. The id, not the name, must be specified.
• ServiceUser is currently a reference to a super user or administrator within your organization. For example, the primary IT email. This may change in the future.
• ServiceAccountKey is the base64-encoded string as obtained during initial install of the Google source.

To obtain the service account key, your deployment engineer will run the following Terraform command:

terraform output -raw google_connector_service_account_key

And optionally:

• Domains is a comma-delimited list of domains applicable to this deployment.
• Parallelism is an optimization to increase the concurrency with respect to indexing Google data. Default is 32, so if set, should be higher than this value.

Once the Google source is configured, the Google Drive and Gmail sources can be configured with the same values.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

This connector uses the GCP Project ID and Google Workspace admin email listed in the Google connector’s Required Configuration Values. In addition:

Provided by Atolio: the connector service account key (service_account_key), generated from Terraform output by your Deployment Engineer, used for authenticating the connector to Google Workspace.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  google:
    default:
      gmail:
        connector: google
        instance: default
        source: gmail
        common:
          enabled: true
          identity-provider: false
        secrets:
          service_account_key:
            value: "<SERVICE_ACCOUNT_KEY_FROM_TERRAFORM>"
        configuration:
          project-id: "<GCP_PROJECT_ID>"
          service-user: "<WORKSPACE_ADMIN_EMAIL>"
          domains:                   # Only if you limit indexing to some domains
            - "<ALLOWED_DOMAIN>"

Configuration

Source configuration can be applied in two ways:

Recommended (v4.24+): Use the Admin console to configure this source through the browser. You can paste the YAML from the Configuration Example directly into the Admin console.

Alternative: Use atolioctl via the command line (for Deployment Engineers). If using atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address the feeder server’s gRPC endpoint, both described next.

Edit and Apply the Configuration

Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.

Export the current configuration to a local file:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml

Add the entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.

Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.

Import the edited file to write the configuration back into the stack:

atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Google. The connector validates the credentials and connection itself when it starts and picks up the configuration, and it reads users after that. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.

Configuration Schema

In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Gmail source sits at connectors.google.default.gmail.

GmailSourceConfiguration

Gmail connector configuration.

Common

Common settings shared across all connectors.

Secrets

Sensitive configuration values for authentication.

SecretValue

A reference to a secret value.

GmailConfiguration

Gmail-specific connector configuration.

10.2 - Google Chat Connector Setup

Installing the Google Chat Connector

Note: For this step, your deployment engineer will be required.

The Google Chat connector depends on the same Google Workspace configuration section as the Gmail and Google Drive connectors, so ProjectID, ServiceUser, and ServiceAccountKey carry the same values described in the Google connector setup.

Prerequisites

Enable the Required APIs

The GCP project referenced by project-id must have these APIs enabled:

• Google Chat API (chat.googleapis.com)
• Google Workspace Events API (workspaceevents.googleapis.com)
• Cloud Pub/Sub API (pubsub.googleapis.com)

The Atolio Terraform module enables them for you. If the project is managed by hand, enable them from the API Library.

Configure the Chat App

Google requires every project that calls Chat API to have a Chat app configured, manually by an admin.

Open the Google Chat API configuration page and complete the form:

Leave the remaining fields at their defaults and save.

Grant the Chat Scopes

You will need a Service Account as described in Grant Service Account Permissions. In addition to the scopes listed there, the domain-wide delegation entry must carry the following scopes:

https://www.googleapis.com/auth/chat.spaces.readonly
https://www.googleapis.com/auth/chat.messages.readonly
https://www.googleapis.com/auth/chat.memberships.readonly

To add them, open the Google Admin console at https://admin.google.com/ac/owl/domainwidedelegation, edit the existing entry for the Atolio service account client ID, append the three scopes to the list already there, and authorize. Removing a scope that is already in the list will break the Gmail, Drive, or Sites sources.

Create the Pub/Sub Topic and Subscription

To support the indexing of Google Chat messages in real-time (streaming), a Pub/Sub topic is needed.

The Atolio Terraform module creates both, along with the two role bindings. If you are provisioning by hand, make sure to keep the name and bindings as listed above.

Streaming is the only part of the connector that needs Pub/Sub. If you set skip-streaming to true, the topic and subscription are not required, and the connector indexes only what it finds on each backfill.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

This connector uses the GCP Project ID and Google Workspace admin email listed in the Google connector’s Required Configuration Values. In addition:

Provided by Atolio: the connector service account key (service_account_key), generated from Terraform output by your Deployment Engineer, used for authenticating the connector to Google Workspace. Atolio also identifies the connector service account that needs roles/pubsub.subscriber on the GChatEvents subscription, used for streaming Chat events.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  google:
    default:
      gchat:
        connector: google
        instance: default
        source: gchat
        common:
          enabled: true
          identity-provider: false
        secrets:
          service_account_key:
            value: "<SERVICE_ACCOUNT_KEY_FROM_TERRAFORM>"
        configuration:
          project-id: "<GCP_PROJECT_ID>"
          service-user: "<WORKSPACE_ADMIN_EMAIL>"

Configuration

Source configuration can be applied in two ways:

Recommended (v4.24+): Use the Admin console to configure this source through the browser. You can paste the YAML from the Configuration Example directly into the Admin console.

Alternative: Use atolioctl via the command line (for Deployment Engineers). If using atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address of the feeder server’s gRPC endpoint.

Edit and Apply the Configuration

Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.

Export the current configuration to a local file:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml

Add the entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.

Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.

Import the edited file to write the configuration back into the stack:

atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Google. The connector validates the credentials and connection itself when it starts and picks up the configuration. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.

Limiting the Backfill Window

By default, backfill reads messages created in the last five years. Narrow that with backfill-start-time, which is useful on a large Workspace where the first pass would otherwise run for a long time against Chat’s per-project read quota:

        configuration:
          backfill-start-time: "2026-01-01T00:00:00Z"

Both bounds are RFC 3339 timestamps. backfill-end-time is unbounded when unset, and is mostly useful for pinning a reproducible window.

Backfill resumes from where it left off. Each space records the point its messages were read up to, so an interrupted or repeated pass reads only what is new rather than starting over.

Discoverable Spaces and Target Audiences

A Google Chat space can be made discoverable to a target audience, which is a group of people your Workspace administrator defines. Anyone in that audience can find the space and read its messages without being a member, and Atolio has to reflect that in its permissions.

See Target Audiences to configure target audiences in Google IdP connector.

Configuration Schema

In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Google Chat source sits at connectors.google.default.gchat.

GoogleChatSourceConfiguration

Google Chat connector configuration.

Common

Common settings shared across all connectors.

Secrets

Sensitive configuration values for authentication.

SecretValue

A reference to a secret value.

GChatConfiguration

Google Chat-specific connector configuration.

10.3 - Google Drive Connector Setup

Installing the Google Drive Connector

Note: For this step, your deployment engineer will be required.

The Gmail and Google Drive connectors depend on a shared Google Workspace configuration section. Google Workspace requires three properties to be set during configuration:

• ProjectID must reference the project created in GCP. The id, not the name, must be specified.
• ServiceUser is currently a reference to a super user or administrator within your organization. For example, the primary IT email. This may change in the future.
• ServiceAccountKey is the base64-encoded string as obtained during initial install of the Google source.

To obtain the service account key, your deployment engineer will run the following Terraform command:

terraform output -raw google_connector_service_account_key

And optionally:

• Domains is a comma-delimited list of domains applicable to this deployment.
• Parallelism is an optimization to increase the concurrency with respect to indexing Google data. Default is 32, so if set, should be higher than this value.

Once the Google source is configured, the Google Drive and Gmail sources can be configured with the same values.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

This connector uses the GCP Project ID and Google Workspace admin email listed in the Google connector’s Required Configuration Values. In addition:

Provided by Atolio: the connector service account key (service_account_key), generated from Terraform output by your Deployment Engineer, used for authenticating the connector to Google Workspace.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  google:
    default:
      gdrive:
        connector: google
        instance: default
        source: gdrive
        common:
          enabled: true
          identity-provider: false
        secrets:
          service_account_key:
            value: "<SERVICE_ACCOUNT_KEY_FROM_TERRAFORM>"
        configuration:
          project-id: "<GCP_PROJECT_ID>"
          service-user: "<WORKSPACE_ADMIN_EMAIL>"
          domains:                   # Only if you limit indexing to some domains
            - "<ALLOWED_DOMAIN>"

Configuration

Source configuration can be applied in two ways:

Recommended (v4.24+): Use the Admin console to configure this source through the browser. You can paste the YAML from the Configuration Example directly into the Admin console.

Alternative: Use atolioctl via the command line (for Deployment Engineers). If using atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address the feeder server’s gRPC endpoint, both described next.

Edit and Apply the Configuration

Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.

Export the current configuration to a local file:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml

Add the entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.

Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.

Import the edited file to write the configuration back into the stack:

atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Google. The connector validates the credentials and connection itself when it starts and picks up the configuration, and it reads users after that. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.

Sharing a file with Target Audiences

A Google Drive file can be shared with a target audience, which is a group of people your Workspace administrator defines. Anyone in that audience can access the original file.

See Target Audiences to configure target audiences in Google IdP connector.

Document Permissions

Atolio mirrors each file’s Drive sharing settings when deciding who can find it in search. Drive’s “Can find in search results” toggle (shown when you share to your whole domain) is respected: a file that is not discoverable in Drive is not discoverable in Atolio.

A file whose only sharing setting is link-based has no directly shared users or groups, so it is indexed but returned to no one. Share it with specific people, or turn on “Can find in search results”, to make it discoverable.

Configuration Schema

In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Google Drive source sits at connectors.google.default.gdrive.

GoogleDriveSourceConfiguration

Google Drive connector configuration.

Common

Common settings shared across all connectors.

Secrets

Sensitive configuration values for authentication.

SecretValue

A reference to a secret value.

GDriveConfiguration

Google Drive-specific connector configuration.

10.4 - Google Sites Connector Setup

Installing the Google Sites Connector

Note: For this step, your deployment engineer will be required.

Prerequisites

Google Sites Connector requires you to enable Google Vault for your organization. Google Vault enrollment details are available in the official documentation

You will need a Service Account as described in Grant Service Account Permissions section. In addition to the scopes listed there, you must grant the following scopes to the Service Account:

- https://www.googleapis.com/auth/ediscovery
- https://www.googleapis.com/auth/devstorage.read_only

Vault API access must be manually enabled by the GCP project associated with the Service Account. This can be done via GCP API Console, by following the steps in the official documentation.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

This connector uses the Google Workspace admin email listed in the Google connector’s Required Configuration Values. In addition:

Provided by Atolio: the connector service account key (service_account_key), generated from Terraform output by your Deployment Engineer, used for authenticating the connector to Google Workspace.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  google:
    default:
      gsites:
        connector: google
        instance: default
        source: gsites
        common:
          enabled: true
          identity-provider: false
        secrets:
          service_account_key:
            value: "<SERVICE_ACCOUNT_KEY_FROM_TERRAFORM>"
        configuration:
          service-user: "<WORKSPACE_ADMIN_EMAIL>"
          site-urls:
            - "<GOOGLE_SITE_URL>"
          site-configs:              # Only if a site needs a custom domain, page links or restricted access
            "<GOOGLE_SITE_URL>":
              top-level-url: "<CUSTOM_DOMAIN_URL>"   # Only if the site is published on a custom domain
              page-name-to-url-suffix:               # Only if you want links to individual pages
                Home: /home
                About: /about
              restricted: true                       # Only if the site uses "Restricted" sharing

Configuration

Source configuration can be applied in two ways:

Recommended (v4.24+): Use the Admin console to configure this source through the browser. You can paste the YAML from the Configuration Example directly into the Admin console.

Alternative: Use atolioctl via the command line (for Deployment Engineers). If using atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address the feeder server’s gRPC endpoint, both described next.

Edit and Apply the Configuration

Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.

Export the current configuration to a local file:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml

Add the entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.

If you do not prefer to have a static configuration for deeplinks, you can also provide a “url path” on each Google Site page. Simply add a text in the form of urlpath:[/relative-path-to-your-page] in the body of each page, and Atolio will index the page with the user-provided deeplink.

Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.

Import the edited file to write the configuration back into the stack:

atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889

Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Google. The connector validates the credentials and connection itself when it starts and picks up the configuration, and it reads users after that. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.

Configuration Schema

In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Google Sites source sits at connectors.google.default.gsites.

GoogleSitesSourceConfiguration

Google Sites connector configuration.

Common

Common settings shared across all connectors.

Secrets

Sensitive configuration values for authentication.

SecretValue

A reference to a secret value.

GSitesConfiguration

Google Sites-specific connector configuration.

SiteConfig

Per-site configuration for Google Sites.

11 - HubSpot Connector

The HubSpot connector enables the indexing of HubSpot CRM tickets and users using the HubSpot API through a private app in your HubSpot account.

Indexed tickets respect HubSpot’s record-level permissions: a ticket is only searchable in Atolio by users who are permitted to view that ticket in HubSpot.

Create a Private App

The connector authenticates using a private app access token. To create the private app, you must be a super admin in your HubSpot account.

1. In HubSpot, navigate to Settings → Integrations → Private Apps.
2. Click Create a private app.
3. On the Basic Info tab, choose a name for the app, e.g. Atolio.
4. On the Scopes tab, add the following scopes:
   • oauth
   • tickets
   • crm.objects.users.read
   • settings.users.read
5. Click Create app.

All four scopes are required. The connector verifies them on startup and will fail with a “missing scopes” error if any are absent.

After the app is created, the app’s Auth tab shows the two credentials the connector needs: the access token and the client secret (used to verify the authenticity of webhook events sent by HubSpot). Both are listed in Required Configuration Values.

Configure Webhooks

The connector receives ticket changes in real time via HubSpot webhooks. HubSpot does not allow these subscriptions to be created via the API, so they must be configured manually in the private app.

1. Open your private app and go to the Webhooks tab.

2. Set the Target URL to your Atolio deployment’s callback URL. Your Deployment Engineer will provide this; it has the form:

   https://<your-atolio-connector-host>/connectors/hubspot/<instance>/callback/hubspot
   

3. Create subscriptions for the following event types, applied to Tickets:

   • Creation (ticket.creation)
   • Property change (ticket.propertyChange)
   • Deletion (ticket.deletion)

Note: The target URL registered in HubSpot must exactly match the callback URL configured on the Atolio side. Webhook signatures are computed over the full URL, so any mismatch causes all events to be rejected.

Webhooks cover ticket changes only. Changes to users (and to ticket permissions) are picked up by the periodic sync described below.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the webhook Target URL (https://<your-atolio-connector-host>/connectors/hubspot/<instance>/callback/hubspot), used for Configure Webhooks.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  hubspot:
    default:
      hubspot:
        connector: hubspot
        instance: default
        source: hubspot
        common:
          enabled: true
          identity-provider: false
          cron-spec: "*/30 * * * *"
        secrets:
          access_token:
            value: "<ACCESS_TOKEN>"
          client_secret:
            value: "<CLIENT_SECRET>"
        configuration:
          base_url: "https://api.hubapi.com"

The cron-spec schedules the periodic sync that picks up user and permission changes. See Sync Behavior.

Reference: Configuration Parameters

Sync Behavior

The connector syncs incrementally: each run fetches only tickets and users modified since the previous run. On first run, all tickets and users are indexed.

Periodic syncs are scheduled with the connector’s cron-spec, e.g. */30 * * * * to sync every 30 minutes. The periodic sync is what picks up user changes and ticket permission changes, since webhooks only deliver ticket content changes.

The connector automatically paces its requests to stay within HubSpot’s search API rate limits.

Content Filtering

Atolio supports the ability to avoid certain content from being indexed at all. You can provide inclusions or exclusions with respect to each resource type:

• Users can be filtered w.r.t. their email addresses.
• Tickets can be filtered w.r.t. their ticket IDs.

An example of an inclusions/exclusions configuration would be as follows:

resources:
    user:
        excluded:
            - external_contractor@example.com # user email
    ticket:
        excluded:
            - "1234567890" # ticket ID

If you need content filtering, provide the inclusions & exclusions similar to the above to your Deployment Engineer.

Limitations

• Only tickets with an owner and a subject are indexed. Tickets without an assigned owner are skipped.
• HubSpot service accounts (integration users) are excluded from user indexing.
• Other CRM objects (contacts, companies, deals) are not currently indexed.

12 - Keycloak Connector

The Keycloak connector reads the users in a realm so that Atolio can resolve each signed-in identity to an Atolio user. Every user is ingested under their Keycloak user ID, which is the same identifier Keycloak puts in the sub claim of an OIDC token. That match is what lets Atolio connect a login back to the right user.

Note: As part of Atolio Configuration, this connector should be selected as Identity Provider if it is used to resolve app-user mappings. Point it at the same realm your users authenticate against in Keycloak authentication.

Setting up the connector takes three steps:

1. Create a read-only service account in Keycloak (done by your Keycloak admin).
2. Deploy the connector pod into the Atolio cluster (done by your Deployment Engineer).
3. Configure the source with atolioctl (done by your Deployment Engineer).

Step 1: Create a Service Account in Keycloak

The connector talks to the Keycloak Admin REST API as a realm user, so we recommend creating a dedicated service account rather than using a personal admin account. The connector only reads users; it does not write to Keycloak or read any other resource.

1. In the admin console, select your realm and create a user under “Users” (e.g. atolio-connector).
2. On the new user’s “Credentials” tab, set a password and turn off “Temporary” so it does not expire on first use.
3. On the “Role mapping” tab, click “Assign role”, filter by clients, and assign the view-users role from realm-management. This is the lowest-privilege role that lets the connector list users in the realm.

The connector authenticates with this username and password against the realm’s built-in admin-cli client using the OAuth2 password grant, then refreshes the resulting token as needed. No additional client setup is required, since admin-cli exists in every realm by default.

Note: view-users is scoped to the realm you assign it in. If your users live in more than one realm, deploy a connector instance per realm, each with its own service account. If you rotate the service account’s password in Keycloak, update the password secret (Step 3) so the connector can keep authenticating.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  keycloak:
    default:
      keycloak:
        connector: keycloak
        instance: default
        source: keycloak
        common:
          enabled: true
          identity-provider: true    # Set to true when Keycloak is your identity provider
          cron-spec: "0 7 * * *"     # Re-sync users on this schedule
        secrets:
          password:
            value: "<SERVICE_ACCOUNT_PASSWORD>"
        configuration:
          base_url: "<KEYCLOAK_BASE_URL>"
          realm: "<REALM>"
          username: "<SERVICE_ACCOUNT_USERNAME>"

Step 2: Deploy the Connector

Before the source can be configured, the connector pod must be running in the cluster. Connectors are deployed by enabling them in values-lumen.yaml. See Deploying Connectors for the full details, including how instances work.

connectors:
  keycloak:
    default:
      enabled: true

default is the instance identifier. Treat it as immutable once set, because the instance is part of every resource identifier the connector ingests. Apply the change by upgrading the lumen Helm chart with create-infra.sh (recommended) or helm upgrade as described in the sources overview.

Step 3: Configure the Source

Source configuration can be applied in two ways:

Recommended (v4.24+): Use the Admin console to configure this source through the browser. You can paste the YAML from the Configuration Example directly into the Admin console.

Alternative: Use atolioctl via the command line (for Deployment Engineers). If using atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address the feeder server’s gRPC endpoint, both described next.

atolioctl Prerequisites

JWT_TOKEN holds the API token atolioctl uses to authenticate to the feeder. Generate one from the deployment’s Helm values directory, which holds the values-lumen.yaml and values-lumen-templated.yaml files produced by create-infra.sh. The token is signed with the deployment’s jwtSecretKey, and the atolio:*:*:* match pattern grants access across all connectors, instances, and resources:

export JWT_TOKEN=$(atolioctl connector create-jwt --raw --config-dir . "atolio:*:*:*")

--feeder-address is the gRPC address of the feeder server that is used to read and write source configuration through. The standard path is to port-forward the in-cluster service/feeder and point atolioctl at it on localhost:

# Run K8s port forward to feeder as background to re-use same shell
kubectl port-forward -n atolio-svc service/feeder 8889 &

With that port-forward running in current terminal or another, the examples below use --feeder-address :8889. The empty host means localhost, and because the port is not 443 the connection is made in plaintext, which is what the port-forward exposes. If the feeder is instead published on a load balancer, pass that address with port 443, for example --feeder-address feed.search.example.com:443, which connects over TLS.

Edit and Apply the Configuration

Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.

Export the current configuration to a local file:

atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml

Add the entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.

Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.

Import the edited file to write the configuration back into the stack:

atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889
Writing key: /lumen/connectors/keycloak/default/keycloak

Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Keycloak. The connector validates the credentials and connection itself when it starts and picks up the configuration, and it reads users after that. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.

Configuration Schema

In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Keycloak source sits at connectors.keycloak.default.keycloak.

KeycloakSourceConfiguration

Keycloak connector configuration for user identity mapping.

Common

Common settings shared across all connectors.

Secrets

Sensitive configuration values for authentication.

SecretValue

A reference to a secret value.

KeycloakConfiguration

Keycloak-specific connector configuration.

Troubleshooting

Almost all errors surface in the connector pod logs, not in the atolioctl output. The import command only checks that the YAML is well formed before writing it to the stack, so a malformed file is the one thing it catches directly. Everything else is reported by the connector as it runs through its lifecycle.

The connector validates first. When it starts and picks up the configuration, it logs in to Keycloak to check the credentials and connection. If that fails, the source moves to a misconfigured state and stops before reading any users. Once validation passes, the connector lists users during its first sync, which is where a missing role shows up. Use the lifecycle phase in the table below to tell the two apart.

13 - Linear Connector

The Linear connector indexes teams, projects, issues, and comments across your Linear workspace.

Note: Atolio currently uses an API key for this connector, but will support a published Integration in a future release.

Create the Linear API key & Webhooks

As an administrator:

1. Navigate to Workspace Settings by selecting your tenant and choosing Preferences.
2. Under My Account, click on API.

To create an API key, complete the following steps: 3. Under Personal API keys, provide a Label in the text box, e.g. Atolio. 4. Click on Create new API key. 5. Copy the value of the Personal API key. You’ll provide it as described in Required Configuration Values.

To create a Webhook, complete the following steps: 6. Under Webhooks, click on Create new webhook. 7. Provide a Label in the text box, e.g. Atolio. 8. For URL, specify the URL provided by your Deployment Engineer. It will be similar to the following with your chosen DNS name:

• https://relay.search.example.com/connectors/linear/default/callback/linear
• Note: if you have multiple Linear instances, please make sure to provide the correct instance name in the URL above (i.e. default in the example).

9. Under Data change events, ensure the following are checked:
   • Issues
   • Projects
   • Comments
10. Copy the value of the Signing secret. You’ll provide it as described in Required Configuration Values.
11. Click on Create webhook to finalize the creation of this webhook.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Provided by Atolio: the webhook URL (e.g. https://relay.search.example.com/connectors/linear/default/callback/linear), used for the webhook in step 8.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  linear:
    default:
      linear:
        connector: linear
        instance: default
        source: linear
        common:
          enabled: true
          identity-provider: false
        secrets:
          token:
            value: "<PERSONAL_API_KEY>"
          webhook_secret:
            value: "<WEBHOOK_SIGNING_SECRET>"
        configuration:
          url-key: "<WORKSPACE_URL_KEY>"

14 - Microsoft Connectors

The Microsoft connectors support:

• Entra ID - for resolving user mappings across apps as an Identity Provider
• SharePoint - for indexing sites, docs, spreadsheets, slides, and other files
• Teams - for indexing channels, chat, and messages
• OneDrive - for indexing docs, spreadsheets, slides, and other files
• Outlook - for indexing emails and calendar events

Security Best Practice: Separate App Registrations

We recommend creating separate App Registrations for each Microsoft source you plan to use. This follows the principle of least privilege and provides:

• Reduced attack surface - Each connector only has the minimum permissions required
• Better audit trails - Azure logs can isolate which application accessed which data
• Independent lifecycle management - Rotate certificates, disable connectors, or modify permissions without affecting other sources
• Granular admin consent - Security teams can approve smaller, focused permission requests
• Independent rate limits - Each App Registration has separate Microsoft Graph API rate limits, preventing sources from competing for shared quotas in large environments

Architecture Options

Choose the approach that best fits your organization’s security and scale requirements:

Option 1: Separate App Registration per Source (Recommended)

Create individual App Registrations for:

• Entra ID (if using as identity provider)
• SharePoint
• Teams
• OneDrive
• Outlook

Each registration gets only the permissions needed for that specific source.

Option 2: Single Shared App Registration (Evaluation Only)

Use one App Registration for all sources. This requires the superset of all permissions and is not recommended for production environments due to security and rate limiting concerns.

Prerequisites

Before configuring any Microsoft connector, ensure you have:

Azure Access Requirements

• Azure tenant with appropriate licensing (Microsoft 365 E1+ for SharePoint/OneDrive, Teams Premium for meeting transcripts)
• Global Administrator or Cloud Application Administrator role to:
  • Create App Registrations
  • Grant application permissions
  • Provide admin consent
• Certificate for authentication - See Certificate Generation Guide below

Atolio Environment

• Active Atolio deployment - Contact your Atolio onboarding team if you need assistance
• Configuration values - You’ll collect these from Azure, as listed in Required Configuration Values

Certificate Generation

Microsoft Graph API requires certificate-based authentication. Choose one option:

Note: A password is required on the .PFX file for new certificates. Keep the password; you’ll need to provide it with the certificate.

Option 1: Self-Signed Certificate (OpenSSL)

# Generate private key and certificate
openssl req -x509 -newkey rsa:2048 -keyout atolio-key.pem -out atolio-cert.pem -days 365 -nodes

# Create password-protected PFX file (you will be prompted to choose a password)
openssl pkcs12 -export -out atolio-cert.pfx -inkey atolio-key.pem -in atolio-cert.pem

# Create CER file for Azure upload
openssl x509 -outform der -in atolio-cert.pem -out atolio-cert.cer

# Get base64-encoded PFX for Atolio configuration
cat atolio-cert.pfx | base64

Option 2: Self-Signed Certificate (PowerShell/Windows)

# Generate certificate
$cert = New-SelfSignedCertificate -Subject "CN=Atolio" -CertStoreLocation "Cert:\CurrentUser\My" -KeyExportPolicy Exportable -KeySpec Signature

# Export CER file for Azure
Export-Certificate -Cert $cert -FilePath "atolio-cert.cer"

# Export PFX file for Atolio (replace '<your-pfx-password>' with your chosen password)
Export-PfxCertificate -Cert $cert -FilePath "atolio-cert.pfx" -Password (ConvertTo-SecureString -String "<your-pfx-password>" -Force -AsPlainText)

# Get base64-encoded PFX
[Convert]::ToBase64String([IO.File]::ReadAllBytes("atolio-cert.pfx"))

Common Setup Steps

All Microsoft connectors require these foundational steps:

1. Create App Registration - Follow Entra ID - Create New App Registration
2. Upload Certificate - Add your .cer file to the App Registration
3. Grant API Permissions - Add Application permissions (not Delegated) for each source:
   • For Entra ID identity provider: Use only the directory permissions listed in the authentication guide
   • For content sources: Use the specific permissions listed in each connector’s documentation
4. Admin Consent - Click “Grant admin consent” in the API permissions page

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Each Microsoft connector needs its own App Registration, so each connector page lists its own copy of these values. The values below are common to every Microsoft connector.

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Each connector page has a Configuration Example with the YAML block for connector-values.yaml: OneDrive, Outlook, SharePoint and Teams.

Resource Filtering

All Microsoft connectors support filtering to limit indexing scope using YAML configuration in your connector settings:

resources:
  user:
    included:
      - arthur.dent@example.com
  group:
    excluded:
      - contractors@example.com

Common filter types include:

• User filters - Index content for specific user accounts
• Group filters - Index content accessible by specific Microsoft 365 groups
• LDAP group filters - Index content for members of specific Entra ID security/distribution groups (Outlook only)
• Site/Team filters - Index specific SharePoint sites or Teams

For detailed filtering options, see the individual connector documentation pages.

14.1 - OneDrive Connector Setup

Index OneDrive files by creating a dedicated App Registration with the appropriate permissions.

Setup Checklist

Before starting, ensure you have:

• Azure Global Administrator or Cloud Application Administrator role
• Microsoft 365 E1+ licensing for OneDrive content
• Certificate generated (see Certificate Generation Guide)
• Active Atolio deployment

Required API Permissions

Step 1: Create App Registration

1. Navigate to Azure Portal > App Registrations
2. Click “New registration”
3. Enter name: Atolio OneDrive
4. Select “Accounts in this organizational directory only”
5. Click “Register”

Step 2: Add API Permissions

⚠️ Important: Only add Application permissions, not Delegated permissions

1. In your App Registration, click “API permissions” in the left menu
2. Click “Add a permission”
3. Select “Microsoft Graph”
4. Click “Application permissions” (not Delegated permissions)

Add these Microsoft Graph permissions:

Identity & Directory Permissions (Required for ACL mapping)

• Application.Read.All
• Group.Read.All
• GroupMember.Read.All
• User.Read.All
• Directory.Read.All (optional)

OneDrive Content Permissions

• Files.Read.All
• Reports.Read.All (required for indexing progress display)

Step 3: Grant Admin Consent

1. After adding all permissions, click “Grant admin consent for [your tenant name]” at the top of the API permissions page
2. Confirm by clicking “Yes”
3. Verify all permissions show “Granted for [your tenant name]” with green checkmarks

Step 4: Upload Certificate

1. Click “Certificates & secrets” in the left menu
2. Click “Upload certificate”
3. Select your .cer file (not the .pfx file)
4. Add description: Atolio OneDrive Authentication
5. Click “Add”

Step 5: Collect Configuration Values

From the “Overview” page, copy:

• Application (client) ID
• Directory (tenant) ID

You’ll provide these, along with your base64-encoded .pfx certificate and its password, as listed in Required Configuration Values.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).

Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.

Configuration Example

Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.

connectors:
  microsoft:
    default:
      onedrive:
        connector: microsoft
        instance: default
        source: onedrive
        common:
          enabled: true
          identity-provider: false
        secrets:
          client_cert:
            value: "<BASE64_PFX_CERTIFICATE>"
          client_cert_password:
            value: "<PFX_PASSWORD>"
        configuration:
          tenant-id: "<DIRECTORY_TENANT_ID>"
          client-id: "<APPLICATION_CLIENT_ID>"

Resource Filtering

Limit indexing scope using resource filters. OneDrive supports multiple filter types:

User Filtering

Index OneDrive files for specific user accounts:

resources:
  user:
    included:
      - arthur.dent@example.com
      - ford.prefect@example.com
    excluded:
      - galactic-president@example.com

Group Filtering

Index OneDrive files accessible by Microsoft 365 groups:

resources:
  group:
    included:
      - engineering-group@example.com
    excluded:
      - contractors@example.com

Optional Settings Example

These optional settings go in the same block as the Configuration Example, under common: and configuration:. This example shows optional settings for OneDrive, including a sync schedule and resource filters:

common:
  cron-spec: "10 */1 * * *"
configuration:
  resources:
    user:
      included:
        - john.doe@contoso.com
        - jane.smith@contoso.com
    group:
      excluded:
        - contractors@contoso.com

Troubleshooting

14.2 - Outlook Connector Setup

Index Outlook emails and events by creating a dedicated App Registration with the appropriate permissions.

Setup Checklist

Before starting, ensure you have:

• Azure Global Administrator or Cloud Application Administrator role
• Microsoft 365 licensing for Outlook content
• Certificate generated (see Certificate Generation Guide)
• Active Atolio deployment

Required API Permissions

Step 1: Create App Registration

1. Navigate to Azure Portal > App Registrations
2. Click “New registration”
3. Enter name: Atolio Outlook
4. Select “Accounts in this organizational directory only”
5. Click “Register”

Step 2: Add API Permissions

⚠️ Important: Only add Application permissions, not Delegated permissions

1. In your App Registration, click “API permissions” in the left menu
2. Click “Add a permission”
3. Select “Microsoft Graph”
4. Click “Application permissions” (not Delegated permissions)

Add these Microsoft Graph permissions:

Identity & Directory Permissions (Required for ACL mapping)

• Application.Read.All
• Group.Read.All
• GroupMember.Read.All
• User.Read.All
• Directory.Read.All (optional)

Outlook Content Permissions

• Mail.Read
• Calendars.Read
• MailboxSettings.Read
• Domain.Read.All

Step 3: Grant Admin Consent

1. After adding all permissions, click “Grant admin consent for [your tenant name]” at the top of the API permissions page
2. Confirm by clicking “Yes”
3. Verify all permissions show “Granted for [your tenant name]” with green checkmarks

Step 4: Upload Certificate

1. Click “Certificates & secrets” in the left menu
2. Click “Upload certificate”
3. Select your .cer file (not the .pfx file)
4. Add description: Atolio Outlook Authentication
5. Click “Add”

Step 5: Collect Configuration Values

From the “Overview” page, copy:

• Application (client) ID
• Directory (tenant) ID

You’ll provide these, along with your base64-encoded .pfx certificate and its password, as listed in Required Configuration Values.

Required Configuration Values

Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).