The Google connector enables the indexing of:
- Documents, Sheets, Slides, and other Files in Google Drive
- Email in Gmail
- Pages in Google Sites
- Spaces and messages in Google Chat
- Users and Groups as metadata in your Google Workspace to enable user mapping / permissions
Content and updates are streamed as changes are detected and processed in your Google Workspace.
Step 0: Create Terraform Service Account for Google
Note: Depending on installation being BYOK, Managed, or otherwise this step may be skipped. Consult with an Atolio Support Engineer for any clarification.
Terraform will need a service account to make changes to your GCP project. We will grant it permissions which will essentially allow it to create, read, and delete all project resources. As such, you should keep the credentials for this account safe and not reuse them. The Terraform scripts create a much more restricted service account for normal operation.
- Visit the IAM Service Accounts page https://console.cloud.google.com/iam-admin/serviceaccounts and select the project created previously to set up the Terraform service account.
- Click CREATE SERVICE ACCOUNT
- Choose a name. While it can be anything, the name “Atolio Terraform” is recommended.
- Select CREATE AND CONTINUE.
- Grant the following roles: “Owner”, “Security Center Admin”, “Project IAM Admin” (without conditions).
- Click DONE. Skip step 5, you will not need to grant other users access to this service account.
Now the service account is created we can create API keys:
- Navigate to https://console.cloud.google.com/iam-admin/serviceaccounts and select the project created previously.
- Click the service user that was created in the previous step.
- Select the KEYS tab click the ADD KEY pull-down menu
- Select Create new key and use JSON for key type. Click CREATE.
This will create another JSON file with the API key and related information. This file will be needed by your Deployment Engineer (in their deploy/terraform directory).
In order for Terraform to programmatically make modifications, you need to manually enable the Google Cloud Resource Manager API.
- Navigate to https://console.cloud.google.com/apis/library
- Search for “Cloud Resource Manager API”
- Select “Cloud Resource Manager API”. Click ENABLE.
The project will use additional APIs, but now that this API is available to Terraform, the Atolio deployment scripts will use Terraform to enable those APIs automatically.
Step 1: Grant Service Account Permissions
The Google Workspace domain needs to grant permission to the integration’s default service account to perform operations. Unfortunately Terraform can’t automate this, so it has to be performed manually.
Your Deployment Engineer will need to make note of the unique ID for the default App Engine service account and the list of scopes. Be aware that this ID is of the default service account, typically named App Engine default service account. It is not the Service Account you’ve previously created.
And the list of scopes are:
https://www.googleapis.com/auth/gmail.readonly,https://www.googleapis.com/auth/calendar.readonly,https://www.googleapis.com/auth/calendar.acls.readonly,https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/admin.directory.group.readonly,https://www.googleapis.com/auth/admin.directory.domain.readonly,https://www.googleapis.com/auth/drive.readonly,https://www.googleapis.com/auth/drive.activity.readonly,https://www.googleapis.com/auth/chat.spaces.readonly,https://www.googleapis.com/auth/chat.messages.readonly,https://www.googleapis.com/auth/chat.memberships.readonly,https://www.googleapis.com/auth/ediscovery,https://www.googleapis.com/auth/devstorage.read_only
Grant every scope in that list. Each source requests only the scopes it needs, in its own token exchange, and Google refuses that exchange outright when the grant is missing any one of them — so a gap takes down every source sharing the exchange, not only the feature the scope belongs to. Gmail asks for calendar.readonly and calendar.acls.readonly alongside gmail.readonly, for example, so an ungranted calendar scope stops mail indexing too.
The scopes cover all five Google sources: mail and calendar, Drive, Chat, Sites, and the directory. Granting the full list keeps a later source from failing when you enable it.
We now need to set up Domain-wide Delegation:
- Navigate to the Google Admin console: https://admin.google.com/ac/owl. An account with admin access is required.
- Scroll down and click “MANAGE DOMAIN WIDE DELEGATION”.
- Click Add new and enter your service account client ID (obtained from your Deployment Engineer in the previous step). No overwrite.
- Enter the value for
google_service_account_idas obtained from your Deployment Engineer. - Enter the list of scopes as listed above (as a single comma-delimited string).
- Click AUTHORIZE.
Now Atolio is authorized to retrieve Google Workspace data.
Required Configuration Values
Enter these values in the Atolio Admin console (v4.24 and later), or send them to Atolio Support (your Deployment Engineer).
| Value | Where to find it | Example / format | Secret |
|---|---|---|---|
| GCP Project ID | The project used for Atolio, shown in the Google Cloud console project selector (the ID, not the project name) | atolio-123456 | No |
| Google Workspace admin email | A Workspace administrator account that the connector acts as when reading directory data | admin@example.com | No |
| Terraform service account JSON key file (not needed for BYOK or Managed deployments) | Step 0, the JSON key created for the “Atolio Terraform” service account | Downloaded .json file | Yes |
| Target audiences (optional, only if your organization shares content with target audiences) | Google Admin console, Directory > Target audiences (see Target Audiences) | Audience ID with its member addresses, e.g. 00zu0gcz154hwnv: engineering@example.com | No |
Provided by Atolio: the client ID of the App Engine default service account and the google_service_account_id, used for setting up domain-wide delegation in Step 1. The connector service account key (service_account_key) is generated from Terraform output by your Deployment Engineer, used for the source configuration in Step 3.
Keep secrets secure: Share values marked “Yes” in the Secret column only through a password-manager share (e.g. 1Password) or another secure channel agreed with Atolio Support. Never send them in plain email, chat, or support tickets.
Configuration Example
Add this block to connector-values.yaml. Replace each <PLACEHOLDER> with the matching value from the table above, and remove any line marked “Only if” that doesn’t apply.
connectors:
google:
default:
google:
connector: google
instance: default
source: google
common:
enabled: true
identity-provider: true # Set to true when Google is your identity provider
cron-spec: "0 7 * * *" # Re-sync users on this schedule
secrets:
service_account_key:
value: "<SERVICE_ACCOUNT_KEY_FROM_TERRAFORM>"
configuration:
project-id: "<GCP_PROJECT_ID>"
service-user: "<WORKSPACE_ADMIN_EMAIL>"
audiences: # Only if you share content with target audiences
"<TARGET_AUDIENCE_ID>":
- "<GROUP_OR_USER_EMAIL>"
Step 2: Deploy the Connector
Before the source can be configured, the connector pod must be running in the cluster. Connectors are deployed by enabling them in values-lumen.yaml. See Deploying Connectors for the full details, including how instances work.
connectors:
google:
default:
enabled: true
default is the instance identifier. Treat it as immutable once set, because the instance is part of every resource identifier the connector ingests. Apply the change by upgrading the lumen Helm chart with create-infra.sh (recommended) or helm upgrade as described in the sources overview.
Step 3: Configure the Source
Source configuration is applied with atolioctl using a YAML file. If you have not already installed atolioctl, follow Configuring Sources first. The commands below also need an API token in JWT_TOKEN and the address the feeder server’s gRPC endpoint, both described next.
atolioctl Prerequisites
JWT_TOKEN holds the API token atolioctl uses to authenticate to the feeder. Generate one from the deployment’s Helm values directory, which holds the values-lumen.yaml and values-lumen-templated.yaml files produced by create-infra.sh. The token is signed with the deployment’s jwtSecretKey, and the atolio:*:*:* match pattern grants access across all connectors, instances, and resources:
export JWT_TOKEN=$(atolioctl connector create-jwt --raw --config-dir . "atolio:*:*:*")
--feeder-address is the gRPC address of the feeder server that is used to read and write source configuration through. The standard path is to port-forward the in-cluster service/feeder and point atolioctl at it on localhost:
# Run K8s port forward to feeder as background to re-use same shell
kubectl port-forward -n atolio-svc service/feeder 8889 &
With that port-forward running in current terminal or another, the examples below use --feeder-address :8889. The empty host means localhost, and because the port is not 443 the connection is made in plaintext, which is what the port-forward exposes. If the feeder is instead published on a load balancer, pass that address with port 443, for example --feeder-address feed.search.example.com:443, which connects over TLS.
Edit and Apply the Configuration
Source configuration is stored as a single document that covers every configured connector. The supported workflow is to export the current configuration, edit it, then import it back. Always start from a fresh export so you do not overwrite a change made elsewhere.
Export the current configuration to a local file:
atolioctl configure export --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889 lumen-connectors.yaml
Add the Google entry from the Configuration Example alongside the connectors already in the file, keeping the existing entries in place.
Exported entries also carry a rendezvous-completed flag and a status block reporting each source’s current state and last validation result. Those fields are managed by Atolio. Leave them untouched on existing entries, and omit them when adding a new entry by hand.
Import the edited file to write the configuration back into the stack:
atolioctl configure import lumen-connectors.yaml --jwt-token-sdk ${JWT_TOKEN} --feeder-address :8889
Import writes the configuration into the stack after checking that the YAML is well formed. It does not contact Google. The connector validates the credentials and connection itself when it starts and picks up the configuration, and it reads users after that. If something is wrong, the source reports as misconfigured and the connector pod logs the error. See Troubleshooting for the specific messages.
Target Audiences
A target audience is a group of people your Workspace administrator defines so that content can be shared with them without naming each person. Google Drive and Google Chat can grant access this way.
Map each audience id to the groups and users it grants access to:
configuration:
audiences:
00zu0gcz154hwnv: # target audience id
- engineering@example.com
- all-contractors@example.com
- jane.doe@example.com
Each entry is a group address or an individual’s address.
Finding target audiences
In the Google Admin console, go to Directory > Target audiences and select the audience. Its id is the last segment of the page’s address:
https://admin.google.com/ac/targetaudiences/00zu0gcz154hwnv
^^^^^^^^^^^^^^^
The same page lists the groups and individuals the audience contains, which are the values to put under the id.
This section applies only if your organization shares content with target audiences. If it does not, you can omit audiences entirely.
Configuration Schema
In the exported file, every source lives at connectors.<connector>.<instance>.<source>, so the Google source sits at connectors.google.default.google.
GoogleSourceConfiguration
Google Workspace connector configuration.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
connector | string | yes | N/A | Parent connector to reference (can be self) |
instance | string | yes | N/A | Multi-state config variable used for certain connectors, often default |
source | string | yes | N/A | Sub-connector reference (can be self), often object key |
common | Common | yes | N/A | Common connector settings. |
secrets | Secrets | yes | N/A | Secret credentials for authentication. |
configuration | Configuration | yes | N/A | Connector-specific configuration. |
Common
Common settings shared across all connectors.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
enabled | boolean | yes | N/A | Set to true to run the source. |
identity-provider | boolean | yes | N/A | Set to true so Atolio uses this connector to resolve logins to users. |
cron-spec | string | no | N/A | Cron expression for periodic re-sync of users (e.g., 0 7 * * *). Omit to sync only on demand. |
Secrets
Sensitive configuration values for authentication.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
service_account_key | SecretValue | yes | N/A | Google service account key in Base64 to authenticate to Google Workspace. |
SecretValue
A reference to a secret value.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
value | string | yes | N/A | The secret value (typically Base64-encoded). |
Configuration
Google Workspace connector-specific configuration.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
domains | string | no | "" | Comma-separated list of allowed customer domains. If empty, all domains are considered. |
project-id | string | yes | N/A | GCP project ID under which the integration is operating. |
service-user | string | yes | N/A | Email address of an administrator within the Google Workspace account. |
parallelism | integer | yes | 0 | Number of users to process in parallel. Set to 0 for automatic parallelism based on system resources. |
audiences | map[string][]string | no | {} | Target audiences and the group and user addresses each one grants access to, keyed by audience id. See Target Audiences. |