When to use this

Set helm.deploy_charts = false to have Terraform provision the GCP infrastructure and cluster controllers only, without deploying the ao-data-platform chart. You then install and manage the Helm release yourself.

module "ao_data_platform" {
  source  = "monte-carlo-data/ao-data-platform/google"
  version = "~> 1.0"

  location = "us-central1-a"

  helm = {
    deploy_charts = false
  }
}

On this path Terraform still creates the cluster and VPC (including the Gateway's proxy-only subnet), the Cloud KMS keys, the Secret Manager secrets with their per-secret access grants, the cert-manager and External Secrets Operator controllers with the gcp-secret-manager ClusterSecretStore, and the montecarlo namespace. The domain and DNS inputs are not required — but setting dns_zone is still worthwhile: it makes the module install external-dns with its zone-scoped access, so your Gateway hostnames publish automatically.

The Terraform-managed path is the reference implementation: when self-managing, you reproduce the same values it would otherwise pass to the chart.

1. Apply the infrastructure

Run terraform apply with helm.deploy_charts = false, then collect the outputs you will need for the chart:

terraform output

Keep these handy — the mapping table below shows where each goes: gke_cluster_name, and the Secret Manager secrets behind clickhouse_otel_credentials_secret_id, clickhouse_schema_owner_credentials_secret_id, clickhouse_llm_worker_credentials_secret_id, and clickhouse_monte_carlo_credentials_secret_id (plus the optional clickhouse_readonly_user_credentials_secret_id / clickhouse_admin_credentials_secret_id).

2. Confirm cluster prerequisites

The chart expects these to be present in the cluster:

• Helm 3
• cert-manager — installed by Terraform unless you disabled helm.install_cert_manager; internal TLS is always enabled and requires it
• External Secrets Operator with the gcp-secret-manager ClusterSecretStore — installed and configured by Terraform unless you disabled helm.install_external_secrets_operator
• trust-manager — not installed on this path; see step 3
• The GKE Gateway API enabled on the standard channel (module-created clusters have it; for an existing cluster: gcloud container clusters update <cluster> --location <location> --gateway-api=standard)
• A REGIONAL_MANAGED_PROXY proxy-only subnet in the cluster's region (created with the module's VPC; bring your own on a BYO-VPC and set networking.proxy_only_cidr to its range — the module creates the proxy → pod firewall rule from it in both modes)
• DNS automation for the two hostnames — external-dns (installed when you set dns_zone), or manage the records manually

3. Provide what the module skips on this path

The module creates the following only when it manages the release. Self-managing means providing them yourself. The Workload Identity principals below use the direct-binding format — substitute your project number in the projects/ segment and project ID in the pool name:

principal://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<PROJECT_ID>.svc.id.goog/subject/ns/<NAMESPACE>/sa/<SERVICEACCOUNT>

1. trust-manager. Install it (its trust namespace must equal the release namespace, montecarlo). The Gateway's backend re-encryption depends on it: cert-manager self-signs an in-cluster CA, and trust-manager distributes it for the chart's BackendTLSPolicy.
2. cert-manager DNS-01 access. Grant the ns/cert-manager/sa/cert-manager principal access to your Cloud DNS zone so the Let's Encrypt DNS-01 solver can write validation records — zone-scoped roles/dns.admin on the managed zone, plus enough project-level read access to list zones. cert-manager authenticates via ambient Workload Identity; no key files.
3. LLM worker Vertex AI access. Grant the ns/montecarlo/sa/llm-worker principal permission to invoke Vertex AI models — a custom role holding aiplatform.endpoints.predict matches the module's least-privilege grant. The Model Garden enablement is still required as well.
4. Cloud Armor policy (optional). To restrict which sources reach the endpoints, create a regional Cloud Armor security policy (allow rules for your CIDRs — including the VPC's node and pod ranges, so in-cluster clients keep working — and a deny catch-all) and reference it by name via gateway.gcpBackendSecurityPolicy in your values.
5. Dedicated node pools (recommended for production). Attach a tainted node pool for ClickHouse (dedicated=clickhouse:NoSchedule) — and, for the HA topology, one single-zone pool per ClickHouse zone plus one per Keeper zone (dedicated=keeper:NoSchedule) — and wire the matching nodeSelector/tolerations in your values. Persistent disks are zone-locked: keep each pool in its volume's zone.

4. Configure values

At minimum: enable the Gateway with provider: gke, point each ClickHouse user's externalSecret at the Secret Manager secret Terraform created, and set the LLM worker to the Vertex provider. The chart uses a per-user externalSecret under each clickhouse.<user> key — the always-on otel, schemaOwner, llmWorker, and monteCarlo users. (The always-on probe user is passwordless and needs no wiring.) The chart's keys map to the Terraform outputs as follows:

The module names the secrets <cluster_name>-clickhouse-<user> (e.g. monte-carlo-clickhouse-otel); remoteRef.key takes that name. A minimal values.yaml covering the required wiring:

gateway:
  enabled: true
  provider: gke                       # className derives to gke-l7-rilb
  otelHostname: otel.acme.com
  clickhouseHostname: clickhouse.acme.com
  # gcpBackendSecurityPolicy: <cloud-armor-policy-name>   # optional; from step 3
  tls:
    source: letsencrypt
    letsencrypt:
      email: ""                       # optional ACME contact
      cloudDNS:
        project: <dns-project-id>     # project hosting the Cloud DNS zone

clickhouse:
  storageClass: clickhouse-premium    # the module-created class, or your own
  storageSize: 500Gi
  replicasCount: 1                    # 2 for the HA topology
  service:
    type: ClusterIP                   # the Gateway is the serving path
  # Each always-on user reads its password from its own Secret Manager secret
  # (module-created, named <cluster_name>-clickhouse-<user>). secretStoreRef
  # points at the module's ClusterSecretStore.
  otel:
    restrictGrants: true              # INSERT-only ingest user (matches the managed path)
    externalSecret:
      secretStoreRef:
        name: gcp-secret-manager
        kind: ClusterSecretStore
      remoteRef:
        key: <cluster_name>-clickhouse-otel
  schemaOwner:
    externalSecret:
      secretStoreRef:
        name: gcp-secret-manager
        kind: ClusterSecretStore
      remoteRef:
        key: <cluster_name>-clickhouse-schema-owner
  llmWorker:
    externalSecret:
      secretStoreRef:
        name: gcp-secret-manager
        kind: ClusterSecretStore
      remoteRef:
        key: <cluster_name>-clickhouse-llm-worker
  monteCarlo:
    externalSecret:
      secretStoreRef:
        name: gcp-secret-manager
        kind: ClusterSecretStore
      remoteRef:
        key: <cluster_name>-clickhouse-monte-carlo
  # If you run a dedicated ClickHouse node pool, match its label/taint:
  nodeSelector:
    dedicated: clickhouse
  tolerations:
    - key: dedicated
      operator: Equal
      value: clickhouse
      effect: NoSchedule

keeper:
  replicasCount: 1                    # odd count (Raft quorum); 3 for HA
  # storageClass: standard-rwo
  # storageSize: 10Gi
  # For HA with dedicated Keeper pools, match their label/taint:
  # nodeSelector: {dedicated: keeper}
  # tolerations: [{key: dedicated, operator: Equal, value: keeper, effect: NoSchedule}]

tls:
  enabled: true                       # required in gateway mode
  certManager:
    createCA: true                    # required in gateway mode

llmWorker:
  provider: vertex
  vertex:
    project: <gcp-project-id>         # project the Vertex calls bill to
    region: global
  image:
    repository: montecarlodata/ao-llm-worker
    tag: "1.1.0-gcp"
  # No serviceAccount annotation needed — Workload Identity binds the
  # Kubernetes ServiceAccount principal directly (step 3).

opentelemetry-collector:
  service:
    type: ClusterIP                   # the Gateway is the serving path
  ports:
    otlp-http:
      appProtocol: HTTPS              # required for the Gateway's backend re-encryption

To enable the optional readonly_user (external SQL clients) or admin (gated break-glass superuser, loopback-only by default), set clickhouse.readonlyUser.enabled: true / clickhouse.admin.enabled: true and configure their externalSecret the same way as the always-on users above, pointing at the matching clickhouse_readonly_user_credentials_secret_id / clickhouse_admin_credentials_secret_id secret.

5. Install the chart

Install the published chart from Docker Hub into the montecarlo namespace (Terraform already created it on this path). Its subchart dependencies — the Altinity ClickHouse operator and the OpenTelemetry Collector chart — are bundled in the published OCI artifact, so no separate helm dependency build is needed:

helm install ao-data-platform oci://registry-1.docker.io/montecarlodata/ao-data-platform \
  --version 4.0.0 \
  -n montecarlo \
  -f values.yaml

6. Verify

Confirm the components are healthy using the checklist in Connect to Monte Carlo — including the Gateway (PROGRAMMED True with a private address) and the Let's Encrypt certificates — then continue with the credential handoff there. The module's Workload Identity verification script works on this path too.