Agent Observability Platform: Self-Managed Helm Install

Advanced: provision infrastructure with Terraform and manage the Helm release yourself

πŸ“˜

This is an advanced path. Most deployments should let Terraform deploy the chart in a single pass β€” see Installation. Use this path only if you need to manage chart upgrades and values on your own cadence.

When to use this

Set helm.deploy_charts = false to have Terraform provision the AWS infrastructure and cluster controllers only, without deploying the ao-data-platform chart. You then install and manage the Helm release yourself.

module "ao_data_platform" {
  source  = "monte-carlo-data/ao-data-platform/aws"
  version = "2.0.0"

  region = "us-east-1"

  helm = {
    deploy_charts = false
  }
}

Terraform still creates the IRSA roles, ACM certificates, KMS key, and Secrets Manager secrets β€” you wire their values into your Helm release (see Configure values). The Terraform-managed path is the reference implementation: when self-managing, you reproduce the same values it would otherwise pass to the chart.

1. Apply the infrastructure

Run terraform apply with helm.deploy_charts = false, then collect the outputs you will need for the chart:

terraform output

Keep these handy β€” the mapping table below shows where each goes:
otel_collector_irsa_role_arn, llm_worker_irsa_role_arn, otel_collector_certificate_arn, clickhouse_certificate_arn, and the Secrets Manager secrets behind clickhouse_otel_credentials_secret_arn, clickhouse_schema_owner_credentials_secret_arn, clickhouse_llm_worker_credentials_secret_arn, and clickhouse_monte_carlo_credentials_secret_arn (plus the optional clickhouse_readonly_user_credentials_secret_arn / clickhouse_admin_credentials_secret_arn).

2. Confirm cluster prerequisites

The chart expects these to be present in the cluster (Terraform installs them unless you disabled the corresponding helm.install_* flag):

3. Configure values

At minimum, point each ClickHouse user's externalSecret at the AWS Secrets Manager secret holding its password (the ones Terraform created), wire in the IRSA roles and ACM certificates, and set the ClickHouse hostname. Chart 2.0.0 uses a per-user externalSecret under each clickhouse.<user> key β€” the always-on otel, schemaOwner, llmWorker, and monteCarlo users. The chart's keys map to the Terraform outputs as follows:

Terraform outputChart value
clickhouse_otel_credentials_secret_arn (the Secrets Manager secret it points to)clickhouse.otel.externalSecret.remoteRef.key
clickhouse_schema_owner_credentials_secret_arnclickhouse.schemaOwner.externalSecret.remoteRef.key
clickhouse_llm_worker_credentials_secret_arnclickhouse.llmWorker.externalSecret.remoteRef.key
clickhouse_monte_carlo_credentials_secret_arnclickhouse.monteCarlo.externalSecret.remoteRef.key
otel_collector_irsa_role_arnopentelemetry-collector.serviceAccount.annotations."eks.amazonaws.com/role-arn"
llm_worker_irsa_role_arnllmWorker.serviceAccount.annotations."eks.amazonaws.com/role-arn"
clickhouse_certificate_arnclickhouse.service.annotations (NLB TLS termination)
otel_collector_certificate_arnopentelemetry-collector.service.annotations (NLB TLS termination)

A minimal values.yaml covering the required wiring:

clickhouse:
  hostname: clickhouse.acme.com        # adds the external-dns hostname annotation
  storageClass: clickhouse-gp3         # or your own StorageClass
  storageSize: 500Gi
  # Each always-on user reads its password from its own AWS Secrets Manager secret.
  # secretStoreRef points at your ClusterSecretStore; remoteRef.key is the AWS
  # Secrets Manager secret name β€” the module names them <cluster_name>/clickhouse/<user>-credentials
  # (each is also exposed as a clickhouse_<user>_credentials_secret_arn output).
  otel:
    externalSecret:
      secretStoreRef:
        name: aws-secretsmanager       # your ClusterSecretStore
        kind: ClusterSecretStore
      remoteRef:
        key: <cluster_name>/clickhouse/otel-credentials
  schemaOwner:
    externalSecret:
      remoteRef:
        key: <cluster_name>/clickhouse/schema-owner-credentials
  llmWorker:
    externalSecret:
      remoteRef:
        key: <cluster_name>/clickhouse/llm-worker-credentials
  monteCarlo:
    externalSecret:
      remoteRef:
        key: <cluster_name>/clickhouse/monte-carlo-credentials
  # If you run a dedicated ClickHouse node group, match its label/taint:
  nodeSelector:
    dedicated: clickhouse
  tolerations:
    - key: dedicated
      operator: Equal
      value: clickhouse
      effect: NoSchedule

tls:
  enabled: true                        # requires cert-manager
  certManager:
    createCA: true                     # or set existingIssuerRef to reuse an Issuer

llmWorker:
  image:
    repository: montecarlodata/ao-llm-worker
    tag: "1.0.1"
  aws:
    region: us-east-1                  # Bedrock region
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: <llm_worker_irsa_role_arn>

To enable the optional readonly_user (external SQL clients) or admin (gated break-glass superuser, loopback-only by default), set clickhouse.readonlyUser.enabled: true / clickhouse.admin.enabled: true and configure their externalSecret the same way as the always-on users above. Point each at the matching clickhouse_readonly_user_credentials_secret_arn / clickhouse_admin_credentials_secret_arn secret. To tighten the ingest user to INSERT-only after Monte Carlo connects as monte_carlo, set clickhouse.otel.restrictGrants: true.

4. Install the chart

Install the published chart from Docker Hub into the montecarlo namespace. Its subchart dependencies β€” the Altinity ClickHouse operator and the OpenTelemetry Collector chart β€” are bundled in the published OCI artifact, so no separate helm dependency build is needed:

helm install ao-data-platform oci://registry-1.docker.io/montecarlodata/ao-data-platform \
  --version 2.0.0 \
  -n montecarlo --create-namespace \
  -f values.yaml

5. Verify

Confirm the components are healthy using the checklist in Connect to Monte Carlo, then continue with the credential handoff there.


Did this page help you?