Agent Observability Platform: Prerequisites (AWS)

What you need before deploying the Agent Observability data platform on AWS

☁️

This page covers AWS. Also available for Azure and GCP.

This page walks through everything you need before running the installation: the right tooling, an AWS account with sufficient permissions, a deployment path, domains, and the platform artifacts. Work through the steps in order, then run the preflight check to confirm you are ready.

1. Install the required tooling

Install the following on the machine you will run Terraform from:

ToolVersionUsed for
Terraform>= 1.3Provisioning the platform
AWS CLIlatestAuthenticating to AWS and retrieving outputs
kubectllatestVerifying the cluster after deployment
Helm3.xOnly for the self-managed Helm path

Configure the AWS CLI with credentials for your target account before continuing.

2. Prepare your AWS account

You deploy into your own AWS account. The principal running terraform apply needs permissions to create and manage:

  • VPC and networking (subnets, NAT gateways, route tables, security groups) β€” for the new-cluster path
  • EKS cluster and managed node groups
  • IAM roles, policies, and an OIDC provider (for IRSA)
  • KMS keys and AWS Secrets Manager secrets
  • ACM certificates and Route 53 records

The platform artifacts are public (Terraform Registry and Docker Hub), so no registry credentials or cross-account pull grants are needed β€” only outbound access to Docker Hub: the helm provider pulls the ao-data-platform chart during apply, and the EKS nodes pull the ao-llm-worker image at runtime.

The principal that runs terraform apply becomes the cluster's administrator automatically (the module sets enable_cluster_creator_admin_permissions = true), so the same credentials you deploy with can administer the cluster afterward.

AWS services and actions the deploy uses

The table below lists the AWS services and representative actions the deploy exercises, by category. It reflects what the module and its upstream VPC/EKS modules create β€” treat it as the breadth of access the deploying principal needs, not a hand-minimized least-privilege policy.

CategoryServiceRepresentative actions
VPC & networking (new-cluster path)ec2Create*/Delete*/Describe*/Modify* on Vpc, Subnet, RouteTable, Route, InternetGateway, NatGateway, Address (EIP), SecurityGroup/SecurityGroupRules; CreateTags/DeleteTags
EKS cluster & node groupseksCreateCluster, DescribeCluster, CreateNodegroup, CreateAddon, CreateAccessEntry, AssociateAccessPolicy, TagResource; plus ec2 launch-template/ENI actions and iam:CreateServiceLinkedRole for EKS
IAM & OIDC (IRSA)iamCreateRole, GetRole, CreatePolicy, AttachRolePolicy, PutRolePolicy, CreateOpenIDConnectProvider, GetOpenIDConnectProvider, TagRole/TagPolicy, PassRole
KMSkmsCreateKey, CreateAlias, DescribeKey, EnableKeyRotation, PutKeyPolicy, ScheduleKeyDeletion, TagResource
Secrets ManagersecretsmanagerCreateSecret, PutSecretValue, DescribeSecret, GetSecretValue, TagResource, DeleteSecret
ACM & Route 53acm, route53acm:RequestCertificate/DescribeCertificate/AddTagsToCertificate; route53:ListHostedZones/GetHostedZone/ChangeResourceRecordSets/GetChange (DNS-01 validation)
IdentitystsGetCallerIdentity; eks get-token for the kubernetes/helm providers (which then act against the cluster's Kubernetes API using the cluster-admin access above β€” not via IAM)

For a tuned least-privilege policy, generate one from a captured terraform apply (for example with IAM Access Analyzer policy generation from CloudTrail).

3. Choose your deployment path

The Terraform module supports two paths. Decide which applies before configuring it:

PathWhen to useAdditional inputs
New clusterYou want Terraform to create the VPC and EKS cluster.None β€” uses defaults (cluster name monte-carlo; the created VPC spans the three AZs the default HA topology needs).
Existing clusterYou already run an EKS cluster and want to deploy the platform into it.cluster.existing_cluster_name, networking.existing_vpc_id, and networking.existing_private_subnet_ids in different Availability Zones β€” three AZs for the default HA topology (two minimum for a single-replica deployment via the self-managed Helm install), one subnet per AZ.

Whichever path you choose, confirm the cluster has a NetworkPolicy engine enabled β€” on EKS, the VPC CNI's network policy agent. The platform isolates ClickHouse Keeper's client port with a Kubernetes NetworkPolicy that's only enforced when such an agent is present; see High availability.

πŸ“˜

Existing cluster β€” three things to check:

  1. OIDC provider. If your cluster's OIDC provider was created outside Terraform, import it before applying:

    terraform import 'module.ao_data_platform.aws_iam_openid_connect_provider.cluster[0]' <arn>
  2. Existing controllers. If cert-manager, the External Secrets Operator, the AWS Load Balancer Controller, or external-dns are already installed, set the matching helm.install_* flag to false to skip reinstalling them (see Installation).

  3. HA node groups. The module creates the per-AZ ClickHouse and Keeper node groups only on the new-cluster path. On an existing cluster you attach them yourself β€” see High availability.

4. Configure domains and DNS

The OpenTelemetry Collector and ClickHouse are each exposed through a Network Load Balancer (NLB) with a DNS name and TLS. Decide on:

  • A domain for the OpenTelemetry Collector endpoint (e.g. otel.acme.com) β€” passed as otel_collector_domain
  • A domain for the ClickHouse endpoint (e.g. clickhouse.acme.com) β€” passed as clickhouse_domain
  • A Route 53 hosted zone ID (hosted_zone_id) covering those domains

When hosted_zone_id is set, Terraform creates the IRSA roles for cert-manager (ACME DNS-01 validation) and external-dns (automatic CNAME management). Leave it unset to manage DNS records manually.

5. Enable Bedrock model access

The LLM worker runs evaluations against Claude models on Amazon Bedrock. The module grants the worker IAM permission to invoke them, but each account must enable access to the Claude models once, per region, in the Bedrock console under Model access β€” there is no Terraform resource or API for this step. Enable them in the region the worker calls (helm.llm_worker.bedrock_region, default your deployment region). Do it ahead of the deploy, or the worker's evaluation calls fail until access lands.

6. Get the platform artifacts

All three artifacts are public β€” no Monte Carlo-issued credentials or registry login are required:

ArtifactLocationVersion
Terraform moduleTerraform Registry β€” monte-carlo-data/ao-data-platform/aws2.3.0
ao-data-platform Helm chartDocker Hub (OCI) β€” oci://registry-1.docker.io/montecarlodata/ao-data-platform4.0.0
ao-llm-worker imageDocker Hub β€” montecarlodata/ao-llm-worker1.1.0-aws

The worker image is published with per-cloud tags: pinned releases like 1.1.0-aws, plus a floating latest-aws tag. The module defaults to latest-aws; pin a released tag for production.

You don't fetch these by hand: terraform init pulls the module from the Registry, the helm provider pulls the chart during terraform apply, and the EKS nodes pull the image at runtime. You set the chart registry and version (and pin the image tag) as module inputs on the installation page.

πŸ“˜

Some optional features require a minimum chart version (noted in the configuration reference).

7. Run the preflight check

Confirm your environment is ready. Each command below should succeed before you proceed to installation:

# Terraform is installed and >= 1.3
terraform version

# AWS credentials resolve to the target account
aws sts get-caller-identity

# kubectl is installed (server check comes after the cluster exists)
kubectl version --client

# If you set hosted_zone_id: the hosted zone resolves
aws route53 get-hosted-zone --id <hosted_zone_id>

# Optional: confirm you can reach the public chart on Docker Hub (no login required)
helm pull oci://registry-1.docker.io/montecarlodata/ao-data-platform --version 4.0.0

If aws sts get-caller-identity returns the wrong account, or terraform version reports below 1.3, fix those before continuing.

Next steps

Continue to Installation.


Did this page help you?