📝

Prerequisites

1. You are an admin in AWS.
2. You have admin permissions in your data warehouse
3. You are an Account Owner.

This guide outlines how to setup an Agent (with object storage) and the OpenTelemetry Collector in your AWS cloud using CloudFormation.

These FAQs answer common questions like how to review resources and what integrations are supported.

📘

If you already have the Monte Carlo Agent deployed to your cloud vendor and want to add the OpenTelemetry Collector to your deployment, skip Step 1-2 and reference this FAQ. Afterwards, continue with Step 3.

Steps

1. Deploy the Agent

Before getting started please review the Monte Carlo AWS account your collection service is hosted in.

Specifying a VPC is required to run the agent and enables certain connectivity scenarios like when you have an IP allowlist for your resource, want to peer, or deploy in your existing VPC. See more details here.

Deploy with CloudFormation

You can use the quick-create link below to deploy the Agent with OpenTelemetry Collector on any supported region in your AWS account:

If you need to share with a colleague or first review the template you can download a copy here (source).

📘

Note: by default the OpenTelemetry Collector deployed with the Monte Carlo Agent will only allow ingress from resources associated with the SecurityGroup created by the CF template. You might need to provide the ExistingSecurityGroupId parameter to the stack in order for the OpenTelemetry Collector to receive incoming traces from your Agent. These SecurityGroup definition will vary depend on your network configuration.

If you wish to use an existing S3 bucket to store the OpenTelemetry trace data, specify the ARN of the existing bucket in parameter ExistingOpenTelemetryCollectorBucketArn. By default the Data Store S3 bucket created by the MC Agent deployment will be used to store the OpenTelemetry trace data unless this parameter is specified.

2. Register the Agent

After deploying the agent you can register either via the Monte Carlo UI or CLI.

And see here for examples on how to retrieve deployment output (i.e. registration input).

UI

1. Navigate to settings/integrations/agents and select the Create button.
2. Follow the onscreen wizard for the "AWS" Platform and "Data Store + Agent" Type.

CLI

Use montecarlo agents register-aws-agent to register.

See reference documentation here. And see here for how to install and configure the CLI. For instance:

montecarlo agents register-aws-agent \
  --lambda-arn arn:aws:lambda:us-east-1:123456789:function:mcd-agent-AgentLambda \
  --assumable-role arn:aws:iam::123456789:role/mcd-agent-InvocationRole-12345 \
  --external-id f3840b31-772e-4fe3-8a5f-3aa5ff7e6fec

3. Configure your Data Warehouse Ingestion Pipeline

📘

Prerequisite: Data Warehouse S3 Access Configuration

Before continuing, your data warehouse must be configured to access the AWS S3 bucket containing the OpenTelemetry trace data. If your data warehouse is not currently configured to access the S3 bucket, refer to the guides below for Monte Carlo's recommendation on how to configure S3 access in your data warehouse.

Warehouse Vendor	Guide
Snowflake	Configure Snowflake Storage Integration and Stage
Databricks	Configure Databricks External Location

Next, we need to configure your data ingestion pipeline to write the OpenTelemetry trace data from S3 to your data warehouse so it can be monitored by Monte Carlo. Follow the guide relevant to your data warehouse vendor for steps on how to configure this pipeline.

4. Configure your AI Agent

Congrats! You have now configured the Monte Carlo Agent and OpenTelemetry Collector to process traces from your AI agent and write them to your data warehouse.

The final step is to configure your AI agent to begin sending traces to the OpenTelemetry Collector.

1. Add the Monte Carlo OpenTelemetry SDK to your AI agent's source code.
2. Use the OpenTelemetryCollectorHTTPEndpoint output from the CloudFormation stack deployed earlier as the URL to provide to the Monte Carlo OpenTelemetry SDK.
3. Follow the Monte Carlo OpenTelemetry SDK library's instructions to configure instrumentation.

You can now validate the deployment is working as expected by observing files being written to the S3 bucket and data being ingested into the relevant table in your warehouse.

FAQs

What if I already deployed the Monte Carlo Agent?

If you already have the Monte Carlo Agent deployed to your cloud vender, you can deploy the OpenTelemetry Collector separately alongside it via CloudFormation.

Before getting started please review the Monte Carlo AWS account your collection service is hosted in.

Specifying a VPC is required to run the collector and enables certain connectivity scenarios like when you have an IP allowlist for your resource, want to peer, or deploy in your existing VPC. See more details here. Be sure to use the same VPC associated to your Monte Carlo Agent.

You can use the quick-create link below to deploy the OpenTelemetry Collector in your AWS account alongside your existing Agent:

If you need to share with a colleague or first review the template you can download a copy here (source).

📘

Note: by default the OpenTelemetry Collector deployed with the Monte Carlo Agent will only allow ingress from resources associated with the SecurityGroup created by the CF template. You might need to provide the ExistingSecurityGroupId parameter to the stack in order for the OpenTelemetry Collector to receive incoming traces from your Agent. These SecurityGroup definition will vary depend on your network configuration.

You will be required to specify the ARN of the existing bucket to store telemetry trace data in parameter TelemetryDataBucketArn.

Continue with Step 3 above.

Can I review agent resources and code?

Absolutely! You can find details here:

*Note that due to an AWS limitation the agent image is also uploaded and then sourced from AWS ECR when executed on Lambda.

Repository: 590183797493.dkr.ecr.*.amazonaws.com/mcd-agent

Additional FAQs?

Other applicable FAQs for deploying the Monte Carlo Agent to AWS can be found here.