📝

Prerequisites

1. You are an admin in AWS (for steps 1 and 2).
2. You are an Account Owner (for step 3).

👍

Automation with Infrastructure as Code (IaC)

This template can be used to automate steps 1 and 2 and manage resources as code with CloudFormation: https://prod-us-east-1-mcd-data-collector.s3.amazonaws.com/enablement/v0/apollo_s3_data_store.yaml

If you wish to use it you can download, review, and then deploy in your AWS account.

This guide outlines how to setup a Data Store for storing troubleshooting and temporary data in your AWS cloud.

Steps

1. Create an S3 Bucket

Use the AWS Console, CLI or favorite IaC tool to create a new S3 Bucket in your AWS account with no public access.

Note that registration (step 3) will fail if the bucket is publicly accessible.

We strongly recommend you do not use an existing bucket or share with other jobs as Monte Carlo might overwrite existing data. And we also recommend the following settings:

• SSE-S3 Server side encryption.
• Expiration lifecycles. 90 days or less for all objects in the following prefixes:

custom-sql-output-samples/
rca
idempotent

2. Create an Assumable IAM Role

Then follow the steps outlined in Creating IAM Roles to create a role in your AWS account with the policy below, replacing the BUCKET with the bucket you created above.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetBucketPublicAccessBlock",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketAcl"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET>",
                "arn:aws:s3:::<BUCKET>/*"
            ],
            "Effect": "Allow"
        }
    ]
}

All references to the Data Collector in the guide above refer to a service that is automatically hosted and managed by Monte Carlo. No additional infrastructure or resources are required.

When generating the trust policy this service will be hosted in one of the following AWS accounts:

• 190812797848
• 799135046351

And when creating a role please add MonteCarloData as a tag key or this role will be unable to be assumed by Monte Carlo.

You can see information regarding your automatically managed Data Collector service in the settings/integrations/collectors tab on the UI or via the list command on the CLI.

3. Register the Data Store

After creating the bucket and role you can register either via the Monte Carlo UI or CLI.

After this step is complete all integrations that you add to this Data Collector Service will automatically use this bucket for storing troubleshooting and temporary data. You can add these integrations as you normally would using Monte Carlo's UI wizard or CLI.

UI

1. Navigate to settings/integrations/agents and select the Create button.
2. Follow the onscreen wizard for the "AWS" Platform Type. The "Bucket name" is the bucket you created in step one and "AWS assumable role"(and optional "External ID") are the ARN (and ID) from step two.

CLI

Use the command montecarlo agents register-s3-store to register.

For reference on this command please see here. And see here for how to install and configure the CLI.

The bucket-name is the bucket you created in step one and assumable-role (and optional external-id) are the ARN (and ID) from step two.

montecarlo agents register-s3-store \
  --bucket-name example-bucket \
  --assumable-role arn:aws:iam::123456789:role/mcd-test-apollo-role \
  --external-id 1234567890

FAQs

Can I deploy resources as code?

Absolutely! This template can be used to automate steps 1 and 2 and manage resources as code with CloudFormation. If you wish to use it you can download, review, and then deploy in your AWS account.

Can I further constraint access to this Data Store (S3 Bucket)?

Absolutely! By default this is done via the defined trust policy, but if you prefer you can further restrict requests via an IP allowlist. For instance you can:

1. Reach out to your Monte Carlo representative or support at [email protected] for your dedicated IP Address. All inbound requests to the S3 data store will originate here.
2. Create a S3 bucket and Role. You can do this by following the steps here or via this automation.
3. Update the IAM Permissions policy to include a aws:SourceIp condition with the IP address from step 1. For instance:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetBucketPublicAccessBlock",
                "s3:GetBucketPolicyStatus",
                "s3:GetBucketAcl"
            ],
            "Resource": [
                "arn:aws:s3:::<BUCKET>",
                "arn:aws:s3:::<BUCKET>/*"
            ],
            "Effect": "Allow",
            "Condition": {
                "IpAddress": {
                    "aws:SourceIp": [
                        "<IP>/32"
                    ]
                }
            }
        }
    ]
}

3. Continue with registration.

How do I check the reachability between Monte Carlo and the Data Store?

Reachability is automatically validated during registration, but you can also use this CLI command or "test" button on the UI to test anytime.