AWS: Data Store Deployment

How-to create and register

πŸ“

Prerequisites

  1. You are an admin in AWS (for step 1).
  2. You are an Account Owner (for step 2).

This guide outlines how to setup a Data Store for storing troubleshooting and temporary data in your AWS cloud.

Steps

1. Deploy

You can deploy the object storage and associated resources in your AWS cloud using the following quick-create link:

And if you need to share with a colleague or first review the template you can download a copy here (source).

When provisioning resources for Monte Carlo deployments on the V2 Platform, use the Monte Carlo AWS Account ID 590183797493. Accounts created after April 24th, 2024, will automatically be on the V2 platform or newer.

If you are using an older version of the platform, please contact your Monte Carlo representative for the ID.

2. Register

After creating the bucket and role you can register either via the Monte Carlo UI or CLI.

After this step is complete all integrations that you add to this collection service will automatically use this bucket for storing troubleshooting and temporary data. You can add these integrations as you normally would using Monte Carlo's UI wizard or CLI.

UI

πŸ‘

If you are onboarding a new account, you can also register by following the steps on the onscreen


  1. Navigate to settings/integrations/agents and select the Create button.
  2. Follow the onscreen wizard for the "AWS Data Store" Platform type. The relevant values can be retrieved from the CloudFormation Outputs for the stack created in step 1. Note that the External ID is required if using this stack.
AWS Data Store Creation Wizard

AWS Data Store Registration Wizard

CLI

Use the command montecarlo agents register-s3-store to register.

For reference on this command please see here. And see here for how to install and configure the CLI.

The relevant values can be retrieved from the CloudFormation Outputs for the stack created in step 1. Note that the External ID is required if using this stack.

montecarlo agents register-s3-store \
  --bucket-name example-bucket \
  --assumable-role arn:aws:iam::123456789:role/mcd-test-apollo-role \
  --external-id 1234567890

FAQs

Can I further constraint access to this Data Store (S3 Bucket)?

For all deployments on the V2 Platform or newer, Monte Carlo uses VPC endpoints to communicate with AWS Object Storage in supported regions. Additional details and limitations can be found here.

If you wish to constraint this you can update the IAM policy permissions to include a aws:SourceVpce condition.

For instance, you can add the following element to the s3-policy document:

"Condition": {
    "StringEquals": {
        "aws:SourceVpce": "<VPCE_ID>"
    }
}

For reference the mapping per supported region can be found below:

RegionVPCE ID
us-east-1vpce-004394176d889792d
us-east-2vpce-090022b0ef5e83ba9
us-west-1vpce-0e16caf67c05371fe
us-west-2vpce-07802dc845e1bdc33
af-south-1vpce-07de4a15107004567
ap-south-1vpce-07d66ddd8818bb7c9
ap-south-2vpce-0ef033dc31dc15fc1
ap-southeast-1vpce-0ea18c9725c52de53
ap-southeast-2vpce-0268e5ba31f63745a
ap-southeast-3vpce-026d7688a2beb3324
ap-southeast-4vpce-08d83169799eb7a32
ap-northeast-1vpce-02e680d8515648a0b
ap-northeast-2vpce-0fa4b42dc2a0dcea5
ap-northeast-3vpce-05a7c91d103abf35f
ca-central-1vpce-0ba5e11a63241e656
ca-west-1vpce-025b9264ccfdc0cf4
eu-central-1vpce-0c031174af3b40c5f
eu-central-2vpce-0a643e6db02e5b4f3
eu-west-1vpce-026aa8ed1be81e54c
eu-west-2vpce-0ebbd41740f0c2951
eu-west-3vpce-0d5af6bade8e5244c
eu-north-1vpce-091b17d5a8bb076e8
eu-south-1vpce-09734ac9c572f135b
eu-south-2vpce-0c24407352e9d0267
il-central-1vpce-090d9f377c40ebce0
sa-east-1vpce-05607f980709214e9

Note that if you want to use features that rely on pre-signed URLs, such as the Data Explorer, you also need to add the following statement to the s3-policy document when using this constraint:

{
    "Action": [
        "s3:GetObject"
    ],
    "Resource": [
        "arn:aws:s3:::<BUCKET_NAME>/*"
    ],
    "Effect": "Allow",
    "Condition": {
        "StringEquals": {
            "s3:signatureversion": "AWS4-HMAC-SHA256",
            "s3:authType": "REST-QUERY-STRING"
        }
    }
}

If your deployment is not on the V2 Platform, you can still further constrain access with the aws:SourceIp condition. Please Reach out to your Monte Carlo representative or support at [email protected] for an IP Address to allowlist. Then, for instance, you can add the following element to the s3-policy document:

"Condition": {
    "IpAddress": {
        "aws:SourceIp": [
            "<IP>/32"
        ]
    }
}

For more information on connectivity, please refer to our Network Connectivity documentation.

How do I check the reachability between Monte Carlo and the Data Store?

Reachability is automatically validated during registration, but you can also use this CLI command or "test" button on the UI to test anytime.