Announcement: Platform Migrations

2024 Platform Migrations and Updates

About

Monte Carlo is upgrading its internal collection platform architecture and will be phasing out the Data Collector. To support this upgrade, customers must complete certain actions within specified timeframes to maintain seamless connectivity to active integrations.

👍

Note: If you have not been contacted by Monte Carlo, please disregard the following information until further notice. No action may be required for your deployment at this time.

If you are unsure whether this applies to you, to which deployment it applies if you have more than one, or cannot find the original message that indicated the affected deployment, please see this FAQ.

Thank you for your cooperation during this transition period. For most customers, these actions should be quick and straightforward, such as updating an IP allowlist, modifying an account list, or performing an upgrade. If you have any questions or require assistance, please contact us at [email protected]. We are always happy to assist you! The FAQs provided here can also help answer some common questions

Definitions and terminology

To ensure clarity, we have defined the following terms and phrases:

Cloud Deployments: Refers to customers who connect to an integration without deploying any infrastructure. Previously Monte Carlo would internally host and manage a Data Collector for you. This term is synonymous with SaaS deployments.

Hybrid Deployments: Refers to customers who connect to an integration using an Agent, Data Store, or remote Data Collector (defined here). Note that the remote Data Collector is considered legacy.

Integrations: This term encompasses a variety of connections supported by the Monte Carlo Platform, independent of the developer toolkit. These include, but are not limited to:

  • Data Warehouses (e.g., Snowflake, Databricks, Redshift)
  • BI Tools (e.g., Looker, Tableau)
  • Data Lakes (e.g., Glue, Athena, Presto)
  • Others (e.g., Fivetran, dbt Cloud)

What actions must be taken?

If you have been contacted or notified, you must complete one or more of the following steps to ensure ongoing connectivity to the platform. Please complete these steps by the specified dates listed on the timeline; otherwise, your integration may be disabled.

These steps vary based on your deployment type and configuration. Be sure to complete only the steps relevant to your specific deployment type, as indicated in the message you received. Note that if you have multiple deployments, you may have received multiple notifications.

If you are unsure which deployment it applies to if you have more than one, or you cannot find the original message that indicated the affected deployment, please see this FAQ.

Once you have completed the necessary steps, no further action is required to migrate. Monte Carlo will automatically detect and migrate all compatible deployments in batches.

For Cloud Deployments

To determine the necessary changes, you can use either our Command Line Interface (CLI) or our API for detailed instructions on what needs to be updated, if anything.

Transition Instructions

Step 1: Determining the Necessary Changes

Using the CLI (recommended)
  1. Install and Configure the CLI:
    If you haven't done so already, follow the installation and configuration instructions. Ensure you have at least version v0.100.0 of the CLI and that you are either an Account Owner or using the Account Owner assignment.

  2. Execute the Command:
    Open your terminal and run the following command (reference docs). Note that it may take a few minutes for the tests to complete:

    montecarlo platform test-migration
    

    Please Note: If you have multiple deployments, you will need to provide a service-id with the command above. You can find the service-id by using the following command (reference docs):

    montecarlo platform list
    
Using the API
  1. Access the API Explorer:
    Visit the Monte Carlo API Explorer. Alternatively, you can generate an API key and use tools such as cURL or Postman to make API calls.

  2. Trigger the Test:
    Use this API to initiate an async test. This command returns a migrationUuid to be used in step 3.

  3. Fetch the Results:
    Use this API to retrieve the test results. Keep in mind that tests might take a few minutes to complete, so the results may not be immediately available. Note that the CLI automatically handles polling and waiting for test results, whereas with the API, you may need to manually check for completion.

Regardless of whether you use the CLI or the API, please proceed to step 2 afterward.

Step 2: Updating IP Addresses

👍

This step is only necessary if you are using IP allowlisting. Otherwise, please continue to step 3.

If the CLI command or API response indicates that you need to update connectivity for any integrations and you are using IP allowlisting:

  1. Preserve Existing IP Address: Do not remove the current Monte Carlo IP address from your IP restriction list.
  2. Add both New IP Addresses:
    • 34.200.118.118
    • 35.169.25.209

Otherwise please continue to step 3.

Step 3: Updating IAM Role's Trust Policy

👍

This step is only necessary if you a policy that needs to be updated. Otherwise, please continue to step 4.

If the CLI command or API response indicates you have an integration where the IAM role's trust policy needs to be updated:

  1. Update Trust Policy: Update the IAM role's trust policy to grant access to the following new AWS account: 590183797493.
  2. Preserve Existing Trust Principals: Do not remove the previous Monte Carlo AWS account ID from the trust principal during the transition period. Migrations will occur in weekly batches, which may result in overlap periods.

Otherwise please continue to step 4.

Step 4: Handling Private Link or Peering

👍

This step is only necessary if you are using Private Link or Peering. Otherwise, please continue to step 5.

If the CLI command or API response indicates that you need to update connectivity and you are using using private link or peering:

  1. Contact Support: Email us at [email protected] for further assistance.
    • Note: Peering is no longer supported for Cloud Deployments. We are happy to discuss alternatives.
    • Private Link: Supported but requires coordination with our team.

Otherwise please continue to step 5.

Step 5: Rechecking and Finalizing

  1. Verify Changes: Repeat steps 2-4 for all integrations. You can rerun the CLI command or API call from Step 1 to confirm the changes. If the CLI or API indicates that no further action is required, Monte Carlo will automatically detect and migrate all compatible deployments in batches.

See here for details on when you can start removing any old (previous) IP addresses or accounts. Please ensure that you do not remove any IP addresses or accounts before those conditions are met.

For Hybrid Deployments (v1.5)

AWS Agents or Data Stores

To ensure compatibility with Monte Carlo's platform, customers utilizing remote AWS agents or data stores must grant access to the new AWS account: 590183797493.

Transition Instructions

To comply with these requirements, simply upgrade your AWS agent or data store configuration. This process is automatically handled by our templates. Do not remove the previous Monte Carlo AWS account ID from the trust principal during the transition period, as migrations will occur in batches, which might lead to overlapping periods.

  • AWS Remote Agent using CloudFormation: Update the stack with the latest template here (source). Do not change the "Monte Carlo AWS Account ID" parameter to 590183797493. This is done automatically.

  • AWS Object Store using CloudFormation: Update the stack using the latest template here (source). Do not change the "Monte Carlo AWS Account ID" parameter to 590183797493. This is done automatically.

  • AWS Remote Agent using Terraform: Update to the latest version (0.1.5 or later) of the module. Do not change the "cloud_account_id" variable to 590183797493. This is done automatically.

Please note, if you are currently using an IP allowlist to restrict inbound access to an AWS data store or agent, this method will no longer be supported. Instead, the Monte Carlo platform will utilize private endpoints to communicate with the AWS agent and data stores by default. Therefore, we kindly ask you to remove the IP allowlist constraint. You can still apply restrictions using VPC Endpoints (VPCe). If this change affects you, please contact us at [email protected], and we will be happy to assist you with the transition.

Important: After completing this, please refer to the Cloud Deployment guidelines to determine whether any actions are necessary for the integrations you are using with this deployment. You may also need to update IP allowlists, roles, etc.

Azure Agents or Data Stores

If you've previously restricted inbound access (ingress) to the agent or data store, please add Monte Carlo's new IP addresses to ensure uninterrupted connectivity.

If no IP allowlist has been set up, no action is required.

Transition Instructions

  1. Preserve Existing IP Address: Do not remove the current Monte Carlo IP address from your IP restriction list.
  2. Add both New IP Addresses:
    • 34.200.118.118
    • 35.169.25.209

For detailed instructions on modifying these settings, refer to the provided guides for the agent and data store.

Important: After completing this, please refer to the Cloud Deployment guidelines to determine whether any actions are necessary for the integrations you are using with this deployment. You may also need to update IP allowlists, roles, etc.

See here for details on when you can start removing any old (previous) IP addresses or accounts. Please ensure that you do not remove any IP addresses or accounts before those conditions are met.

GCP Agents or Data Stores

If you've previously restricted inbound access (ingress) to the agent or data store, please add Monte Carlo's new IP addresses to ensure uninterrupted connectivity.

If no IP allowlist has been set up, no action is required.

Transition Instructions

  1. Preserve Existing IP Address: Do not remove the current Monte Carlo IP address from your IP restriction list.
  2. Add both New IP Addresses:
    • 34.200.118.118
    • 35.169.25.209

For detailed instructions on modifying these settings, refer to the provided guides for the agent and data store.

Important: After completing this, please refer to the Cloud Deployment guidelines to determine whether any actions are necessary for the integrations you are using with this deployment. You may also need to update IP allowlists, roles, etc.

See here for details on when you can start removing any old (previous) IP addresses or accounts. Please ensure that you do not remove any IP addresses or accounts before those conditions are met.

For Hybrid Deployments (legacy)

AWS Data Collector

Please see the documentation here.

Timeline

For Cloud Deployments

Please complete any required actions by September 30th, 2024. Failure to do so may result in the disabling of your integrations, which will affect Monte Carlo's ability to monitor assets and generate alerts, insights, and detections.

For Hybrid Deployments (v1.5)

Please complete any required actions by September 30th, 2024. Failure to do so may result in the disabling of your integrations, which will affect Monte Carlo's ability to monitor assets and generate alerts, insights, and detections.

For Hybrid Deployments (legacy)

Please check back soon. We expect to have more information in the upcoming months.

FAQs

How can I learn more about the platform and new deployment options?

To learn more about our platform and explore new deployment options, please visit our documentation.

How do I know if any action is required for my account?

Account owners will be notified if any action is required. If you have not been contacted by Monte Carlo, no action is necessary at this time. Please note that these migrations are being conducted in phases, so your timeline might vary.

If you cannot locate your email or are otherwise unsure if action is required, you can use this command to determine your deployment type and interpret whether any action is required at this time.

If an action is required, please follow the guide available here before the corresponding date on the timeline here.

What happens if I don't complete the required actions by the indicated timeline?

Failure to complete the required actions by the specified dates may result in the disabling of your integrations. This means Monte Carlo will no longer be able to monitor assets or generate alerts, insights, and detections for all integrations using that deployment.

What should I do if I need an extension?

Extensions are considered on a case-by-case basis. If you require an extension, please contact us as soon as possible at [email protected].

How do I validate my integrations?

Before migrating any resources, it's a good idea to verify that your integration is functioning as expected. Monte Carlo makes this easy with built-in validators accessible through the Monte Carlo UI. To run a test, follow these steps:

  1. Navigate to the relevant integration here.
  2. Click on the three-dot menu next to it.
  3. Select the "Test" button.

Monte Carlo also automatically validates your integrations during the migration process. If you want to conduct a manual validation post-migration, you can rerun the tests anytime.

Validator Example

Validator Example

How do I check if my deployment has been migrated?

To check the status of your deployment, follow these steps using our Command Line Interface (CLI):

  1. Install and Configure the CLI:
    If you haven't done so already, follow the installation and configuration instructions. Ensure you have at least version v0.100.0 of the CLI.

  2. Execute the Command:
    Open your terminal and run the following command (reference docs):

    montecarlo platform list
    
  3. Interpret the Output:
    Review the "Deployment Type" displayed by this command to determine the status of your deployment:

    TypeDescriptionMigration RequiredActions needed
    CLOUD_V1Legacy cloud deploymentYesUse this command to check if further actions are needed
    CLOUD_V2Cloud deployment using the new platformNoN/A
    REMOTE_V1Legacy hybrid deployment (data collector)YesPlease check back soon. We expect to have more information in the upcoming months
    REMOTE_V1.5Hybrid deployment (remote agent or data store) not using the new platformYesUse this command to check if further actions are needed.
    REMOTE_V2Hybrid deployment (remote agent or data store) using the new platformNoN/A

If you prefer, you can also use this API to fetch deployment details via the API Explorer or a tool like cURL or Postman to make API calls. For instance:

query getPlatformServices {
  getPlatformServices {
    uuid
    deployment {
      deploymentType
    }
  }
}

How do I obtain the relevant ID for my deployments?

You can use the same command outlined here. The ID are listed under "Service ID".

How do I test if my integrations can be migrated?

For Cloud and Hybrid (v1.5) deployments, you can use our CLI to test if any actions are necessary. Follow these steps:

  1. Install and Configure the CLI:
    If you haven't done so already, follow the installation and configuration instructions. Ensure you have at least version v0.100.0 of the CLI and that you are either an Account Owner or using the Account Owner assignment.

  2. Execute the Command:
    Open your terminal and run the following command (reference docs). Note that it may take a few minutes for the tests to complete:

    montecarlo platform test-migration
    

    Please Note: If you have multiple deployments, you will need to provide a service-id with the command above. You can find the service-id by using the following command (reference docs):

    montecarlo platform list
    

After running this command, please follow the steps outlined here before the corresponding dates listed here.

When can I remove old (A.K.A. previous) IP addresses and accounts from my allowlists?

If the output of this command indicates that your deployment is using the new platform (i.e., CLOUD_V2 or REMOTE_V2), you can safely remove any old IP addresses and accounts from your integration allowlists or configuration.

However, do not remove any old IP addresses or accounts if the command output shows any other type.

Additionally, please be aware that Monte Carlo might begin releasing the old IP addresses back into the global pool at least 30 days after the dates specified on the timeline.

Resources and support

If you have any questions, please don't hesitate to reach out to us at [email protected]. We are always happy to assist you!