AWS PrivateLink
About
For AWS resources or deployments that are not publicly accessible, or if you prefer to connect privately, you can utilize VPC endpoints. This helps ensure that traffic between the Monte Carlo Platform and the service can connect without being exposed to the internet. For additional details, please refer to the AWS PrivateLink Overview.
To get started, please follow the guide appropriate for your deployment type after reviewing the requirements:
If you are unsure about the deployment type you are using, please see the definitions below or reach out to your account representative.
Definitions and Terminology
To help with clarity, we have defined the following terms and phrases:
-
Cloud Deployments: Refers to customers who connect to an integration without deploying any infrastructure. This term is synonymous with SaaS deployments.
-
Hybrid Deployments: Refers to customers who connect to an integration using an Agent or have a Data Store. For more details, please see the definition here.
-
Integrations: This term encompasses a variety of connections supported by the Monte Carlo Platform, independently of the developer toolkit.
Refer to the documentation here to determine your deployment type and check if it is supported.
Requirements
- Monte Carlo Role and Subscription: You must have the Account Owner role in Monte Carlo. Additionally, your account must be subscribed to either the Scale or Enterprise tier with an ESP (Enterprise Support Plan) and a supported deployment type.
- AWS Admin: Administrative access to AWS is necessary for the setup.
- Additional Admin Roles: Depending on the specific integration, you may also need administrative rights. Additionally, please note that certain integrations require specific subscription tiers and configurations with their respective vendors. Refer to your vendor's documentation for more details.
Please see the FAQs for further information on supported integrations, regions, and limitations.
For Cloud Deployments
Leveraging PrivateLink between the Monte Carlo Platform (cloud service) and an integration varies based on the resource. Please follow the guide that is relevant to the integration you wish to add.
Before getting started, make sure to check the region of your resource. VPC endpoints are not supported across regions, so ensure you are in the correct region and that it is supported by Monte Carlo before proceeding.
If you prefer, you can also use the AWS API, CLI, or any other tool of your choice to manage and update any configuration.
Redshift Provisioned
With Redshift, you can leverage either managed endpoints or interface-type VPC endpoints. Managed endpoints are easier to configure but come with certain considerations and requirements, such as the need for RA3 node types, cluster relocation, Multi-AZ configurations, and specific ports. For more details, see this page.
Managed Endpoint
-
Navigate to Redshift Console
Navigate to the Properties tab for your cluster in the Redshift console. -
Grant Access
In the Granted Accounts section, select Grant Access.Enter
590183797493
for the AWS account ID and select the corresponding VPC for your region from the table below:Region VPC af-south-1 vpc-0cfd9701e114b98cd ap-northeast-1 vpc-0dc5ec4cbc174a8e2 ap-northeast-2 vpc-04fc76089beab82a4 ap-northeast-3 vpc-0f2127db072700cdf ap-south-1 vpc-067f8326f7c39d65b ap-south-2 vpc-01437cd90f859efeb ap-southeast-1 vpc-0b95942891b157ec6 ap-southeast-2 vpc-0a34738b84f60b2ef ap-southeast-3 vpc-04466388bef4b43b2 ap-southeast-4 vpc-0f491dc21979ed836 ca-central-1 vpc-0aa1b261db6ec642c ca-west-1 vpc-0ac4e8af328f6b763 eu-central-1 vpc-0d6ca800d0f5723e4 eu-central-2 vpc-00fd1adfa38e3914f eu-north-1 vpc-08c174e3e1b47ab71 eu-south-1 vpc-039443cee88b151e0 eu-south-2 vpc-085a3691aa226b341 eu-west-1 vpc-086d1843a199397a0 eu-west-2 vpc-0a3f13360a128f8de eu-west-3 vpc-0bf729f30810d159e il-central-1 vpc-015145a29408d3e2c sa-east-1 vpc-0651dc4141953b5cd us-east-1 vpc-0e8005de91e3543ac us-east-2 vpc-0765359e13aa4b632 us-west-1 vpc-04ab0f24ed1443bb1 us-west-2 vpc-0409245d6d5e4cf66 -
Save Cluster Identifier, AWS Account ID, and Region
From the General Information section, save the Cluster Identifier and the AWS Account ID/Region where the cluster is deployed. You can retrieve the region and account ID from the Cluster Namespace ARN found in the same section. -
Request PrivateLink Establishment
Contact us at [email protected] to request the establishment of a VPC endpoint for AWS.
Be sure to include the following information in your message:- Monte Carlo Account ID (can be found here)
- Integration type (i.e., Redshift Provisioned Managed)
- Cluster identifier
- AWS region
- AWS account ID
Note: Do not proceed until you receive confirmation from Monte Carlo that the endpoint has been created.
Typically, you will receive a response within 24-48 hours (US business days).
- Onboard Redshift Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo. Importantly, when entering the Host, use the value provided by Monte Carlo. This would be the VPCE DNS name.
Interface-Type endpoint
-
Create an Endpoint Service
Follow the steps outlined here to create an endpoint service.When creating and registering the target group (Step 1 in the link above), you can use the following values:
-
Target type - IP addresses
You can find the IP addresses under the "Network Interfaces" section for the cluster (note that there might be more than one interface). Additionally, you can use a tool like nslookup on the endpoint. -
Protocol : Port - TCP 5439
This might vary if you selected a non-standard port when creating your cluster. Check under Properties -> Database Configuration -> Port.
-
-
Onboard Redshift Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo. Importantly, when entering the Host, use the value provided by Monte Carlo. This would be the VPCE DNS name.
Redshift Serverless
With Redshift, you can leverage either managed endpoints or interface-type VPC endpoints. Managed endpoints are easier to configure but come with certain considerations and requirements. For more details, see this page.
Managed Endpoint
-
Navigate to Redshift Console
Navigate to the Workgroup configuration subsection of the Redshift Console and select the relevant workgroup. -
Grant Access
Under Granted accounts in the Data access section, select Grant Access.Enter
590183797493
for the AWS account ID and select the corresponding VPC for your region from the table below:Region VPC af-south-1 vpc-0cfd9701e114b98cd ap-northeast-1 vpc-0dc5ec4cbc174a8e2 ap-northeast-2 vpc-04fc76089beab82a4 ap-northeast-3 vpc-0f2127db072700cdf ap-south-1 vpc-067f8326f7c39d65b ap-south-2 vpc-01437cd90f859efeb ap-southeast-1 vpc-0b95942891b157ec6 ap-southeast-2 vpc-0a34738b84f60b2ef ap-southeast-3 vpc-04466388bef4b43b2 ap-southeast-4 vpc-0f491dc21979ed836 ca-central-1 vpc-0aa1b261db6ec642c ca-west-1 vpc-0ac4e8af328f6b763 eu-central-1 vpc-0d6ca800d0f5723e4 eu-central-2 vpc-00fd1adfa38e3914f eu-north-1 vpc-08c174e3e1b47ab71 eu-south-1 vpc-039443cee88b151e0 eu-south-2 vpc-085a3691aa226b341 eu-west-1 vpc-086d1843a199397a0 eu-west-2 vpc-0a3f13360a128f8de eu-west-3 vpc-0bf729f30810d159e il-central-1 vpc-015145a29408d3e2c sa-east-1 vpc-0651dc4141953b5cd us-east-1 vpc-0e8005de91e3543ac us-east-2 vpc-0765359e13aa4b632 us-west-1 vpc-04ab0f24ed1443bb1 us-west-2 vpc-0409245d6d5e4cf66 -
Save Workgroup, Endpoint, AWS Account ID, and Region
From the General Information section, save the Workgroup name, Endpoint, and the AWS Account ID/Region where the workgroup is deployed. You can retrieve the region and account ID from the Workgroup ARN or endpoint found in the same section. -
Request PrivateLink Establishment
Contact us at [email protected] to request the establishment of a VPC endpoint for AWS.
Be sure to include the following information in your message:- Monte Carlo Account ID (can be found here)
- Integration type (i.e., Redshift Serverless Managed)
- Workgroup name
- Endpoint
- AWS region
- AWS account ID
Note: Do not proceed until you receive confirmation from Monte Carlo that the endpoint has been created.
Typically, you will receive a response within 24-48 hours (US business days).
- Onboard Redshift Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo. Importantly, when entering the Host, use the value provided by Monte Carlo. This would be the VPCE DNS name.
Interface-Type endpoint
-
Create an Endpoint Service
Follow the steps outlined here to create an endpoint service.When creating and registering the target group (Step 1 in the link above), you can use the following values:
-
Target type - IP addresses
You can find the IP address by using a tool likenslookup
on the endpoint (this should be the URL without the port or database):nslookup <ENDPOINT> Server: ... Address: ... Non-authoritative answer: ... Name: ... Address: 192.158.1.38 # Note that there might be more than one please include all
-
Protocol : Port - TCP 5439
This might vary if you selected a non-standard port when creating your cluster
-
-
Onboard Redshift Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo. Importantly, when entering the Host, use the value provided by Monte Carlo. This would be the VPCE DNS name.
Databricks
Databricks does not support PrivateLink in
us-west-1
. For other vendor requirements, see here.
-
Request PrivateLink Establishment
Contact us at [email protected] to request the establishment of a VPC endpoint for AWS.Be sure to include the following information in your message:
- Monte Carlo Account ID (can be found here)
- Integration type (i.e., Databricks)
- Databricks workspace URL
- AWS Region
- AWS Account ID
Note: Do not proceed until you receive confirmation from Monte Carlo that the endpoint has been created.
Typically, you will receive a response within 24-48 hours (US business days).
-
Register the the endpoint
Navigate to the "Cloud Resources" page in your Databricks account and select "VPC Endpoints." Then, choose "Register VPC Endpoint" and fill out the form. Monte Carlo will provide you with the VPCE ID in response to your message from Step 1. Make sure you select the same region as mentioned above. -
Onboard Databricks Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo.
Snowflake
This feature requires Business Critical (or higher) in Snowflake. For other vendor requirements, see here.
- Contact Snowflake Support
Open a support case with Snowflake to authorize the MC AWS account, and ensure you provide the following information:- Your Snowflake account URL
- The Monte Carlo AWS Account ID:
590183797493
- A request to allowlist the above AWS Account ID for use with AWS PrivateLink.
Note: Do not proceed until you receive confirmation from Snowflake that the account has been approved.
-
Execute Snowflake Command
As anACCOUNTADMIN
in Snowflake, execute the following command and save the results:USE ROLE ACCOUNTADMIN; SELECT SYSTEM$GET_PRIVATELINK_CONFIG();
-
Request PrivateLink Establishment
Contact us at [email protected] to request the establishment of a VPC endpoint for AWS.
Be sure to include the following information in your message:- Monte Carlo Account ID (can be found here)
- Integration type (i.e., Snowflake)
- AWS Region
- Results of the command executed in Step 2.
Note: Do not proceed until you receive confirmation from Monte Carlo that the endpoint has been created.
Typically, you will receive a response within 24-48 hours (US business days).
- Onboard Snowflake Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo. Importantly, when entering the ACCOUNT, use the value provided by Monte Carlo in response to step 3. The account should generally follow this format:{account}.{region}.privatelink
.
If you are also using network policies in Snowflake to restrict external access, please see details here on how to allowlist Monte Carlo's IPs in addition to using PrivateLink.
Tableau Server on EC2
-
Create an Endpoint Service
Follow the steps outlined here to create an endpoint service.When creating and registering the target group (Step 1 in the link above), you can use the following values:
- Target type - Instances or Load Balancer if applicable (Note: This should not be the TSM load balancer)
- Protocol : Port - HTTPS 443
- Health check protocol - Same as the protocol above.
-
Onboard Tableau Integration
Continue to follow this guide to create the relevant service account and register the integration in Monte Carlo.
Database on RDS
-
Create an Endpoint Service
Follow the steps outlined here to create an endpoint service.When creating and registering the target group (Step 1 in the link above), you can use the following values:
-
Target type - IP addresses
You can find the IP address by using a tool likenslookup
on the endpoint (this should be the URL without the port or database):nslookup <ENDPOINT> Server: ... Address: ... Non-authoritative answer: ... Name: ... Address: 192.158.1.38
-
Protocol : Port - TCP varies (defaults: 5432 for PostgreSQL, 3306 for MySQL, 1433 for SQL Server)
-
-
Onboard Integration
Continue to follow the appropriate guide to create the service account and register the integration in Monte Carlo. Importantly, when entering the Host or Server name, use the value provided by Monte Carlo. This would be the VPCE DNS name.
Database on EC2
-
Create an Endpoint Service
Follow the steps outlined here to create an endpoint service.When creating and registering the target group (Step 1 in the link above), you can use the following values:
- Target type - Instances
- Protocol : Port - TCP varies (defaults: 5432 for PostgreSQL, 3306 for MySQL, 1433 for SQL Server)
-
Onboard Integration
Continue to follow the appropriate guide to create the service account and register the integration in Monte Carlo. Importantly, when entering the Host or Server name, use the value provided by Monte Carlo. This would be the VPCE DNS name.
For Hybrid Deployments
For all accounts on the V2 platform, Monte Carlo will use VPC endpoints to communicate with the AWS agent and data store by default in supported regions*.
Ingress vs. Egress
Please note that using VPC endpoints for egress between the agent and your integrations differs from using them for ingress. Only the ingress scenario is described in this section.
For more information, please refer to the details provided here.
Agent
By default, VPC endpoints facilitate communication to the AWS agent on the V2 platform*. No additional actions are required to enable this. If you wish to further constrain the use of VPC endpoints, please refer to this guide.
Data store
By default, VPC endpoints facilitate communication to the AWS data store on the V2 platform*. No additional actions are required to enable this. If you wish to further constrain the use of VPC endpoints, please refer to this guide.
*Features and functionality that rely on pre-signed URLs, such as the Data Explorer, do not use VPC endpoints.
FAQs
How Do I Know What Type of Deployment I Am Using and If It Is Supported?
If your account was created after April 24th, 2024, it will automatically be using the V2 platform or newer.
To check the status of your deployment, follow these steps using our API or Command Line Interface (CLI).
API
-
Access the API Explorer:
Visit the API Explorer in the Monte Carlo UI (learn more about the API Explorer here).
Alternatively, you can generate an API key and use tools such as cURL or Postman to make API calls. -
Trigger the API:
Use this API to fetch deployment details. For instance:query getPlatformServices { getPlatformServices { uuid deployment { deploymentType } } }
CLI
- Install and Configure the CLI:
If you haven't done so already, follow the installation and configuration instructions. Ensure you have at least versionv0.100.0
of the CLI. - Execute the Command:
Open your terminal and run the following command (reference docs):montecarlo platform list
Either way, you can use the following table to interpret the output and determine the status of your deployment:
Type | Platform | Description | Supported |
---|---|---|---|
CLOUD_V1 | V1 | Legacy cloud deployment | No |
CLOUD_V2 | V2 | Cloud deployment using the new platform | Yes |
REMOTE_V1 | V1 | Legacy hybrid deployment (data collector) | No |
REMOTE_V1.5 | V1.5 | Hybrid deployment (remote agent or data store) not using the new platform | No |
REMOTE_V2 | V2 | Hybrid deployment (remote agent or data store) using the new platform | Yes |
Please see the FAQs for further information on supported integrations, regions, and limitations.
What Integrations Are Supported?
If on a compatible platform (see table above), Monte Carlo supports using VPC Endpoints with the following:
- Cloud deployments:
- AWS Redshift Provisioned
- AWS Redshift Serverless
- Databricks on AWS
- Snowflake on AWS
- Various instances and databases on AWS, such as PostgreSQL, MySQL (MySQL on EC2 is not supported and MySQL on RDS requires using an RDS Proxy. Note that the AWS RDS proxy does not support read replicas), Oracle, Tableau Server, etc.
- Hybrid deployments:
- AWS Agents
- AWS Data Stores
Note that for hybrid deployments, this refers to connectivity from the Monte Carlo platform to the agent. See further details here.
What Regions are Supported?
Supported regions include:
Supported Regions |
---|
us-east-1 |
us-east-2 |
us-west-1 |
us-west-2 |
af-south-1 |
ap-south-1 |
ap-south-2 |
ap-southeast-1 |
ap-southeast-2 |
ap-southeast-3 |
ap-southeast-4 |
ap-northeast-1 |
ap-northeast-2 |
ap-northeast-3 |
ca-central-1 |
ca-west-1 |
eu-central-1 |
eu-central-2 |
eu-west-1 |
eu-west-2 |
eu-west-3 |
eu-north-1 |
eu-south-1 |
eu-south-2 |
il-central-1 |
sa-east-1 |
Does the use of VPC endpoints limit any Monte Carlo features or have other limitations?
Limitations include:
- Adding the same VPC endpoint to multiple Monte Carlo workspaces is not supported.
- After enabling VPC endpoints, you cannot use the host without using the VPC endpoint.
- Databricks cannot be added or managed via Partner Connect to leverage PrivateLink.
- PrivateLink might not be used with certain Snowflake queries. This might be the case for large result sets when Snowflake internally stores the results in S3 (e.g., stage).
- Features and functionality that rely on pre-signed URLs, such as the Data Explorer, do not use VPC endpoints.
Please see the FAQs for further information on supported integrations, regions, and deployments.
What Is the Difference Between Using a PrivateLink for Ingress vs. Egress in Agent Hybrid Deployments?
- Ingress: Connectivity from the Monte Carlo Platform to the agent.
- Egress: Connectivity from the agent to your integrations.
The current document outlines how to enable ingress connectivity. Similar to other networking configurations (such as connecting a Virtual Network), you can also use VPC endpoints to further connect an agent to your integrations once it is deployed in your cloud, if you prefer. For additional details on constraining outbound access (egress) from the agent, see here.
What if I want to allowlist Monte Carlo's IPs in addition to using PrivateLink?
If you are using network policies to further restrict access, you can add your deployment’s internal IP addresses to the allowlist. Note that these are different from the public IP addresses the cloud service uses to connect when not using endpoints.
To do so, you can fetch the IP address for your deployment using the getPlatformServices API. For instance:
- Access the API Explorer:
Visit the API Explorer in the Monte Carlo UI (learn more about the API Explorer here).
Alternatively, you can generate an API key and use tools such as cURL or Postman to make API calls. - Trigger the API:
Use the getPlatformServices API to fetch deployment details. For instance:query getPlatformServices { getPlatformServices { uuid deployment { deploymentType platform { ipAddresses } } } }
Importantly, unlike public IP addresses, these internal IPs are not guaranteed to be static. Although they are generally stable, they may occasionally change due to various factors. Therefore, if you want to supplement your usage of PrivateLink with an allowlist, please periodically update it with the results from this endpoint for the relevant deployment. We recommend creating a job that runs daily to compare the results. Otherwise, there might be partial outages in connectivity.
Note that if you have more than one deployment and/or want to see which connections are using a specific deployment, you can also do so via the API. Please refer to these docs for more information.
How Can I Debug or Test Connectivity When I Am Using VPC Endpoints?
Even though each network configuration is unique, you can try the following steps to help debug connectivity:
-
Double Check Connection Details
Verify the connection details provided to Monte Carlo, such as host, port, database, and user, for any typos or omissions. -
Confirm Service User Functionality
Ensure that the service user you created is working correctly (e.g., you are able to log in as the service user). -
Use Monte Carlo Network Utilities
These two utilities are available on the UI by navigating to Settings->Integrations->Test Network, as well as via the CLI and API (Tcp Open and Telnet).
Note that you can use our API from the UI via the API Explorer. Learn more about the API Explorer here.
Updated 8 days ago