Using self-hosted credentials
Monte Carlo provides the option to store integration credentials in your AWS account using Secrets Manager. Note: Self-hosting Tableau credentials is currently not supported.
To configure, follow these steps -
- Create a read-only service account or role for the integration.
See detailed instructions in our dedicated guides for Warehouses, Lakes and BI Tools. - Create a secret in your AWS account.
- Create a role that allows secret access for Monte Carlo's data collector.
- Set up the integration using your self-hosted credentials.
Create a secret in your AWS account
You will create a secret containing the credentials for the integration's service account or role.
- Select and fill in the credentials schema associated with the resource you wish to integrate with.
{
"catalog": {
"type": "string",
"required": false
},
"assumable_role": {
"type": "string",
"required": false
},
"external_id": {
"type": "string",
"required": false
},
"workgroup": {
"type": "string",
"required": false
},
"aws_region": {
"type": "string",
"required": false
}
}
{
"type": {
"type": "string",
"required": true
},
"project_id": {
"type": "string",
"required": true
},
"private_key_id": {
"type": "string",
"required": true
},
"private_key": {
"type": "string",
"required": true
},
"client_email": {
"type": "string",
"required": true
},
"client_id": {
"type": "string",
"required": true
},
"auth_uri": {
"type": "string",
"required": true
},
"token_uri": {
"type": "string",
"required": true
},
"auth_provider_x509_cert_url": {
"type": "string",
"required": true
},
"client_x509_cert_url": {
"type": "string",
"required": true
}
}
{
"bucket": {
"type": "string",
"required": true
},
"prefix": {
"type": "string",
"required": true
},
"assumable_role": {
"type": "string",
"required": false
},
"external_id": {
"type": "string",
"required": false
}
}
{
"assumable_role": {
"type": "string",
"required": false
},
"external_id": {
"type": "string",
"required": false
},
"aws_region": {
"type": "string",
"required": false
}
}
{
"host": {
"type": "string",
"required": true
},
"port": {
"type": "integer",
"required": false,
"nullable": false
},
"username": {
"type": "string",
"required": true
},
"database": {
"type": "string",
"required": false,
"nullable": true
},
"password": {
"type": "string",
"required": true
}
}
{
"db_name": {
"type": "string",
"required": true
},
"host": {
"type": "string",
"required": true
},
"port": {
"type": "string",
"required": true
},
"user": {
"type": "string",
"required": true
},
"password": {
"type": "string",
"required": true
},
"assumable_role": {
"type": "string",
"required": false
},
"external_id": {
"type": "string",
"required": false
},
"ssl_options": {
"type": "dict",
"required": false,
"schema": {
"ca": {
"type": "string",
"required": false
},
"cert": {
"type": "string",
"required": false
},
"key": {
"type": "string",
"required": false
},
"mechanism": {
"type": "string",
"required": false,
"allowed": ["url"]
}
}
}
}
{
"base_url": {
"type": "string",
"required": true
},
"client_id": {
"type": "string",
"required": true
},
"client_secret": {
"type": "string",
"required": true
},
"verify_ssl": {
"type": "boolean",
"required": false
}
}
{
"ssh_key": {
"type": "string",
"required": false
},
"repo_url": {
"type": "string",
"required": true
},
"token": {
"type": "string",
"required": false
},
"username": {
"type": "string",
"required": false
}
}
{
"host": {
"type": "string",
"required": true
},
"port": {
"type": "integer",
"required": false
},
"user": {
"type": "string",
"required": true
},
"catalog": {
"type": "string",
"required": false
},
"schema": {
"type": "string",
"required": false
},
"timeout": {
"type": "integer",
"required": false
},
"http_scheme": {
"type": "string",
"required": false
},
"password": {
"type": "string",
"required": false
},
"ssl_options": {
"type": "dict",
"required": false,
"schema": {
"ca": {
"type": "string",
"required": false
},
"cert": {
"type": "string",
"required": false
},
"key": {
"type": "string",
"required": false
},
"mechanism": {
"type": "string",
"required": false,
"allowed": ["dc-s3"]
},
"skip_verification": {
"type": "boolean",
"required": false
}
}
}
}
{
"db_name": {
"type": "string",
"required": true
},
"host": {
"type": "string",
"required": true
},
"port": {
"type": "string",
"required": true
},
"user": {
"type": "string",
"required": true
},
"password": {
"type": "string",
"required": true
}
}
{
"user": {
"type": "string",
"required": true
},
"password": {
"type": "string",
"required": true
},
"account": {
"type": "string",
"required": true
},
"warehouse": {
"type": "string",
"required": false
}
}
{
"mode": "binary",
"host": {
"type": "string",
"required": false
},
"port": {
"type": "integer",
"required": false,
"nullable": true
},
"username": {
"type": "string",
"required": false
},
"database": {
"type": "string",
"required": false,
"nullable": true
},
"password": {
"type": "string",
"required": false
}
}
{
"mode": "databricks",
"databricks_workspace_url": {
"type": "string",
"required": false
},
"databricks_workspace_id": {
"type": "string",
"required": false
},
"databricks_cluster_id": {
"type": "string",
"required": false
},
"databricks_token": {
"type": "string",
"required": false
}
}
{
"mode": "http",
"username": {
"type": "string",
"required": false
},
"password": {
"type": "string",
"required": false
},
"url": {
"type": "string",
"required": false
}
}
- Go to Secrets Manager on your AWS console and click "Store a new secret".
- Select "Other type of secrets" and paste the filled-in schema from step 1. Then select "Next".
Example with Snowflake
- Give the secret a meaningful name, description and any tags. Then select "Next".
- Configure automatic rotation, if you'd like. Then select "Next".
- Review and select "Store".
- From the Secrets Manager console search for the secret you just created. Save the ARN.
Create a role that allows secret access for Monte Carlo's data collector
You will create an IAM role with the necessary API permissions in order to access to the secret:
- Copy the policy below. Please specify the ARN from above where
<secret_arn>appears.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "secretsmanager:GetSecretValue",
"Resource": "<secret_arn>"
}
]
}
- Follow the steps outlined here to create the IAM role. You will attach the policy from step 1 to this role as part of the process.
Set up the integration
You will provide connection details using Monte Carlo's CLI:
- Follow this guide to install and configure the CLI (requires >=
0.13.2). - Use the command
montecarlo integrations add-self-hosted-credentialsto add the integration.
For reference, see help below:
$ montecarlo integrations add-self-hosted-credentials --help
Usage: montecarlo integrations add-self-hosted-credentials
[OPTIONS]
Setup an integration that uses self-hosted credentials.
Options:
--connection-type [hive-mysql|hive-s3|hive|presto|presto-s3|glue|athena|spark|redshift|snowflake|bigquery|looker|looker-git-ssh|looker-git-clone]
Type of connection. This option requires
setting 'name' when it is set to one of
these values: {'bigquery', 'snowflake',
'redshift'}. [required]
--mechanism [secretsmanager] Credential self-hosting mechanism.
[required]
--key TEXT Identifier for credentials within self-
hosting mechanism. (ARN for the secret) [required]
--name TEXT Friendly name for the warehouse. Required
for warehouses.
--role TEXT Assumable role ARN to use for accessing AWS
resources.
--external-id TEXT An external id, per assumable role
conditions.
--collector-id UUID ID for the data collector. To disambiguate
accounts with multiple collectors.
--skip-validation Skip all connection tests. This option
cannot be used with 'validate-only'.
--validate-only Run connection tests without adding. This
option cannot be used with 'skip-
validation'.
--auto-yes Skip any interactive approval. [default:
False]
--option-file FILE Read configuration from FILE.
--help Show this message and exit.
FAQs
What if I want to migrate an existing integration to use self-hosted credentials?
After creating the secret and role the command montecarlo integrations update can be used to transfer credential management.
Note - The command montecarlo integrations list can be used to echo the <CONNECTION-ID>.
$ montecarlo integrations update --replace-all --connection-id <CONNECTION-ID> --changes '{"self_hosting_mechanism": "secretsmanager", "self_hosting_key": "<SECRET_ARN>", "assumable_role": "<ROLE_ARN>", "external_id": "<EXTERNAL_ID>", "region": "<AWS_REGION>"}'
Updated about 2 months ago
Did this page help you?