📝

Prerequisites

1. You are an admin in Azure and have installed Terraform (>= 1.3) with Azure Authentication (for step 1).
2. You are an Account Owner (for step 2).

This guide outlines how to setup an Agent (with object storage) in your Azure cloud.

The FAQs answer common questions like how to review resources and what integrations are supported.

Steps

1. Deploy the Agent

You can use the mcd-agent Terraform module to deploy the Agent and manage resources as code (IaC).

For instance, using the following example Terraform config:

module "apollo" {
  source  = "monte-carlo-data/mcd-agent/azurerm"
  version = "0.1.2"
}

output "resource_group" {
  value       = module.apollo.mcd_agent_resource_group_name
  description = "Agent service resource group."
}

output "function_url" {
  value       = module.apollo.mcd_agent_function_url
  description = "The URL for the agent."
}

output "function_name" {
  value       = module.apollo.mcd_agent_function_name
  description = "Agent function name."
}

You can build and deploy via:

terraform init && terraform apply

Note that this module is configured to delete all resources when the resource group is deleted (e.g. when executing terraform destroy). Please take appropriate measures and review your resources before doing so. See here.

Additional module inputs, options, and defaults can be found here. And other details can be found here.

2. Register the Agent

After deploying the agent you can register either via the Monte Carlo UI or CLI.

And see here for examples on how to retrieve Terraform output (i.e. registration input). Note that the Application Key is the "Default Host Key" in Azure Function Apps.

After this step is complete all supported integrations will automatically use this agent (and object store for troubleshooting and temporary data). You can add these integrations as you normally would using Monte Carlo's UI wizard or CLI.

UI

👍

If you are onboarding a new account, you can also register by following the steps on the onscreen

1. Navigate to settings/integrations/agents and select the Create button.
2. Follow the onscreen wizard for the "Azure" Platform and "Data Store + Agent" Type.

CLI

Use montecarlo agents register-azure-agent to register.

See reference documentation here. And see here for how to install and configure the CLI.

montecarlo agents register-azure-agent --url https://mcd-agent-service-example.azurewebsites.net --app-key -1

FAQs

What integrations does the Agent support?

The agent supports all integrations. Exceptions:

• Data Lake Query Logs from S3 Buckets: https://docs.getmontecarlo.com/docs/s3-events-query-logs

Note that onboarding (connecting) any supported integration will use the agent if one is provisioned. Otherwise any other integrations will use your automatically managed and hosted data collection service to connect directly.

Some integrations like dbt core, Atlan, and Airflow either leverage our developer toolkit or are managed by a 3rd party and do not require an Agent. These integrations natively push data to Monte Carlo so an Agent is not required.

Can I review agent resources and code?

Absolutely! You can find details here:

How do I retrieve registration input from Terraform?

The endpoint (url) can be retrieved via: terraform output function_url.

The application key (default) can be fetched via the az functionapp keys list command using the function name and resource group from terraform output. For instance:

az functionapp keys list -g $(terraform output -raw resource_group) -n $(terraform output -raw function_name) | jq -r '.functionKeys.default'

If you prefer this value can also be retrieved from the Azure portal from Function Apps service page. For instance:

How do I monitor the Agent?

The Agent automatically generates a log of all operations, which can retrieved via this CLI command or the Azure portal from the Function Apps service page.

Operational metrics can similarly be retrieved from the Function Apps service page.

How do I upgrade the Agent?

If you have opted out of remote upgrades (i.e. set remote_upgradable = false) you can upgrade the agent image by setting the image variable in the module.

Please reach out to your Monte Carlo representative or support at [email protected] for the correct tag for your deployment.

Otherwise, and by default, Monte Carlo will automatically manage upgrades for you. If this is the case and you'd still like to explicitly upgrade you can do so via the upgrade command on the CLI or "Upgrade" button on the UI.

Can I further constraint inbound access (ingress) to the Agent?

👍

Updated IPs

For all accounts created after April 24th, 2024 the Monte Carlo platform will generally use the following IP addresses to connect to your integration (cloud-only), agent (GCP and Azure), and/or object store:

• 34.200.118.118
• 35.169.25.209

Please be sure to allowlist both as requests from the Monte Carlo Platform* can originate from either one. If your account was created before this date, please reach out to your Monte Carlo representative.

*If you are leveraging a Customer-hosted Agent these are not the same as the IP addresses that the agent will use to connect to your resource. See "Egress" FAQs per platform for more details and options to constraint outbound access.

Absolutely! By default this is done via the function app key, but if you prefer you can further restrict requests via an IP allowlist. For instance you can:

1. Reach out to your Monte Carlo representative or support at [email protected] for an IP Address to allowlist. All inbound requests to the agent will originate here.
2. Update the configuration for the agent's function app. For instance on the Azure portal*:
   1. Navigate to the Networking setting and select the "Public network access" configuration.
   2. Update the network access to "Enabled from select virtual networks and IP addresses" and add a rule for the IP address from step #1. Save your configuration.

*Note that changes made on the Azure portal might be reverted via Terraform. You might want to consider using the ip_restriction block to manage as code.

Can I further constraint outbound access (egress) from the Agent?

Absolutely! As with any Azure Function you can control egress in multiple ways. See details here.

Depending on your integration this might be necessary to establish connectivity.

How do I check the reachability between Monte Carlo and the Agent?

Reachability is automatically validated during registration, but you can also use this CLI command or "test" button on the UI to test anytime.

How do I debug connectivity between the Agent and my integration?

Even though each network configuration is unique, you can try the following to help debug connectivity:

1. Double check the connection details provided to Monte Carlo, such as host, port, database, user for typos/omissions.
2. Confirm that the service user you created works (e.g. you are able to log in as the service user).
3. Use the MC network utilities on the integrations page. These utilities are also available via the CLI.
   1. Test TCP Open: Tests if a destination exists and accepts requests. Opens a TCP Socket to a specific port from the agent.
   2. Test Telnet: Checks if Telnet connection is usable from the agent.