📝

Prerequisites

1. Docker and Docker Compose installed.
2. You are an Account Owner in Monte Carlo.

This guide walks through a basic example of deploying the Generic Agent with Docker Compose. This example uses MinIO for S3-compatible object storage, making the deployment fully self-contained.

👍

An example is available in the mcd-public-resources repository.

The steps below explain each part of the setup.

Steps

1. Register the Agent

2. Deploy the Agent

2.1 Create the Directory Structure

mkdir -p mcd-agent/secrets/integrations
cd mcd-agent

2.2 Create the Docker Compose File

Create a docker-compose.yml file:

services:
  mcd-agent:
    image: montecarlodata/agent:${AGENT_IMAGE_TAG:-latest-generic}
    ports:
      - "8080:8080"
    environment:
      - PORT=8080
      - BACKEND_SERVICE_URL=${BACKEND_SERVICE_URL}
      - MCD_AGENT_WRAPPER_TYPE=DOCKER
      - MCD_STORAGE_BUCKET_NAME=mcd-agent-storage
      - MCD_STORAGE=S3_COMPATIBLE
      - MCD_STORAGE_ENDPOINT_URL=http://minio:9000
      - MCD_STORAGE_ACCESS_KEY=${MINIO_ROOT_USER}
      - MCD_STORAGE_SECRET_KEY=${MINIO_ROOT_PASSWORD}
      - MCD_TOKEN_FILE_PATH=/etc/secrets/mcd-agent-token/contents.json
    volumes:
      - ./secrets/token.json:/etc/secrets/mcd-agent-token/contents.json:ro
      - ./secrets/integrations:/etc/secrets/integrations:ro
    restart: unless-stopped
    depends_on:
      create-bucket:
        condition: service_completed_successfully

  create-bucket:
    image: quay.io/minio/mc:latest
    depends_on:
      minio:
        condition: service_healthy
    environment:
      - MC_HOST_local=http://${MINIO_ROOT_USER}:${MINIO_ROOT_PASSWORD}@minio:9000
    entrypoint: ["mc", "mb", "local/mcd-agent-storage", "--ignore-existing"]

  minio:
    image: quay.io/minio/minio:latest
    command: server /data --console-address ":9001"
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      - MINIO_ROOT_USER=${MINIO_ROOT_USER}
      - MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD}
    volumes:
      - minio-data:/data
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  minio-data:

Create a .env file with your environment-specific values:

BACKEND_SERVICE_URL=<YOUR_BACKEND_SERVICE_URL>
MINIO_ROOT_USER=minioadmin
MINIO_ROOT_PASSWORD=change-me-to-a-secure-password
# Optional: pin the agent image version (defaults to latest-generic)
# AGENT_IMAGE_TAG=0.8.0-generic

Docker Compose automatically reads .env when you start the stack.

📘

Set BACKEND_SERVICE_URL to the Public endpoint shown in Account Information > Agent Service in the Monte Carlo app.

2.3 Create the Agent Token

Using the mcd_id and mcd_token obtained during registration, create the token file:

cat > secrets/token.json << 'EOF'
{"mcd_id": "<YOUR_MCD_ID>", "mcd_token": "<YOUR_MCD_TOKEN>"}
EOF
chmod 600 secrets/token.json

2.4 Start All Services

docker compose up -d

This starts MinIO, automatically creates the storage bucket, and launches the agent.

2.5 Verify

Check that the agent is running:

docker compose logs -f mcd-agent

Run the reachability test to confirm the agent can communicate with the Monte Carlo platform:

curl -s -X POST http://localhost:8080/api/v1/test/reachability

A successful response contains "ok": true.

⚠️

MinIO with default credentials is suitable for development and testing only. For production deployments, configure MinIO with proper credentials and TLS, or use a cloud-native storage service.

3. Enable the Agent

After verifying the agent is running, click Enable on the agent registration screen.

📘

If you've navigated away from the registration screen, go to Settings > Deployments, select your agent, and click Enable.

Once validations pass, click Enable in the validations dialog to activate the agent.

Adding Integration Credentials

Place integration credential files in ./secrets/integrations/. They are mounted read-only into the container at /etc/secrets/integrations/.

After adding the files, restart the agent:

docker compose restart mcd-agent

See the Self-Hosted Credentials documentation for the JSON format for each integration type and how to register the credentials in Monte Carlo.

After adding the secret, register the integration in Monte Carlo. You can do this via the UI (see Self-Hosted Credentials) or via the CLI:

montecarlo integrations add-self-hosted-credentials-v2 \
  --connection-type <integration> \
  --self-hosted-credentials-type FILE \
  --file-path /etc/secrets/integrations/<integration>.json \
  --name <connection_name>

Updating the Agent

Pull the latest image and recreate the container:

docker compose pull mcd-agent
docker compose up -d mcd-agent

Tearing Down

docker compose down -v

FAQs

Does the agent require inbound network access?

No. The Generic Agent is egress-only. All connections are initiated from the agent to the Monte Carlo platform. No inbound connectivity to the agent is required.

Can I use a different S3-compatible storage service instead of MinIO?

Yes. The agent works with any S3-compatible storage. Update MCD_STORAGE_ENDPOINT_URL, MCD_STORAGE_ACCESS_KEY, MCD_STORAGE_SECRET_KEY, and MCD_STORAGE_BUCKET_NAME in the mcd-agent environment to point to your storage service.

Can I use cloud-native storage (S3, GCS, Azure Blob) instead of MinIO?

Yes. Remove the minio and create-bucket services, and update the agent environment variables:

• For Amazon S3: Set MCD_STORAGE=S3, remove MCD_STORAGE_ENDPOINT_URL, and configure AWS credentials (via environment variables or IAM roles).
• For Google Cloud Storage: Set MCD_STORAGE=GCS and configure GCP credentials.
• For Azure Blob Storage: Set MCD_STORAGE=AZURE_BLOB, set MCD_STORAGE_ACCOUNT_NAME, and configure Azure credentials.

How do I view MinIO storage contents?

The MinIO Console is accessible at http://localhost:9001. Log in with the MINIO_ROOT_USER and MINIO_ROOT_PASSWORD values from your .env file.

How do I add credentials for data source integrations?

Place JSON credential files in the ./secrets/integrations/ directory. The agent container mounts this directory read-only at /etc/secrets/integrations/. After adding files, restart the agent with docker compose restart mcd-agent.

See the Self-Hosted Credentials documentation for the JSON format for each integration type.

How do I route agent traffic through a forward proxy?

If your environment requires outbound traffic to go through a proxy, you can configure the agent with HTTP_PROXY and HTTPS_PROXY environment variables. A complete Docker Compose example using Squid as a forward proxy is available in the mcd-public-resources repository. This setup logs all destination hosts and ports the agent connects to.

How do I inspect HTTPS traffic from the agent?

To audit the full content of HTTPS traffic sent by the agent — including URLs, headers, and payloads — you can use TLS inspection tools such as mitmproxy, Charles Proxy, or Fiddler. A complete Docker Compose example using mitmproxy with a browser-based UI is available in the mcd-public-resources repository.

How do I scale the agent?

There are several ways to scale the agent:

Multiple instances: Scale horizontally by running multiple instances:

docker compose up -d --scale mcd-agent=3

Thread count: Increase the number of concurrent operations a single instance can process:

environment:
  - MCD_OPS_RUNNER_THREAD_COUNT=36

The default is 18. Increasing this value allows each instance to handle more operations in parallel, which can be useful before adding more instances.

Container resources: Set CPU and memory limits to ensure instances have adequate resources, especially when increasing thread count:

services:
  mcd-agent:
    deploy:
      resources:
        limits:
          cpus: '4'
          memory: 4G

How do I monitor the agent?

You can monitor the agent using docker compose logs and standard Docker monitoring tools. The agent exposes a health endpoint at /api/v1/test/healthcheck that returns OK when the agent is running.