📝

Prerequisites

1. A running Kubernetes cluster (any distribution — e.g. EKS, AKS, GKE, k3s, kind, minikube, or on-premises).
2. kubectl and helm CLI tools installed and configured to access your cluster.
3. You are an Account Owner in Monte Carlo.

This guide walks through an example of deploying the Generic Agent on a Kubernetes cluster using Helm. The steps are platform-agnostic and can work on any Kubernetes distribution.

👍

The examples below use k3s (via Rancher Desktop) as a reference environment, but all steps apply to any Kubernetes cluster. Platform-specific guides for managed Kubernetes services (EKS, AKS, GKE) — including cloud-native storage and secrets management — are available separately.

Steps

1. Register the Agent

2. Deploy the Agent

2.1 Deploy Object Storage

The agent requires a supported object storage service. You can use a cloud-native service (e.g. Amazon S3, Azure Blob Storage, Google Cloud Storage) or any S3-compatible service (e.g. MinIO).

This guide uses MinIO deployed in the same cluster as an example:

kubectl apply -f https://raw.githubusercontent.com/monte-carlo-data/hermes-agent/main/environments/local/minio/k8s.yaml

This creates a minio namespace with a PersistentVolumeClaim, Deployment, and Services.

After MinIO is running, create a bucket called mcd-agent-storage:

kubectl port-forward -n minio deploy/minio 9000:9000 9001:9001

Open http://localhost:9001/, log in with your MinIO credentials, and create the bucket.

⚠️

MinIO with default credentials is suitable for development and testing only. For production deployments, use a cloud-native storage service or configure MinIO with proper credentials, TLS, and persistent storage.

2.2 Create the Agent Namespace

kubectl create namespace mcd-agent

Label and annotate the namespace so Helm can manage it:

kubectl label namespace mcd-agent app.kubernetes.io/managed-by=Helm
kubectl annotate namespace mcd-agent meta.helm.sh/release-name=mcd-agent meta.helm.sh/release-namespace=default

2.3 Create Secrets

The agent requires two Kubernetes secrets before deployment.

Agent Token

1. Using the mcd_id and mcd_token obtained during registration, create the token.
2. Create a file named agent-token.json with those values:

{
  "mcd_id": "<your-mcd-id>",
  "mcd_token": "<your-mcd-token>"
}

3. Create the secret:

kubectl create secret generic mcd-agent-token-secret -n mcd-agent \
  --from-file=contents.json=agent-token.json

Integrations Secrets

This secret holds credentials for data source integrations. It can start empty and be populated later:

kubectl create secret generic mcd-integrations-secrets -n mcd-agent \
  --from-file=empty.json=<(echo '{}')

2.4 Configure and Deploy with Helm

Create a values.yaml file with your deployment configuration:

namespace: mcd-agent
replicaCount: 1

image:
  pullPolicy: Always
  tag: "latest-generic"

container:
  backendServiceUrl: "<your-backend-service-url>"
  storageBucketName: "mcd-agent-storage"
  storageType: "S3_COMPATIBLE"
  storageEndpointUrl: "http://minio-api.minio.svc.cluster.local:9000"
  storageAccessKey: "<your-minio-access-key>"
  storageSecretKey: "<your-minio-secret-key>"

skipExternalSecrets: true

📘

Set backendServiceUrl to the Public endpoint shown in the Agent Service section of the Account Information page in Monte Carlo.

Deploy the agent:

📘

Check Docker Hub for the latest available chart version and update the --version flag accordingly.

helm upgrade --install mcd-agent \
  oci://registry-1.docker.io/montecarlodata/generic-agent-helm \
  --version 0.1.0 \
  -f values.yaml

2.5 Verify

Check the agent pod is running:

kubectl get pods -n mcd-agent
kubectl logs -n mcd-agent -l app=mcd-agent --tail=30

Run the reachability test to confirm the agent can communicate with the Monte Carlo platform:

kubectl exec -n mcd-agent deploy/mcd-agent-deployment -- \
  curl -s -X POST localhost:8080/api/v1/test/reachability

A successful response contains "ok": true.

3. Enable the Agent

After verifying the agent is running, click Enable on the agent registration screen.

📘

If you've navigated away from the registration screen, go to Settings > Deployments, select your agent, and click Enable.

Once validations pass, click Enable in the validations dialog to activate the agent.

Adding Integration Credentials

To connect data sources, replace the empty integrations secret with one containing connection details:

kubectl delete secret mcd-integrations-secrets -n mcd-agent
kubectl create secret generic mcd-integrations-secrets -n mcd-agent \
  --from-file=<integration>.json=<path-to-credentials>.json
kubectl rollout restart deployment/mcd-agent-deployment -n mcd-agent

See the Self-Hosted Credentials documentation for the JSON format for each integration type and how to register the credentials in Monte Carlo.

Updating the Agent Version

To deploy a newer version, update the --version flag and the image.tag in your values.yaml, then re-run the helm upgrade command.

Tearing Down

helm uninstall mcd-agent
kubectl delete namespace mcd-agent

If you deployed MinIO:

kubectl delete -f https://raw.githubusercontent.com/monte-carlo-data/hermes-agent/main/environments/local/minio/k8s.yaml

FAQs

What Kubernetes distributions are supported?

The Generic Agent runs on any Kubernetes cluster that supports standard workloads — including EKS, AKS, GKE, k3s, kind, minikube, OpenShift, and on-premises distributions.

Does the agent require inbound network access?

No. The Generic Agent is egress-only. All connections are initiated from the agent to the Monte Carlo platform. No inbound connectivity to the agent is required.

Can I use a different S3-compatible storage service instead of MinIO?

Yes. The agent works with any S3-compatible storage. Set storageType to S3_COMPATIBLE and update storageEndpointUrl, storageBucketName, storageAccessKey, and storageSecretKey in your values.yaml to point to your storage service.

How do I add credentials for data source integrations?

The mcd-integrations-secrets secret created during setup starts empty. To add credentials for an integration, replace it with a secret containing the connection details:

kubectl delete secret mcd-integrations-secrets -n mcd-agent
kubectl create secret generic mcd-integrations-secrets -n mcd-agent \
  --from-file=<integration>.json=<path-to-credentials>.json
kubectl rollout restart deployment/mcd-agent-deployment -n mcd-agent

You can include multiple integration files in the same secret by passing additional --from-file flags.

See the Self-Hosted Credentials documentation for the JSON format for each integration type.

After adding the secret, register the integration in Monte Carlo. You can do this via the UI (see Self-Hosted Credentials) or via the CLI:

montecarlo integrations add-self-hosted-credentials-v2 \
  --connection-type <integration> \
  --self-hosted-credentials-type FILE \
  --file-path /etc/secrets/integrations/<integration>.json \
  --name <connection_name>

How do I scale the agent?

There are several ways to scale the agent:

Manual replicas: Increase replicaCount in your values.yaml:

replicaCount: 3

Thread count: Increase the number of concurrent operations a single replica can process:

container:
  opsRunnerThreadCount: "36"

The default is 18. Increasing this value allows each replica to handle more operations in parallel, which can be useful before adding more replicas.

Pod resources: Set CPU and memory requests/limits to ensure replicas have adequate resources, especially when increasing thread count:

container:
  resources:
    requests:
      cpu: "500m"
      memory: "512Mi"
    limits:
      cpu: "2"
      memory: "2Gi"

Autoscaling (HPA): Enable the Horizontal Pod Autoscaler to scale replicas automatically based on CPU (and optionally memory) utilization:

autoscaling:
  enabled: true
  minReplicas: 1
  maxReplicas: 5
  targetCPUUtilizationPercentage: 70

When autoscaling is enabled, replicaCount is ignored — the HPA manages replicas. container.resources.requests must be set for the HPA to calculate utilization, and metrics-server must be installed in the cluster (standard in EKS, AKS, and GKE).

How do I monitor the agent?

You can monitor the agent using standard Kubernetes monitoring tools such as Prometheus + Grafana, or the monitoring tools provided by your Kubernetes platform. The agent exposes a health endpoint at /api/v1/test/healthcheck that can be used for liveness and readiness probes.

How do I rotate the agent token?

1. Delete and recreate the Kubernetes secret with the new credentials:

   Bash

   kubectl delete secret mcd-agent-token-secret -n mcd-agent
   kubectl create secret generic mcd-agent-token-secret -n mcd-agent \
     --from-file=contents.json=agent-token.json

2. Restart the agent services:

   Bash

   kubectl rollout restart deployment mcd-agent-deployment -n mcd-agent
   kubectl rollout restart daemonset logs-collector metrics-collector -n mcd-agent

How do I route agent traffic through a forward proxy?

Configure HTTPS_PROXY, HTTP_PROXY, and NO_PROXY environment variables using container.extraEnv in your values.yaml. See the Proxies and Traffic Inspection guide for details. A complete example using Squid is available in the mcd-public-resources repository.

How do I inspect HTTPS traffic from the agent?

To audit the full content of HTTPS traffic — including URLs, headers, and payloads — you can use TLS inspection tools such as mitmproxy, Charles Proxy, or Fiddler. This requires configuring both a forward proxy and a custom CA certificate. See the Proxies and Traffic Inspection guide for details. A complete example using mitmproxy is available in the mcd-public-resources repository.