Guides
ChangelogStatus PageAPI ExplorerAPI ReferenceCLI ReferenceBlogIMPACT 2025

Generic Agent: Proxies and Traffic Inspection

Route agent traffic through a proxy or inspect HTTPS traffic

You may need to route agent traffic through a corporate proxy, inspect the data the agent sends to Monte Carlo or add custom CA certificates when a firewall or proxy performs TLS interception.

This page explains the available options. For platform-specific step-by-step examples, see the links in the Proxies and Traffic Inspection section on the main page.

Forward Proxy

To route agent traffic through a forward proxy, configure the standard HTTPS_PROXY, HTTP_PROXY, and NO_PROXY environment variables on the agent.

Kubernetes (Helm)

Use container.extraEnv in your values.yaml to set the proxy environment variables:

container:
  extraEnv:
    - name: HTTPS_PROXY
      value: "http://your-proxy-host:3128"
    - name: HTTP_PROXY
      value: "http://your-proxy-host:3128"
    - name: NO_PROXY
      value: "minio-api.minio.svc.cluster.local,localhost,127.0.0.1"
πŸ“˜

Set NO_PROXY to include any internal services that should not go through the proxy, such as the object storage endpoint.

A complete example using Squid as a forward proxy is available in the mcd-public-resources repository.

Docker Compose

Set the environment variables directly on the agent service in docker-compose.yml:

environment:
  - HTTPS_PROXY=http://your-proxy-host:3128
  - HTTP_PROXY=http://your-proxy-host:3128
  - NO_PROXY=minio,localhost,127.0.0.1

A complete example using Squid is available in the mcd-public-resources repository.

Custom CA Certificates

When a proxy or firewall performs TLS interception (also known as SSL inspection or SSL bumping), it re-signs HTTPS traffic with its own certificate. The agent must trust this certificate to communicate through the proxy without TLS errors.

Kubernetes (Helm)

The Helm chart supports the firewallCa.cert value, which automatically builds a combined CA bundle (merging your custom certificate with the system certificates) and configures the agent to use it:

firewallCa:
  cert: |
    -----BEGIN CERTIFICATE-----
    <YOUR_CA_CERTIFICATE_PEM_CONTENT>
    -----END CERTIFICATE-----

The chart uses an init container to merge the custom CA with the system certificates and sets REQUESTS_CA_BUNDLE on the agent so that Python's requests library uses the combined bundle.

Alternatively, if you store the CA certificate in a cloud secret manager (e.g. Azure Key Vault, AWS Secrets Manager), you can reference it via an ExternalSecret instead of inlining the PEM content:

firewallCa:
  externalSecretRef: my-firewall-ca-secret

This creates an ExternalSecret that pulls the CA certificate from your secret store and mounts it into the agent pod. This approach is recommended for production environments where the CA certificate is managed centrally and may be rotated. See the Secrets Management guide for how to configure the secret store.

Docker Compose

Mount the CA certificate into the container and configure the agent to trust it using environment variables:

volumes:
  - ./certs/ca-cert.pem:/usr/local/share/ca-certificates/proxy-ca.crt:ro
environment:
  - REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
command: >
  sh -c "update-ca-certificates && exec gunicorn ..."

TLS Inspection (Full Traffic Audit)

To audit the full content of HTTPS traffic sent by the agent β€” including URLs, headers, and payloads β€” you can use TLS inspection tools such as mitmproxy, Charles Proxy, or Fiddler.

This approach combines a forward proxy with a custom CA certificate: the inspection tool acts as a man-in-the-middle proxy, terminating TLS connections with dynamically generated certificates signed by a local CA. The agent must trust this CA (see Custom CA Certificates above).

Complete examples using mitmproxy with a browser-based UI are available for:

Intermediate Proxy (MITM Service)

An alternative approach is to route the agent's backend connection through an intermediate service β€” for example, an Apigee proxy or similar API gateway. In this configuration:

  1. Set backendServiceUrl (or BACKEND_SERVICE_URL) to point to your intermediate service instead of the Monte Carlo endpoint.
  2. The intermediate service forwards requests to the actual Monte Carlo Agent Service.
  3. If the intermediate service uses a custom TLS certificate, add it using the firewallCa.cert configuration described above.

This pattern is useful when your organization already has an API gateway in place for outbound traffic management, logging, or policy enforcement.

Examples

AWS Network Firewall

AWS Network Firewall can be used to control and inspect outbound traffic from the agent when deployed on EKS. AWS Network Firewall supports TLS inspection, which decrypts HTTPS traffic for deep packet inspection and re-signs it with a CA certificate. The agent must trust this certificate.

πŸ“˜

If you only need connection-level visibility (destination hosts and ports) without decrypting traffic, you can use AWS Network Firewall with SNI-based rules and no TLS inspection. In that case, no CA certificate configuration is needed on the agent.

Agent configuration (TLS inspection only): When TLS inspection is enabled, the firewall re-signs traffic with its own CA certificate. Add the firewall's CA certificate so the agent trusts the re-signed traffic. You can either:

  • Store the CA certificate in AWS Secrets Manager and reference it via an ExternalSecret (recommended for production):

    firewallCa:
  externalSecretRef: <external-secret-name>

    See Secrets Management for how to configure the secret store.

  • Embed the CA certificate directly in your values.yaml:

    firewallCa:
  cert: |
    -----BEGIN CERTIFICATE-----
    <YOUR_FIREWALL_CA_CERTIFICATE_PEM_CONTENT>
    -----END CERTIFICATE-----

The Helm chart automatically builds a combined CA bundle and sets REQUESTS_CA_BUNDLE on the agent when either firewallCa option is configured. See Custom CA Certificates above for more details.

Azure Firewall

Azure Firewall can be used to control and inspect outbound traffic from the agent when deployed on AKS. When Azure Firewall performs TLS inspection, it re-signs HTTPS traffic with its own CA certificate. The agent must trust this certificate.

Azure Firewall setup: Follow the Azure Firewall documentation to deploy a firewall in your VNet and configure a route table to direct outbound traffic from the AKS subnet through the firewall.

Agent configuration: The only change required on the agent side is adding the firewall's CA certificate so the agent trusts the re-signed traffic. You can either:

  • Store the CA certificate in Azure Key Vault and reference it via an ExternalSecret (recommended for production):

    firewallCa:
  externalSecretRef: <external-secret-name>

    See Secrets Management for how to configure the secret store.

  • Embed the CA certificate directly in your values.yaml:

    firewallCa:
  cert: |
    -----BEGIN CERTIFICATE-----
    <YOUR_FIREWALL_CA_CERTIFICATE_PEM_CONTENT>
    -----END CERTIFICATE-----

The Helm chart automatically builds a combined CA bundle and sets REQUESTS_CA_BUNDLE on the agent when either firewallCa option is configured. See Custom CA Certificates above for more details.

Forward Proxy (Squid)

Squid can be used as a forward proxy to log all destination hosts and ports the agent connects to. This provides connection-level visibility without inspecting the actual request content.

Complete examples are available in the mcd-public-resources repository:

TLS Inspection (mitmproxy)

mitmproxy can be used to inspect the full content of HTTPS traffic β€” including URLs, headers, and payloads β€” through a browser-based UI. This combines a forward proxy with a custom CA certificate for TLS interception.

Complete examples are available in the mcd-public-resources repository:

Apigee API Proxy

Apigee can be used as an intermediate proxy to route, inspect, and enforce policies on agent traffic before it reaches the Monte Carlo platform. Unlike TLS-intercepting firewalls, Apigee acts as a reverse proxy β€” the agent connects directly to the Apigee endpoint, which forwards requests to the Monte Carlo Agent Service. No custom CA certificate is required.

Apigee configuration: The Generic Agent uses long-lived Server-Sent Events (SSE) connections to receive operations from the Monte Carlo platform. Apigee requires specific configuration to support these connections:

  1. Separate target endpoint for SSE traffic: Create a dedicated target endpoint for the SSE stream path (/stream) using a route rule. Non-SSE traffic can use the default target endpoint. This follows the Apigee recommendation to not mix SSE and non-SSE targets.

    <RouteRule name="sse">
  <Condition>proxy.pathsuffix MatchesPath "/stream"</Condition>
  <TargetEndpoint>sse</TargetEndpoint>
</RouteRule>
<RouteRule name="default">
  <TargetEndpoint>default</TargetEndpoint>
</RouteRule>

  2. EventFlow on the SSE target: Add an EventFlow element to the SSE target endpoint so Apigee handles the response as a stream:

    <EventFlow name="EventFlow" content-type="text/event-stream">
  <Response/>
</EventFlow>

  3. Read timeout: Set io.timeout.millis to at least 90000 (90 seconds) on the SSE target endpoint. The agent receives heartbeats every 60 seconds β€” if the timeout is shorter than the heartbeat interval, the connection will be closed and re-established repeatedly:

    <HTTPTargetConnection>
  <Properties>
    <Property name="io.timeout.millis">90000</Property>
  </Properties>
  <URL>https://artemis.getmontecarlo.com</URL>
</HTTPTargetConnection>

Agent configuration: Set backendServiceUrl to your Apigee proxy endpoint instead of the Monte Carlo Agent Service endpoint:

container:
  backendServiceUrl: "https://your-apigee-proxy.example.com"

Or for Docker Compose:

environment:
  - BACKEND_SERVICE_URL=https://your-apigee-proxy.example.com

Updated about 2 hours ago