Self-Hosted Kafka and other Platforms (Private Beta)

Kafka Clusters

Monte Carlo relies on a REST Proxy to integrate with your Kafka Cluster.

Self-Hosted Kafka or MSK

If you have self hosted Kafka or using MSK, please setup a Rest Proxy for your Cluster.
Popular Proxies includes:

Authentication: Monte Carlo supports Bearer or Basic API tokens over https.

Authorization/Access Scope: Monte Carlo needs access to the list topics API only.

Please let your Monte Carlo representative know, or contact [email protected] for supporting other types of authentication methods.

Kafka Connect Options

Self-Hosted Kafka Connect Clusters

Kafka Connect is intended to be run as a service, it supports a Rest API endpoint for interacting with the Connect Cluster.

Authentication: Monte Carlo supports Bearer or Basic API tokens over https.

Authorization/Access Scope: Monte Carlo needs access to the list connectors API only.

Other Managed Kafka Connect Clusters

MSK and others are coming soon...

Create Self-Hosted Integration in Monte Carlo

Please get the API token and the token type ready for rest Access.

  1. Navigate to the integration settings and the Streaming Connections section.

    Alt text
  2. Connect a Self-Hosted Kafka Cluster:

    Alt text
    • Enter a Streaming System Name: this is a container/grouping for multiple clusters.
    • Specify the Cluster Name (e.g., MyKafkaCluster).
    • Specify the Cluster ID that uniquely identify the cluster at your environment.
    • Specify the rest URL.
    • Choose the auth token type, and specify the API token.
    • Save the integration details.Test the integration to ensure successful connectivity.
  3. Connect a Self-Hosted Kafka Connect Cluster:

    • Enter the Streaming System Name, that also contains the Kafka Cluster(s) this connect cluster associate with.
    • Specify the Cluster Name (e.g., MyKafkaCluster).
    • Specify the Cluster ID that uniquely identify the cluster at your environment.
    • Specify the connect Rest Endpoint.
    • Choose the auth token type, and specify the API token.
    • Save the integration details.
    • Test the integration to ensure successful connectivity.

Troubleshooting

If you encounter any issues during the integration process, refer to the following troubleshooting tips:

  • Check the access permissions for the provided API Token.
  • Ensure that the Clusters are reachable from Monte Carlo.

For additional assistance, contact Monte Carlo support.