Kafka Clusters

Monte Carlo relies on a REST Proxy to integrate with your Kafka Cluster.

Self-Hosted Kafka

If you have self hosted Kafka, please setup a Rest Proxy for your Cluster.
Popular Proxies includes:

• Confluent Rest Proxy
• Kapa Rest Proxy (open sourced by Aiven)
• Or any other ones you'd like to choose.

Authentication: Monte Carlo supports Bearer or Basic API tokens over https.

Authorization/Access Scope: Monte Carlo needs access to the list topics API only.

Please let your Monte Carlo representative know, or contact [email protected] for supporting other types of authentication methods.

Kafka Connect Options

Self-Hosted Kafka Connect Clusters

Kafka Connect is intended to be run as a service, it supports a Rest API endpoint for interacting with the Connect Cluster.

Authentication: Monte Carlo supports Bearer or Basic API tokens over https.

Authorization/Access Scope: Monte Carlo needs access to the list connectors API only.

Create Self-Hosted Integration in Monte Carlo

Please get the API token and the token type ready for rest Access.

1. Navigate to the integration settings and the Streaming Connections section.

   

2. Connect a Self-Hosted Kafka Cluster:

   
   • Enter a Streaming System Name: this is a container/grouping for multiple clusters.
   • Specify the Cluster Name (e.g., MyKafkaCluster).
   • Specify the Cluster ID that uniquely identify the cluster at your environment.
   • Specify the rest URL.
   • Choose the auth token type, and specify the API token.
   • Save the integration details.Test the integration to ensure successful connectivity.

3. Connect a Self-Hosted Kafka Connect Cluster:

   • Enter the Streaming System Name, that also contains the Kafka Cluster(s) this connect cluster associate with.
   • Specify the Cluster Name (e.g., MyKafkaCluster).
   • Specify the Cluster ID that uniquely identify the cluster at your environment.
   • Specify the connect Rest Endpoint.
   • Choose the auth token type, and specify the API token.
   • Save the integration details.
   • Test the integration to ensure successful connectivity.

Troubleshooting

If you encounter any issues during the integration process, refer to the following troubleshooting tips:

• Check the access permissions for the provided API Token.
• Ensure that the Clusters are reachable from Monte Carlo.

For additional assistance, contact Monte Carlo support.