📘

Kafka Beta Pricing

Monte Carlo Consumption pricing for Kafka is based on Topics. Kafka usage will remain free of charge during the beta, until at least the end of April 2024.

What is Kafka?

Apache Kafka is a distributed event store and stream-processing platform. It provides high throughput, high scalability and low latency for real-time pipelines. Around Kafka, there are different streaming processing frameworks like Kafka Connect, Flink and KsqlDB etc. Kafka users can setup their own Kafka clusters or use providers like Confluent, Aiven, instaclustr and other platforms.

Why connect Kafka to Monte Carlo?

Monte Carlo tracks your Topics and Kafka Connectors to create cross storage lineage. Monte Carlo links tables in your operational database to your warehouse tables through kafka topics. This helps with faster incident resolution and catching issues more upstream in the whole workflow.

Connected Resource Types

The following Kafka and Connect clusters are supported.

Kafka Cluster

• Confluent Cloud
• Other Self-Hosted or Cloud Cluster Providers
  • Confluent Platform (self-hosted)
  • AWS MSK
  • Other Cluster Providers (Aiven, self-hosted etc)

Kafka Connect Cluster

• Confluent Cloud
• Other Self-Hosted or Cloud Providers (Aiven, self-hosted etc)
• AWS MSK

Reach out to your Monte Carlo representative or [email protected] if you are interested in support for Redpanda, Azure EventHubs, or other platforms.

Supported Source and Sink Connectors

Our beta integration creates lineage for:

Sources: Postgres, MySQL, MongoDB, Debezium CDC (for Postgres, MySQL and MongoDB)

Sinks: Snowflake, Redshift, BigQuery

We are adding more -- please reach out if your sources and sinks are not currently supported!