Monte Carlo uses a data collector to connect to data warehouses, data lakes and BI tools in order to extract metadata, logs and statistics. The data collector is deployed in Monte Carlo's secure AWS environment across our Fortune 500 clients to ensure a seamless deployment process and best-in-class customer support. This section outlines the architecture and key components of the data collector.
The data collector architecture is optimized as seen below to ensure maximum security as we interact with your data.
You are all set!
Monte Carlo is a SaaS platform. We will take on deployment and management of the instance.
- A VPC along with public/private subnets and other networking components. This VPC contains all components of the data collector. All outbound communication is routed through a single Elastic IP.
- A lambda function to handle API calls.
- A lambda function that executes distributed collection jobs. It connects to data warehouses, data lakes and BI tools to collect metadata, logs and metrics. This information is streamed back to Monte Carlo’s cloud via a secure Kinesis stream (using HTTPS calls).
- An S3 bucket that contains configuration and any other data required during processing.
- SQS queues to handle events in data lake environments.
Updated 3 months ago