Recommended monitoring strategies for tables
In Data Observability, monitoring a single table is not recommended, as it doesn't provide visibility into upstream issues that could affect the table's quality. To ensure comprehensive monitoring coverage, we recommend considering the following strategies:
- Enable the Data Product for Monitoring: If the table is part of an existing Data Product that isn't monitored, enable monitoring for the entire Data Product. This approach offers a broader view of data health within the product. If the table belongs to multiple Data Products, prioritize enabling the one with the fewest tables for focused monitoring. See existing Data Products
- Create a Data Product: If the table is a key output in a data pipeline and doesn't have downstream dependencies (other than BI dashboards), create a Data Product to monitor it. This allows you to easily monitor, share, and document the table's health. Create a Data Product
- Create a Data Product from a Downstream Table/BI Report: If the table is part of a pipeline, create a Data Product for the entire pipeline. This strategy ensures monitoring for the table and other critical tables connected to it, providing a holistic view of the pipeline's health. Create a Data Product
- Monitor Schema: If the table and its upstream dependencies account for a significant number of the unmonitored tables in its schema, enable monitoring for the entire schema. This approach simplifies monitoring by covering most of the schema's tables. Learn about Configuring Monitoring
- Monitor by Tag: If the table has a tag shared with several tables, enable monitoring for all tables with that tag. This strategy is useful when you want to monitor a group of tables with a common characteristic. Learn about Configuring Monitoring
By implementing these recommendations, you can enhance your monitoring coverage and gain better insights into the health and quality of your data pipelines and tables.
Updated 6 months ago