Available Metrics
The following metrics are supported by Field Metrics Monitors.
Uniqueness
Uniqueness metrics check for duplicates in unique keys like UUIDs, and for changes in cardinality.
| Metric | Description | Column Types | Threshold Types |
|---|---|---|---|
| Unique (%) | Percentage of unique values across all rows | All | Manual & ML |
| Unique (count) | Count of unique values | All | Manual |
Completeness
Completeness metrics check for ways that data can be null or otherwise unpopulated.
| Metric | Description | Column Types | Threshold Types |
|---|---|---|---|
| Null (%) | Percentage of rows where value is null | All | Manual & ML |
| Null (count) | Count of rows with null value | All | Manual |
| Non-null (count) | Count of rows with non-null value | All | Manual |
| Empty string (%) | Percentage of rows where the value is an empty string | String | Manual & ML |
| Empty string (count) | Count of rows where the value is an empty string | String | Manual |
| All spaces (%) | Percentage of rows where the text value is whitespace only | String | Manual & ML |
| All spaces (count) | Count of rows where the text value is whitespace only | String | Manual |
| NaN (%) | Percentage of rows with a value of NaN (Not a Number), meaning the value is undefined | Numeric | Manual & ML |
| NaN (count) | Count of rows with a value of NaN (Not a Number), meaning the value is undefined | Numeric | Manual |
| "none" or "null" (%) | Percentage of rows where the text value is a null keyword ("none", "null", "nil", "nothing" or "n/a") | String | Manual & ML |
| "none" or "null" (count) | Count of rows where the text value is a null keyword ("none", "null", "nil", "nothing" or "n/a") | String | Manual |
Distribution
Distribution metrics check for shifts in the numeric profile of data.
| Metric | Description | Column Types | Threshold Types |
|---|---|---|---|
| Mean | Average value across all rows | Numeric | Manual & ML |
| Median | Median value across all rows | Numeric | Manual & ML |
| Min | Minimum value across all rows | Numeric | Manual & ML |
| Max | Maximum value across all rows | Numeric | Manual & ML |
| 20th percentile | 20th percentile of values | Numeric | Manual & ML |
| 40th percentile | 40th percentile of values | Numeric | Manual & ML |
| 60th percentile | 60th percentile of values | Numeric | Manual & ML |
| 80th percentile | 80th percentile of values | Numeric | Manual & ML |
| Zero (%) | Percentage rows with value equal to zero | Numeric | Manual & ML |
| Zero (count) | Count of rows with value equal to zero | Numeric | Manual |
| Negative (%) | Percentage of rows that have a negative value | Numeric | Manual & ML |
| Negative (count) | Count of rows that have a negative value | Numeric | Manual |
| Standard deviation | Standard deviation of values | Numeric | Manual & ML |
| Sum | Sum of values across all rows | Numeric | Manual |
| True (%) | Percentage of rows where the value is true | Boolean | Manual & ML |
| True (count) | Count of rows where the value is true | Boolean | Manual |
| False (%) | Percentage of rows where the value is false | Boolean | Manual & ML |
| False (count) | Count of rows where the value is false | Boolean | Manual |
Validity
Validity metrics check that values are honoring expected and usable formats, including common data entry errors.
| Metric | Description | Column Types | Threshold Types |
|---|---|---|---|
| String length max | Maximum character length | String | Manual |
| String length min | Minimum character length | String | Manual |
| String length mean | Average character length | String | Manual |
| String length standard deviation | Standard deviation of character length | String | Manual |
| Integer (%) | Percentage of rows where the text value is an integer | String | Manual & ML |
| Integer (count) | Count of rows where the text value is an integer | String | Manual |
| Float (%) | Percentage of rows where the text value is a floating-point number | String | Manual & ML |
| Not float (count) | Count of rows where the text value is not a floating-point number | String | Manual |
| UUID (%) | Percentage of rows where the text value is a UUID (e.g., b391e7d2-80e0-4749-8c60-c76031c43dfe). The comparison is case insensitive. | String | Manual & ML |
| Not UUID (count) | Count of rows where the text value is not a UUID (e.g., b391e7d2-80e0-4749-8c60-c76031c43dfe). The comparison is case insensitive. | String | Manual |
| SSN (%) | Percentage of rows where the text value is formatted as Social Security Number from the United States (e.g., 123-45-6789) | String | Manual & ML |
| Not SSN (count) | Count of rows where the text value is not formatted as Social Security Number from the United States (e.g., 123-45-6789) | String | Manual |
| USA phone number (%) | Percentage of rows where the text value is a USA phone number (e.g., (123) 456-7890, +1 (123) 456-7890 etc) | String | Manual & ML |
| Not USA phone number (count) | Count of rows where the text value is not a USA phone number (e.g., (123) 456-7890, +1 (123) 456-7890 etc) | String | Manual |
| USA state code (%) | Percentage of rows where the text value is a USA state code (e.g., TX) | String | Manual & ML |
| Not USA state code (count) | Count of rows where the text value is not a USA state code (e.g., TX) | String | Manual |
| USA ZIP code (%) | Percentage of rows where the text value is a USA ZIP code (e.g., 94109) | String | Manual & ML |
| Not USA ZIP code (count) | Count of rows where the text value is not a USA ZIP code (e.g., 94109) | String | Manual |
| Email (%) | Percentage of rows where the text value is an email address (e.g., [email protected]) | String | Manual & ML |
| Not email (Count) | Count of rows where the text value is not an email address (e.g., [email protected]) | String | Manual |
| Timestamp (%) | Percentage of rows where the text value is an ISO-8601 format date or timestamp (e.g., 2023-01-12 16:50:11.045746 +00:00, 2023-01-12T16:50:11Z) | String | Manual & ML |
| Not timestamp (count) | Count of rows where the text value is not an ISO-8601 format date or timestamp (e.g., 2023-01-12 16:50:11.045746 +00:00, 2023-01-12T16:50:11Z) | String | Manual |
| In past (%) | Percentage of rows where the value is a date or time occurring before the time when the metric is evaluated, with a granularity of seconds used for comparison | Date | Manual & ML |
| In past (count) | Count of rows where the value is a date or time occurring before the time when the metric is evaluated, with a granularity of seconds used for comparison | Date | Manual |
| In future (%) | Percentage of rows where the value is a date or time occurring after the time when the metric is evaluated, with a granularity of seconds used for comparison | Date | Manual & ML |
| In future (count) | Count of rows where the value is a date or time occurring after the time when the metric is evaluated, with a granularity of seconds used for comparison | Date | Manual |
| Unix time 0 (%) | Percentage of rows where the value is Unix time 0 | Date | Manual & ML |
| Unix time 0 (count) | Count of rows where the value is Unix time 0 | Date | Manual |
Updated about 1 month ago