We currently do not have data on fields from 'read_only' queries and BI lineage.
We currently suggest to delete fields that :
- Are not in fields lineage which means they are absent from insert/copy/create queries.
- The table is not in cleanup_suggestions
- The table has no read_only queries
- The table has no BI queries
FULL_TABLE_ID : A concatenation of project, dataset and table name
PROJECT_NAME : As defined externally
DATASET_NAME : As defined externally(e.g. a schema in snowflake)
TABLE_NAME: As defined externally
TABLE_TYPE : Field values: table, view, external, wildcard_table
FIELD_TO_CLEANUP : A list of fields that can be removed
FIELD_TO_KEEP : A list of fields that should be kept
PERCENT_REMOVABLE : The % of fields in the table that can be deleted, the insight is sorted by this.
TABLE_IMPORTANCE_SCORE : The table importance score, the insight is sorted by this
Updated about 2 months ago