Detecting potential root causes of data quality issues using data lineage graphs

Abstract

An example system includes a processor that can generate a first lineage graph based on a first set of monitored assets and processes used to produce a data asset. The processor can detect a data quality issue at the data asset. The processor can also generate a second lineage graph including a second set of monitored assets and processes that produced the data asset with the data quality issue. The processor can further compare the second lineage graph with the first lineage graph to detect a potential root cause of the data quality issue. The processor can also further modify an asset or process corresponding to the potential root cause of the data quality issue.

What is claimed is: 
     
         1 . A computer system, comprising:
 a hardware processor, one or more computer-readable storage media, and program instructions stored on one or more of the computer-readable storage media executable by the processor to cause the computer system to perform a method comprising:   detecting, by the processor, a data quality issue at the data asset;   generating, by the processor, a first lineage graph for the dataset at which the data quality issue was detected;   generating, by the processor, a second lineage graph for a dataset at which no data quality issue was detected, wherein the dataset at which no data quality issue was detected corresponds to the dataset at which the data quality issue was detected;   comparing, by the processor, the second lineage graph with the first lineage graph to detect a potential root cause of the data quality issue; and   modifying, by the processor, a process in the first lineage graph, based on the detected potential root cause of the data quality issue.