abstract |
Described herein, according to various embodiments, are for use with data integration or other computing environments utilizing machine learning (ML, DataFlow Machine Learning, DFML), for managing data flows (dataflow, DF) and for building complex Data flow software application (data flow application, pipeline) system (data artificial intelligence system, data AI system). According to an embodiment, the system may provide data governance functions for each slice of data related to a particular snapshot in time, such as, for example, provenance (where a particular data came from), lineage (how the data was acquired/processed), security (who is responsible for the data) , classification (what the data is about), impact (how much impact the data has on the business), retention (how long should the data live), and validity (whether the data should be excluded/included for analysis/processing); this data can then be governed Capabilities are used to make lifecycle decisions and dataflow recommendations. |