Cloudera Enterprise 6.0 Beta | Other versions

Lineage Diagram Icons

Entity Types and Icons Reference

In lineage diagrams, entity types are represented by icons that vary depending upon the source system. The table below lists source system and shows the icons that can display in lineage diagrams.

Lineage diagrams are limited to 400 entities. After 400 entities, Cloudera Navigator lineage diagrams use the hidden icon to provide an entry point for exploring additional entities.

Cluster
	Cluster group
	Cluster instance
HDFS
	File
	Directory
Hive and Impala	Hive entities include tables that result from Impala queries and Sqoop jobs.
	Table
	Field
	Operation, sub-operation, execution
	Impala operation, sub-operation, execution
MapReduce and YARN
	MapReduce operation and operation execution
	YARN operation and operation execution
Oozie
	Operation, operation execution
Pig
	Table
	Pig field
	Pig operation, operation execution
Spark	Supported in CDH 5.11 (and higher). Spark lineage is rendered only for data that is read/written or processed using the Dataframe and SparkSQL APIs. Lineage is not available for data that is read/written or processed using Spark RDD APIs. Metadata extraction for Spark can be enabled or disabled. See Enabling and Disabling Metadata Extraction for details.
	Operation, operation execution. (Spark RDDs and aggregation operations are not included in the diagrams.)
Sqoop
	Operation, sub-operation, execution
S3
	Directory
	File
	S3 Bucket

Other Icons and Visual Elements in Lineage Diagrams

	Hidden entities. Used in lineage diagrams containing more than 400 entities. Click this icon to display the hidden entity details. See Exploring Hidden Entities in a Lineage Diagram for more information.
	Placeholder for an entity that has not yet been extracted. This icon is replaced by the correct entity icon after Cloudera Navigator extracts and links the entity. Hive entities deleted from the system before extraction completes also use this icon, in which case, the icon remains in the lineage diagram.
	Plus icon. Click this icon in a lineage diagram to see more information about the entity.

See Metadata Extraction and Indexing for more information the extraction process, including length of time to extract newly created entities.

Types of Relations Shown in Lineage Diagrams

Lineage diagrams render relationships between entities using different line styles and colors. Arrows indicate data flow direction. Cloudera Navigator supports the following types of relations:

Relation Type	Description
Data flow	A relation between data and a processing activity, such as between a file and a MapReduce job or vice versa.
Parent-child	A parent-child relation. For example, the relation between a directory (parent) and a file (child).
Logical-physical	The relation between a logical entity and its physical entity. For example, between a Hive query and a MapReduce job.
Instance of	The relation between an instance and its template. For example, an operation execution is an instance of operation. Instance of relations are not rendered in lineage diagrams, but you can navigate between template and instance lineage diagrams. See Displaying an Instance Lineage Diagram and Displaying the Template Lineage Diagram for an Instance Lineage Diagram.
Control flow	A relation in which a source entity controls the data flow of the target entity. For example, the relation between the columns of an `insert` clause and the `where` clause of a Hive query.

Lineage diagrams contain the line types shown in the table:

	A solid line depicts a data flow relationship, indicating that the columns appear (possibly transformed) in the output (when line has directional arrow) and logical-physical (when line does not have an arrow). For example, a solid line appears between the columns used in a `select` clause.
	A dashed line depicts a control flow relationship, indicating that the columns determine which rows flow to the output. For example, a dashed line appears between the columns used in an `insert` or `select` clause and the `where` clause of a Hive query. Control flow lines are hidden by default. See Filtering Lineage Diagrams.
	A blue line depicts a link within the lineage diagram that has been selected.
	A green line depicts a summary link that contains operations. When clicked, the green line turns blue denoting it has been selected, and the nested operations display in a selected link summary.

Selected Link Summary

Page generated March 7, 2018.