annotating-task-lineage

SKILL.md

Annotate Airflow tasks with data lineage using inlets and outlets.

Supports OpenLineage Dataset objects, Airflow Assets, and Airflow Datasets for defining inputs and outputs across databases, data warehouses, and cloud storage

Use as a fallback when operators lack built-in OpenLineage extractors; follows a four-tier precedence system where custom extractors and OpenLineage methods take priority

Includes dataset naming helpers for Snowflake, BigQuery, S3, and PostgreSQL to ensure consistent namespace and name formatting across platforms

Set lineage statically at DAG parse time or dynamically within the execute() method for non-deferrable operators; implement get_openlineage_facets_on_complete() for custom operators

Lineage annotations are immediately visualized in Astro's enhanced Lineage tab for cross-DAG and cross-deployment visibility

Annotating Task Lineage with Inlets & Outlets

This skill guides you through adding manual lineage annotations to Airflow tasks using inlets and outlets.

Reference: See the OpenLineage provider developer guide for the latest supported operators and patterns.

On Astro

Lineage annotations defined with inlets and outlets are visualized in Astro's enhanced Lineage tab, which provides cross-DAG and cross-deployment lineage views. This means your annotations are immediately visible in the Astro UI, giving you a unified view of data flow across your entire Astro organization.

When to Use This Approach