SQLLineage: SQL Lineage Analysis Tool Powered by Python

Never get the hang of a SQL parser? SQLLineage comes to the rescue. Given a SQL command, SQLLineage will tell you its source and target tables, without worrying about Tokens, Keyword, Identified and all the jagons used by a SQL parser.

Behind the scene, SQLLineage pluggable leverages parser library sqlfluff and sqlparse to parse the SQL command, analyze the AST, stores the lineage information in a graph (using graph library networkx), and bring you all the human-readable result with ease.

First steps

Getting Started: Install SQLLineage and quick use the handy built-in command-line tool
Advanced Usage: Some advanced usage like multi statement SQL lineage and lineage visualization
Beyond Command Line: Using SQLLineage in your Python script

Gear Up

Configuration: Learn how to configure sqllineage
MetaData: Learn how to use MetaDataProvider

Behind the scene

Why SQLLineage: The motivation of writing SQLLineage
How Does SQLLineage Work: The inner mechanism of SQLLineage
DOs and DONTs: Design principles for SQLLineage
Column-Level Lineage Design: Design docs for column lineage
Dialect-Awareness Lineage Design: Design docs for dialect-awareness lineage

Basic concepts

Runner: LineageRunner: The entry point for SQLLineage
Analyzer: LineageAnalyzer: The core functionality of analyze one SQL statement
Holder: LineageHolder: To hold lineage result at different level
Model: The data classes for SQLLineage
MetaDataProvider: MetaDataProvider: provider metadata to assist lineage analysis

Release note

Changelog: See what’s new for each SQLLineage version