Changelog

v1.5.2

Date:

April 7, 2024

Enhancement

• Enable support of sqlfluff context (#548)

• Support Change Configure Default Schema At Runtime (#536)

Bugfix

• Parse column level lineage incorrect (#584)

• Metadata Masked When Table was in a previous UPDATE statement (#577)

• Clickhouse SQL ‘GLOBAL IN’ not support (#554)

• Support json_tuple in SELECT clause in Hive (#483)

v1.5.1

Date:

February 4, 2024

This is a bugfix release mostly driven by community contributors. Thanks everyone for making SQLLineage better.

Enhancement

• Allow unambiguous column reference for JOIN with USING clause (#558)

• Make Lateral Column Alias Reference Configurable (#539)

• Add an Argument to Exclude SubQuery Column Node in Column Lineage Path (#526)

Bugfix

• Not fully processed top-level subquery in DML (#564)

• Missing target table with tsql parsing into statements with union (#562)

• The second and subsequent case when subqueries in the select_clause are not correctly recognized (#559)

• SQLLineageConfig boolean value returns True for all non-empty strings (#551)

• Column lineage does not traverse through CTE containing uppercase letters (#531)

v1.5.0

Date:

January 7, 2024

Great thanks to liznzn for contributing on MetaData-awareness lineage. Now we’re able to generate more accurate column lineage result for select * or select unqualified columns in case of table join through a unified MetaDataProvider interface.

Also a breaking change is made to make ansi the default dialect in v1.5.x release as we target ultimately deprecating non-validating dialect in v1.6.x release.

Breaking Change

• Make ansi the Default Dialect (#518)

Feature

• Metadata Provider to Assist Column Lineage Analysis (#477)

Enhancement

• Add a Configuration for Default Schema (#523)

• Silent Mode Option to Suppress UnsupportedStatementException (#513)

• Support Lateral Column Alias Reference Analyzing (#507)

• Skip Lineage Analysis for SparkSQL Function Related Statement (#500)

• update statement column lineage (#487)

Bugfix

• subquery mistake alias as table name in visualization (#512)

• InvalidSyntaxException When SQL Statement Ends with Multiple Semicolons (#502)

• Misidentify Binary Operator * As Wildcard (#485)

• adding type cast operator produces different results for redshift dialect (#455)

v1.4.9

Date:

December 10, 2023

This is a bugfix release where we closed a bunch of issues concerning CTE and UNION

Bugfix

• Not Using Column Name Specified in Query For CTE within Query (#486)

• CTE (Common Table Expressions) within CTE (#484)

• lineage inaccurate when CTE used in subquery (#476)

• UNION ALL Queries resolves column lineage incorrectly (#475)

• Missing table when parsing sql with UNION ALL (#466)

• No target tables in UPDATE statement using CTE (#453)

v1.4.8

Date:

October 16, 2023

Enhancement

• Support Python 3.12 (#469)

• programmatically list supported dialects (#462)

• add versioning of package to cli (#457)

• Add Support of DROP VIEW statements (#456)

• support split SQL statements without semicolon in tsql (#384)

Bugfix

• SqlFluff RuntimeError Triggers Server Error 500 in Frontend (#467)

• ignore lineage for analyze statement (#459)

v1.4.7

Date:

August 27, 2023

Enhancement

• Support subquery in VALUES clause (#432)

• Dialect=’tsql’ should return warning when no semicolons are detected (#422)

• Restricting folder and files user can access from frontend (#405)

• throw exception when the statement missing the semicolon as splitter (#159)

Bugfix

• AttributeError raised using parenthesized where clause (#426)

• qualified wildcard recognized as wrong column name (#423)

v1.4.6

Date:

July 31, 2023

In this release, we finally reach the milestone to make all sqlparse only test cases passed with sqlfluff implementation. That’s a big step in ultimately deprecating sqlparse. Also by upgrading to latest version of sqlfluff (with our PR merged), we enjoy the benefits of improved sqlfluff performance when parsing some SQLs with nested query pattern.

Enhancement

• Improve sqlfluff Performance Issue on Nested Query Pattern (#348)

• Reduce sqlparse only test cases (#347)

Bugfix

• Missing Source Table for MERGE statement when UNION involved in source subquery (#406)

• Column lineage does not work for CAST to Parameterized Data Type (#329)

• Can’t handle parenthesized from clause (#278)

v1.4.5

Date:

July 2, 2023

Enhancement

• Switch to PyPI Trusted Publishers (#389)

• Support tsql Declare Statement (#357)

Bugfix

• Exception for Subquery Expression Without Source Tables (#401)

• Not Supporting Create Table AS in postgres (#400)

• Failed to handle UNION followed by CTE (#398)

• Not handling CTE inside DML query (#377)

• Failed to parse UNION inside CTE (#376)

v1.4.4

Date:

June 11, 2023

Enhancement

• BigQuery Specific MERGE statement feature support (#380)

• Support snowflake create table…clone and alter table…swap (#373)

• Parse Column Lineage When Specify Column Names in Insert/Create Statement (#212)

Bugfix

• Switching Dialect in UI only works When Explicit Clicked (#387)

• No Column Lineage Parsed for DML with SELECT query in parenthesis (#244)

v1.4.3

Date:

May 13, 2023

Enhancement

• Support postgres style type casts “keyword::TIMESTAMP” (#364)

Bugfix

• Missing column lineage from SELECT DISTINCT using non-validating dialect (#356)

• Missing column lineage with Parenthesis around column arithmetic operation (#355)

• Not Handling CTE at the start of query in DML (#328)

v1.4.2

Date:

April 22, 2023

Bugfix

• sqlparse v0.4.4 breaks non-validating dialect (#361)

v1.4.1

Date:

April 2, 2023

Bugfix

• frontend app unable to load dialect when launched for the first time

v1.4.0

Date:

March 31, 2023

Great thanks to Nahuel, Mayur and Pere from OpenMetadata community for contributing on feature Dialect-awareness lineage. Leveraging sqlfluff underneath, we’re now able to give more correct lineage result with user input on SQL dialect.

Feature

• Dialect-awareness lineage (#302)

• support MERGE statement (#166)

Enhancement

• Use curved lines in lineage graph visualization (#320)

• Click to lock highlighted nodes in visualization (#318)

• Deprecate support for Python 3.6 and Python 3.7, add support for Python 3.11 (#319)

• support t-sql assignment operator (#205)

Bugfix

• exception when insert into qualified table followed by parenthesized query (#249)

• missing columns when current_timestamp as reserved keyword used in select clause (#248)

• exception when non-reserved keywords used as column name (#183)

• exception when non-reserved keywords used as table name (#93)

v1.3.7

Date:

Oct 22, 2022

Enhancement

• migrate demo site off Heroku to GitHub Pages (#288)

• remove flask-related dependencies by implementing a wsgi app (#287)

Bugfix

• exception with VALUES clause (#292)

• exception with Presto unnest function (#272)

• exception with snowflake generator statement (#214)

v1.3.6

Date:

Aug 28, 2022

Enhancement

• support MySQL RENAME TABLE statement (#267)

• auto deploy to Heroku with GitHub Actions (#232)

Bugfix

• handling parenthesis around subquery between union (#270)

• unable to extract alias of columns using function with CTAS (#253)

• exception when using lateral view (#225)

v1.3.5

Date:

May 10, 2022

Enhancement

• support parsing column in cast/try_cast with function (#254)

• support parsing WITH for bucketing in Trino (#251)

Bugfix

• incorrect column lineage with nested cast (#240)

• column lineages from boolean expression (#236)

• using JOIN with ON/USING keyword fails to determine source tables when followed by a parenthesis (#233)

• failure to handle multiple lineage path for same column (#228)

v1.3.4

Date:

March 6, 2022

Enhancement

• update black to stable version (#222)

Bugfix

• table/column lineage mixed up for self dependent SQL (#219)

• problem with SELECT CAST(CASE WHEN …END AS DECIMAL(M,N)) AS col_name (#215)

• failed to parse source table from subquery with more than one parenthesis (#213)

v1.3.3

Date:

December 26, 2021

Enhancement

• smarter column-to-table resolution using query context (#203)

Bugfix

• column lineage for union operation (#207)

• subquery in where clause not parsed for table lineage (#204)

v1.3.2

Date:

December 12, 2021

Enhancement

• support optional AS keyword in CTE (#198)

• support referring to a CTE in subsequent CTEs (#196)

• support for Redshift ‘copy from’ syntax (#164)

v1.3.1

Date:

December 5, 2021

Enhancement

• test against Python 3.10 (#186)

Bugfix

• alias parsed as table name for column lineage using ANSI-89 Join (#190)

• CTE parsed as source table when referencing column from cte using alias (#189)

• window function with parameter parsed as two columns (#184)

v1.3.0

Date:

November 13, 2021

Feature

• Column-Level Lineage (#103)

Bugfix

• SHOW CREATE TABLE parsed as target table (#167)

v1.2.4

Date:

June 14, 2021

Enhancement

• highlight selected node and its ancestors as well as children recursively (#156)

• add support for database.schema.table as identifier name (#153)

• add support for swap_partitions_between_tables (#152)

v1.2.3

Date:

May 15, 2021

Enhancement

• lineage API response exception handling (#148)

v1.2.2

Date:

May 5, 2021

Bugfix

• resize dragger remain on the UI when drawer is closed (#145)

v1.2.1

Date:

May 3, 2021

Enhancement

• option to specify hostname (#142)

• re-sizable directory tree drawer (#140)

• async loading for directory tree in frontend UI (#138)

v1.2.0

Date:

April 18, 2021

Feature

• A Full Fledged Frontend Visualization App (#118)

• Use TPC-DS Queries as Visualization Example (#116)

Enhancement

• Unit Test Failure With sqlparse==0.3.0, update dependency to be >=0.3.1 (#117)

• contributing guide (#14)

v1.1.4

Date:

March 9, 2021

Bugfix

• trim function with from in arguments (#127)

v1.1.3

Date:

February 1, 2021

Bugfix

• UNCACHE TABLE statement parsed with target table (#123)

v1.1.2

Date:

January 26, 2021

Bugfix

• Bring back draw method of LineageRunner to avoid backward incompatible change (#120)

v1.1.1

Date:

January 24, 2021

Bugfix

• SQLLineageException for Multiple CTE Subclauses (#115)

v1.1.0

Date:

January 17, 2021

Feature

• A new JavaScript-based approach for visualization, drop dependency for graphviz (#94)

Enhancement

• Test against Mac OS and Windows (#87)

Bugfix

• buckets parsed as table name for Spark bucket table DDL (#111)

• incorrect result for update statement (#105)

v1.0.2

Date:

November 17, 2020

Enhancement

• black check in CI (#99)

• switch to GitHub Actions for CI (#95)

• test against Python 3.9 (#84)

Bugfix

• cartesian product exception with ANSI-89 syntax (#89)

v1.0.1

Date:

October 17, 2020

Enhancement

• remove upper bound for dependencies (#85)

v1.0.0

Date:

September 27, 2020

New Features

• a detailed documentation hosted by readthedocs (#81)

Enhancement

• drop support for Python 3.5 (#79)

v0.4.0

Date:

August 29, 2020

New Features

• DAG based lineage representation with visualization functionality (#55)

Enhancement

• replace print to stderr with logging (#75)

• sort by table name in LineageResult (#70)

• change schema default value from <unknown> to <default> (#69)

• set up Github actions for PyPi publish (#68)

v0.3.0

Date:

July 19, 2020

New Features

• statement granularity lineage result (#32)

• schema aware parsing (#20)

Enhancement

• allow user to specify combiner (#64)

• trim leading comment for statement in verbose output (#57)

• add mypy as static type checker (#50)

• add bandit as security issue checker (#48)

• enforce black as code formatter (#46)

• dedicated Table/Partition/Column Class (#31)

• friendly exception handling (#30)

Bugfix

• subquery without alias raises exception (#62)

• refresh table and cache table should not count as target table (#59)

• let user choose whether to filter temp table or not (#23)

v0.2.0

Date:

April 11, 2020

Enhancement

• test against Python 3.8 (#39)

Bugfix

• comment in line raise AssertionError (#37)

• white space in left join (#36)

• temp table checking (#35)

• enable case-sensitive parsing (#34)

• support for create table like statement (#29)

• special treatment for DDL (#28)

• empty statement return (#25)

• drop table parsed as target table (#21)

• multi-line sql causes AssertionError (#18)

• subquery mistake alias as table name (#16)

v0.1.0

Date:

July 26, 2019

New Features

• stable command line interface (#2)

Enhancement

• combine setup.py and requirements.txt (#6)

• combine tox and Travis CI (#5)

• table-wise lineage with sufficient test cases (#4)

• a startup docs for sqllineage’s usage (#3)

• pypi badges in README (#1)

v0.0.1

Date:

June 16, 2019

New Features

initial public release