Adding column level lineage

[ismailsimsek]

There is a project which is extracting column level lineage dbt-column-lineage-extractor

Using this prject + customized index.html page it is possible to generate dbt docs with column level lineage feature

high level steps to achieve it (Draft/Proposal)

• Use dbt-column-lineage-extractor and generate cmetadata.json which contains column lineage info.
  • To make it performant, this could be generated inside a sub folder and one .json file per table as well. /clineage/schema.table.json
• Use customized index.html and display column lineage (Enable using custom documentation index.html file per dbt project #15 enabled this option). currently its possible to have customized dbt-docs file with opendbt