feat: added lineage item and lineage entities #90
Conversation
Signed-off-by: Allison Suarez Miranda <asuarezmiranda@lyft.com>
allisonsuarez
requested review from
bolkedebruin,
dikshathakur3119,
feng-tao,
jinhyukchang,
mgorsk1 and
verdan
as code owners
Signed-off-by: Allison Suarez Miranda <asuarezmiranda@lyft.com>
|
|
||
|
|
||
| @attr.s(auto_attribs=True, kw_only=True) | ||
| class LineageItem: |
There was a problem hiding this comment.
could you share some examples on this class? e.g what are the badges and usage using here?
There was a problem hiding this comment.
Just added the actual Badge class for badge list. Not sure what you mean by what the usage is using here?
There was a problem hiding this comment.
we can change usage to be even more general and make it something like sorting_metric or something like that, but usage seems like a natural way in which we would want to highlight more important upstream/downstream resources when you have lots of them
There was a problem hiding this comment.
My question is around how the metadata lineage entity connection will look like with badge and usage.
Signed-off-by: Allison Suarez Miranda <asuarezmiranda@lyft.com>
Signed-off-by: Allison Suarez Miranda <asuarezmiranda@lyft.com>
Signed-off-by: Allison Suarez Miranda <asuarezmiranda@lyft.com>
| @attr.s(auto_attribs=True, kw_only=True) | ||
| class Lineage: | ||
| key: str # current table/col/task key | ||
| direction: str # upstream/downstream/both |
There was a problem hiding this comment.
any reason we need direction and depth if we have lineage_entities_upstream and lineage_entities_downstream?
There was a problem hiding this comment.
LineageItem seems to include level already?
There was a problem hiding this comment.
Just to persist the selection (upstream, downstream, both) send over in the request. Similarly depth might be needed because it tells you how many levels were requested for this response.
Signed-off-by: Allison Suarez Miranda asuarezmiranda@lyft.com
Summary of Changes
Added LineageItem and Lineage to represent lineage for a given resource (table/column/task).
CheckList
Make sure you have checked all steps below to ensure a timely review.