-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[KED-1570] Calculate layers order and return it to the frontend #148
Conversation
2b6efd7
to
f5c9fe6
Compare
@@ -209,46 +306,56 @@ def pretty_name(name): | |||
for node in sorted(pipeline.nodes, key=lambda n: n.name): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't make a deep investigation of the algorithm however I understand that we use toposort
. So, having that in mind I have two questions:
pipeline.nodes
returns toposorted list of nodes. Why do we need to sort them by name?- can toposorted list of nodes simplify somehow layers ordering algorithm?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Not sure. It was there before. I thought of removing this sort by name but that breaks a bunch of tests so didn't change it to reduce the scope of this PR.
- That's a good question. Unfortunately, we can't rely on toposorted list of nodes alone :( There is an example here if you are interested: [KED-1570] Experiment with layer ordering and assignment client-side #147 (comment) -- the example suggests adding transition nodes and toposort again, but adding transition nodes still require knowing layers' dependencies, so there is no avoiding toposorting layers.
@richardwestenra Thank you for commenting. I have made the requested changes. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, I think that's the option I would've chosen too. Thanks! And sorry for the delay in reviewing.
Description
This PR introduces logic to:
null
for every Kedro node in the mock data.null
for some dataset.Development notes
QA notes
Checklist
RELEASE.md
fileLegal notice
I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.