Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KED-1570] Calculate layers order and return it to the frontend #148

Merged
merged 23 commits into from
Apr 23, 2020

Conversation

limdauto
Copy link
Collaborator

Description

This PR introduces logic to:

  • Calculate layer ordering server-side based on the layer's dependencies.
  • Remove the need to use a separate layer id and layer name. Instead, use layer name every where.
  • Specify layer: null for every Kedro node in the mock data.
  • Specify layer: null for some dataset.

Development notes

QA notes

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes

Legal notice

  • I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":

  • I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.

  • I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.

  • I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.

@@ -209,46 +306,56 @@ def pretty_name(name):
for node in sorted(pipeline.nodes, key=lambda n: n.name):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't make a deep investigation of the algorithm however I understand that we use toposort. So, having that in mind I have two questions:

  • pipeline.nodes returns toposorted list of nodes. Why do we need to sort them by name?
  • can toposorted list of nodes simplify somehow layers ordering algorithm?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Not sure. It was there before. I thought of removing this sort by name but that breaks a bunch of tests so didn't change it to reduce the scope of this PR.
  • That's a good question. Unfortunately, we can't rely on toposorted list of nodes alone :( There is an example here if you are interested: [KED-1570] Experiment with layer ordering and assignment client-side #147 (comment) -- the example suggests adding transition nodes and toposort again, but adding transition nodes still require knowing layers' dependencies, so there is no avoiding toposorting layers.

package/kedro_viz/server.py Outdated Show resolved Hide resolved
package/kedro_viz/server.py Outdated Show resolved Hide resolved
package/kedro_viz/server.py Outdated Show resolved Hide resolved
package/kedro_viz/server.py Outdated Show resolved Hide resolved
@limdauto limdauto changed the base branch from feature/layer-attribute-in-response to develop April 21, 2020 17:15
src/selectors/layers.test.js Show resolved Hide resolved
src/store/normalize-data.js Outdated Show resolved Hide resolved
src/selectors/layers.js Outdated Show resolved Hide resolved
@limdauto
Copy link
Collaborator Author

@richardwestenra Thank you for commenting. I have made the requested changes.

Copy link
Contributor

@richardwestenra richardwestenra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, I think that's the option I would've chosen too. Thanks! And sorry for the delay in reviewing.

@limdauto limdauto merged commit cc8e6e2 into develop Apr 23, 2020
@limdauto limdauto deleted the KED-1570/layers-ordering branch April 23, 2020 10:50
@richardwestenra richardwestenra mentioned this pull request May 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants