Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Handle cycles in workflows when partially executing them #11187

Merged

Conversation

despairblue
Copy link
Contributor

Summary

This implement cycle handling according to the specification for the new partial execution flow.
Basically if a start node is inside a cycle the start of the cycle will become the start node.

This is to make it more predictable instead of figuring out if the loop has finished executing or not and thus deciding if the run data should be kept or invalidated. We just start cycles/loops always from the start.

You can review commit by commit.

Additionally this contains:

  • stop converting start nodes to and from arrays unnecessarily
  • import helpers from index file instead of directly

Related Linear tickets, Github issues, and Community forum posts

https://linear.app/n8n/issue/PAY-1757/phase-2-cycle-detection-and-handling

Review / Merge checklist

  • PR title and summary are descriptive. (conventions)
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Oct 9, 2024
@despairblue despairblue force-pushed the pay-1757-phase-2-cycle-detection-and-handling-try-2 branch 2 times, most recently from af5b8b1 to cae9693 Compare October 10, 2024 08:03
@despairblue despairblue marked this pull request as ready for review October 10, 2024 08:08
@despairblue despairblue force-pushed the pay-1757-phase-2-cycle-detection-and-handling-try-2 branch from df0e688 to 2ad28a2 Compare October 10, 2024 13:29
tomi
tomi previously approved these changes Oct 18, 2024
Copy link
Contributor

@tomi tomi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and easy to read code as usual 👌 Couple small suggestions about the code style, but nothing that would block this from being merged 🚀

packages/core/src/PartialExecutionUtils/DirectedGraph.ts Outdated Show resolved Hide resolved
packages/core/src/PartialExecutionUtils/DirectedGraph.ts Outdated Show resolved Hide resolved
packages/core/src/PartialExecutionUtils/DirectedGraph.ts Outdated Show resolved Hide resolved
Comment on lines 121 to 123
expect(stronglyConnectedComponents).toHaveLength(2);
expect(stronglyConnectedComponents).toContainEqual([node4]);
expect(stronglyConnectedComponents).toContainEqual([node3, node2, node1]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we care about the order in which the connected components are returned? Should we instead check

Suggested change
expect(stronglyConnectedComponents).toHaveLength(2);
expect(stronglyConnectedComponents).toContainEqual([node4]);
expect(stronglyConnectedComponents).toContainEqual([node3, node2, node1]);
expect(stronglyConnectedComponents).toEqual([
[node4],
[node3, node2, node1]
])

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we care about the order. It should be stable but it does not convey any information as far as I know, and as far as the use case is concerned.
So it's ok to leave it like this so that we don't have to touch the tests if we change the algorithm.

packages/core/src/PartialExecutionUtils/handleCycles.ts Outdated Show resolved Hide resolved
packages/core/src/PartialExecutionUtils/DirectedGraph.ts Outdated Show resolved Hide resolved
Copy link
Contributor

✅ All Cypress E2E specs passed

Copy link

cypress bot commented Oct 18, 2024

n8n    Run #7434

Run Properties:  status check passed Passed #7434  •  git commit 321d6deef1: 🌳 master 🖥️ browsers:node18.12.0-chrome107 🤖 PR User 🗃️ e2e/*
Project n8n
Run status status check passed Passed #7434
Run duration 05m 13s
Commit git commit 321d6deef1: 🌳 master 🖥️ browsers:node18.12.0-chrome107 🤖 PR User 🗃️ e2e/*
Committer Danny Martini
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 1
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 453

Copy link
Contributor

✅ All Cypress E2E specs passed

@despairblue despairblue merged commit 321d6de into master Oct 18, 2024
34 checks passed
@despairblue despairblue deleted the pay-1757-phase-2-cycle-detection-and-handling-try-2 branch October 18, 2024 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants