Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert rust example to node #494

Merged
merged 6 commits into from
May 2, 2024

Conversation

Michael-J-Ward
Copy link
Contributor

Ref #474

1st commit is a minimalist conversion, tried to keep the behavior the same.

2nd commit removes some extra dependencies.

Note

The end behavior does appear different. On main, I will get ~10 to ~30 messages about dropped inputs, and I will only get 100 - num_dropped for operator received random value

2024-04-30T14:21:36.529492Z DEBUG dora_daemon::node_communication: dropped 1 inputs because event queue was too full
    at binaries/daemon/src/node_communication/mod.rs:328

With this conversion to the node api, i get no messages about dropped inputs and 100 operated received random value.

Ran with:

RUST_LOG=debug cargo run --example rust-dataflow 

@Michael-J-Ward Michael-J-Ward marked this pull request as ready for review April 30, 2024 14:26
@Michael-J-Ward
Copy link
Contributor Author

NOTE: I do not think that last commit fixed the CI failure for 5f87384.

I suspected the failure was spurious, so I added the last commit to trigger a second run.

Copy link
Collaborator

@phil-opp phil-opp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

NOTE: I do not think that last commit fixed the CI failure for 5f87384.

I suspected the failure was spurious, so I added the last commit to trigger a second run.

Yes, that failure seems spurious. I restarted the failed job and it worked on the second try.

examples/rust-dataflow/operator/src/main.rs Outdated Show resolved Hide resolved
examples/rust-dataflow/dataflow.yml Outdated Show resolved Hide resolved
Michael-J-Ward and others added 2 commits May 1, 2024 13:01
the Q was answered in review. stopping is unnecessary because the `event_stream` wills top shortly

Co-authored-by: Philipp Oppermann <dev@phil-opp.com>
@phil-opp
Copy link
Collaborator

phil-opp commented May 2, 2024

Thanks!

@phil-opp phil-opp merged commit 5518b58 into dora-rs:main May 2, 2024
17 checks passed
@haixuanTao haixuanTao mentioned this pull request May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants