Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEATURE] Migrate to plugin mode #340

Closed
3 of 4 tasks
goodwanghan opened this issue Jul 16, 2022 · 1 comment · Fixed by #339 or #375
Closed
3 of 4 tasks

[FEATURE] Migrate to plugin mode #340

goodwanghan opened this issue Jul 16, 2022 · 1 comment · Fixed by #339 or #375
Labels
behavior change refactoring
Milestone
0.8.0

Comments

@goodwanghan
Copy link
Collaborator

goodwanghan commented Jul 16, 2022 •
edited
Loading

Triad already has the standard solution for plugin mode: fugue-project/triad#85

We need to migrate all Fugue plugins to this standard approach. We also need to keep the old way working.

  • Execution engine
  • DataFrames
  • Extensions
  • Annotation converters
@goodwanghan goodwanghan linked a pull request Jul 16, 2022 that will close this issue
Migrate execution engine parsing to triad conditional_dispatcher, add dask client to DaskExecutionEngine #339
Merged
@goodwanghan goodwanghan reopened this Jul 16, 2022
@goodwanghan goodwanghan added this to the 0.7.2 milestone Aug 18, 2022
@goodwanghan goodwanghan modified the milestones: 0.7.2, 0.7.3 Aug 27, 2022
@goodwanghan goodwanghan modified the milestones: 0.7.3, 0.7.4 Oct 3, 2022
@goodwanghan goodwanghan linked a pull request Oct 16, 2022 that will close this issue
Redesign extension and dataframe registry #375
Merged
@goodwanghan
Copy link
Collaborator Author

parse_creator, parse_processor, parse_outputter, parse_transformer are the plugins for register customer extensions, they have much more flexibility.

For example, now Spark engine can support this:

fugue/tests/fugue_spark/test_importless.py

Line 44 in 68975b4

res = transform(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
behavior change refactoring
Projects
None yet
Milestone
0.8.0
1 participant
@goodwanghan