Fixes whitespace #2

skrawcz · 2023-02-23T21:46:12Z

Fixes README whitespace for CI.

# This is the 1st commit message: Update graph_functions.py Describes what to do in `graph_functions.py` # This is the commit message #2: Adds comments to lifecycle base # This is the commit message #3: Update h_ray.py with comments for ray tracking compatibility # This is the commit message #4: Replicate previous error # This is the commit message #5: Inline function, unsure if catching errors and exceptions to be handadled differently # This is the commit message #6: BaseDoRemoteExecute has the added Callable function that snadwisched lifecycle hooks # This is the commit message #7: method fails, says AssertionError about ray.remote decorator # This is the commit message #8: simple script for now to check telemetry, execution yield the ray.remote AssertionError # This is the commit message #9: passing pointer through and arguments to lifecycle wrapper into ray.remote # This is the commit message #10: post-execute hook for node not called # This is the commit message #11: finally executed only when exception occurs, hamilton tracker not executed # This is the commit message #12: atexit.register does not work, node keeps running inui # This is the commit message #13: added stop() method, but doesn't get called # This is the commit message #14: Ray telemtry works for single node, problem with connected nodes # This is the commit message #15: Ray telemtry works for single node, problem with connected nodes # This is the commit message #16: Ray telemtry works for single node, problem with connected nodes # This is the commit message #17: Fixes ray object dereferencing Ray does not resolve nested arguments: https://docs.ray.io/en/latest/ray-core/objects.html#passing-object-arguments So one option is to make them all top level: - one way to do that is to make the other arguments not clash with any possible user parameters -- hence the `__` prefix. This is what I did. - another way would be in the ray adapter, wrap the incoming function, and explicitly do a ray.get() on any ray object references in the kwargs arguments. i.e. keep the nested structure, but when the ray task starts way for all inputs... not sure which is best, but this now works correctly. # This is the commit message #18: ray works checkpoint, pre-commit fixed # This is the commit message #19: fixed graph level telemtry proposal # This is the commit message #20: pinned ruff # This is the commit message #21: Correct output, added option to start ray cluster # This is the commit message #22: Unit test mimicks the DoNodeExecute unit test

Refactor to enable RayGraphAdapter and HamiltonTracker to work well together This is a squash commit: - issue=#1079 - PR=#1103 Update graph_functions.py Describes what to do in `graph_functions.py` Adds comments to lifecycle base Update h_ray.py with comments for ray tracking compatibility Replicate previous error Inline function, unsure if catching errors and exceptions to be handadled differently BaseDoRemoteExecute has the added Callable function that snadwisched lifecycle hooks method fails, says AssertionError about ray.remote decorator simple script for now to check telemetry, execution yield the ray.remote AssertionError passing pointer through and arguments to lifecycle wrapper into ray.remote post-execute hook for node not called finally executed only when exception occurs, hamilton tracker not executed atexit.register does not work, node keeps running inui added stop() method, but doesn't get called Ray telemtry works for single node, problem with connected nodes Ray telemtry works for single node, problem with connected nodes Ray telemtry works for single node, problem with connected nodes Fixes ray object dereferencing Ray does not resolve nested arguments: https://docs.ray.io/en/latest/ray-core/objects.html#passing-object-arguments So one option is to make them all top level: - one way to do that is to make the other arguments not clash with any possible user parameters -- hence the `__` prefix. This is what I did. - another way would be in the ray adapter, wrap the incoming function, and explicitly do a ray.get() on any ray object references in the kwargs arguments. i.e. keep the nested structure, but when the ray task starts way for all inputs... not sure which is best, but this now works correctly. ray works checkpoint, pre-commit fixed fixed graph level telemtry proposal pinned ruff Correct output, added option to start ray cluster Unit test mimicks the DoNodeExecute unit test All commits: - Update graph_functions.py Describes what to do in `graph_functions.py` Adds comments to lifecycle base Update h_ray.py with comments for ray tracking compatibility Replicate previous error Inline function, unsure if catching errors and exceptions to be handadled differently BaseDoRemoteExecute has the added Callable function that snadwisched lifecycle hooks method fails, says AssertionError about ray.remote decorator simple script for now to check telemetry, execution yield the ray.remote AssertionError passing pointer through and arguments to lifecycle wrapper into ray.remote post-execute hook for node not called finally executed only when exception occurs, hamilton tracker not executed atexit.register does not work, node keeps running inui added stop() method, but doesn't get called Ray telemtry works for single node, problem with connected nodes Ray telemtry works for single node, problem with connected nodes Ray telemtry works for single node, problem with connected nodes Fixes ray object dereferencing Ray does not resolve nested arguments: https://docs.ray.io/en/latest/ray-core/objects.html#passing-object-arguments So one option is to make them all top level: - one way to do that is to make the other arguments not clash with any possible user parameters -- hence the `__` prefix. This is what I did. - another way would be in the ray adapter, wrap the incoming function, and explicitly do a ray.get() on any ray object references in the kwargs arguments. i.e. keep the nested structure, but when the ray task starts way for all inputs... not sure which is best, but this now works correctly. ray works checkpoint, pre-commit fixed fixed graph level telemtry proposal pinned ruff Correct output, added option to start ray cluster Unit test mimicks the DoNodeExecute unit test Refactored driver so all tests pass Refactored driver so all tests pass Refactored driver so all tests pass Refactored driver so all tests pass Workaround to not break ray by calling init on an open cluster raw_execute does not have post_graph_execute and is private now Correct version for depraction warning all tests work this looks better ruff version comment Refactored pre- and post-graph-execute hooks outside of raw_execute which now has deprecation warning added readme, notebook and made script cli interactive made cluster init optional through inserting config dict User has option to shutdown ray cluster Co-authored-by: Stefan Krawczyk <stefan@dagworks.io>

Fixes whitespace

d886e42

skrawcz merged commit 78e04ca into main Feb 23, 2023

skrawcz deleted the fix_whitespace branch February 23, 2023 21:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes whitespace #2

Fixes whitespace #2

skrawcz commented Feb 23, 2023

Fixes whitespace #2

Fixes whitespace #2

Conversation

skrawcz commented Feb 23, 2023