Ray-2.39.0

Ray Libraries

Ray Data

🔨 Fixes:

• Fixed InvalidObjectError edge case with Dataset.split() (#48130)
• Made Concatenator preserve order of concatenated columns (#47997)

📖 Documentation:

• Improved documentation around Parquet column and predicate pushdown (#48095)
• Marked num_rows_per_file parameter of write APIs as experimental (#48208)
• One hot encoder now returns an encoded vector (#48173)
• transform_batch no longer fails on missing columns (#48137)

🏗 Architecture refactoring:

• Dataset.count() now uses a Count logical operator (#48126)

🗑 Deprecations:

• Removed long-deprecated set_progress_bars (#48203)

Ray Train

🔨 Fixes:

• Safely check if the storage filesystem is pyarrow.fs.S3FileSystem (#48216)

Ray Tune

🔨 Fixes:

• Safely check if the storage filesystem is pyarrow.fs.S3FileSystem (#48216)

Ray Serve

💫 Enhancements:

• Cancelled requests now return a serve.RequestCancelledError (#48444)
• Exposed application source in app details model (#45522)

🔨 Fixes:

• Basic HTTP deployments will now return “Internal Server Error” instead of a traceback to match FastAPI behavior (#48491)
• Fixed an issue where high values of max_ongoing_requests couldn’t be reached due to an interaction with core’s max_concurrency (#48274)
• Fixed an edge case where pending requests were not canceled properly (#47873)
• Removed deprecated API to set route_prefix per-deployment (#48223)

📖 Documentation:

• Added ProxyStatus model to reference docs (#48299)
• Added ApplicationStatus model to reference docs (#48220)

RLlib

💫 Enhancements:

• Upgrade to gymnasium==1.0.0 (support new API for vector env resets). (#48443, #45328)
• Add off-policy'ness metric to new API stack. (#48227)
• Validate episodes before adding them to the buffer. (#48083)

📖 Documentation:

• New example script for custom metrics on EnvRunners (using MetricsLogger API on the new stack). (#47969)
• Do-over: New RLlib index page. (#48285, #48442)
• Do-over: Example script for AutoregressiveActionsRLM. (#47972)

🏗 Architecture refactoring:

• New API stack on by default for PPO. (#48284)
• Change config.fault_tolerance default behavior (from recreate_failed_env_runners=False to True). (#48286)

🔨 Fixes:

• Various bug and CI fixes: #47993, #48450, #48213
• Cleanup evaluation folder (#48493)

Ray Core

🎉 New Features:

• [CompiledGraphs] Support all reduce collective in aDAG (#47621)
• [CompiledGraphs] Add visualization of compiled graphs (#47958)

💫 Enhancements:

• [Distributed Debugger] The distributed debugger can now be used without having to set RAY_DEBUG=1, see #48301 and https://docs.ray.io/en/latest/ray-observability/ray-distributed-debugger.html. If you want to restore the previous behavior and use the CLI based debugger, you need to set RAY_DEBUG=legacy.
• [Core] Add more infos to each breakpoint for ray debug CLI (#48202)
• [Core] Add demands info to GCS debug state (#48115)
• [Core] Add PENDING_ACTOR_TASK_ARGS_FETCH and PENDING_ACTOR_TASK_ORDERING_OR_CONCURRENCY TaskStatus (#48242)
• [Core] Add metrics ray_io_context_event_loop_lag_ms. (#47989)
• [Core] Better log format when show the disk size (#46869)
• [CompiledGraphs] Support asyncio.gather on multiple CompiledDAGFutures (#47860)
• [CompiledGraphs] Raise an exception if a leaf node is found during compilation (#47757)

🔨 Fixes:

• [Core] Posts CoreWorkerMemoryStore callbacks onto io_context to fix deadlock (#47833)

Dashboard

🔨 Fixes:

• [Dashboard] Reworking dashboard_max_actors_to_cache to RAY_maximum_gcs_destroyed_actor_cached_count (#48229)

Thanks

Many thanks to all those who contributed to this release!

@akyang-anyscale, @rkooo567, @bveeramani, @dayshah, @martinbomio, @khluu, @justinvyu, @slfan1989, @alexeykudinkin, @simonsays1980, @vigneshka, @ruisearch42, @rynewang, @scottjlee, @jjyao, @JoshKarpel, @win5923, @MengjinYan, @MortalHappiness, @ujjawal-khare-27, @zcin, @ccoulombe, @Bye-legumes, @dentiny, @stephanie-wang, @LeoLiao123, @dengwxn, @richo-anyscale, @pcmoritz, @sven1977, @omatthew98, @GeneDer, @srinathk10, @can-anyscale, @edoakes, @kevin85421, @aslonnie, @jeffreyjeffreywang, @ArturNiederfahrenhorst