[RLlib] Change config.fault_tolerance default behavior (from recreate_failed_env_runners=False to True).
#48286
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sven1977
requested review from
maxpumperla,
simonsays1980 and
a team
as code owners
simonsays1980
approved these changes
Oct 28, 2024
| ``recreate_failed_env_runners``: When set to True, your Algorithm will attempt to replace/recreate any failed worker(s) with newly created one(s). This way, your number of workers will never decrease, even if some of them fail from time to time. | ||
| ``restart_failed_sub_environments``: When set to True and there is a failure in one of the vectorized sub-environments in one of your workers, the worker will try to recreate only the failed sub-environment and re-integrate the newly created one into your vectorized env stack on that worker. | ||
| ``restart_failed_env_runners``: When set to True (default), your Algorithm will attempt to restart any failed EnvRunner and replace it with a newly created one. This way, your number of workers will never decrease, even if some of them fail from time to time. | ||
| ``ignore_env_runner_failures``: When set to True, your Algorithm will not crash due to an EnvRunner error, but continue for as long as there is at least one functional worker remaining. This setting is ignored when ``restart_failed_env_runners=True``. |
There was a problem hiding this comment.
Why do we want to ignore this setting when restart_failed_env_runners=True? Following the semantics a user would understand that if failed env runners are restarted and the algorithm does not ignore failed env runners, the setting has no meaning.
There was a problem hiding this comment.
If restart_failed_env_runners=True, then RLlib doesn't have a choice to a) ignore or b) crash, b/c it has to restart the failed EnvRunner, so this setting (ignore_env_runner_failures) becomes irrelevant.
See also: https://docs.ray.io/en/latest/rllib/rllib-training.html#rllib-scaling-guide
|
|
||
| Note that only one of ``ignore_env_runner_failures`` or ``recreate_failed_env_runners`` may be set to True (they are mutually exclusive settings). However, | ||
| Note that only one of ``ignore_env_runner_failures`` or ``restart_failed_env_runners`` should be set to True (they are mutually exclusive settings). However, |
There was a problem hiding this comment.
This might need further explanation.
| @@ -3260,11 +3257,11 @@ def fault_tolerance( | |||
| setting is ignored. | |||
| ignore_env_runner_failures: Whether to ignore any EnvRunner failures | |||
There was a problem hiding this comment.
Ah, here it becomes clear. Maybe we refer to this section or maybe I overread it above.
…ge_fault_tolerance_default_settings
Jay-ju
pushed a commit
to Jay-ju/ray
that referenced
this pull request
Nov 5, 2024
…ate_failed_env_runners=False` to `True`). (ray-project#48286)
JP-sDEV
pushed a commit
to JP-sDEV/ray
that referenced
this pull request
Nov 14, 2024
…ate_failed_env_runners=False` to `True`). (ray-project#48286) Signed-off-by: JP-sDEV <jon.pablo80@gmail.com>
mohitjain2504
pushed a commit
to mohitjain2504/ray
that referenced
this pull request
Nov 15, 2024
…ate_failed_env_runners=False` to `True`). (ray-project#48286) Signed-off-by: mohitjain2504 <mohit.jain@dream11.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change
config.fault_tolerancedefault behavior (fromrecreate_failed_env_runners=FalsetoTrue).The new logic is to always try and restart failed
EnvRunnersby default.Also renamed
recreate_failed_env_runnerstorestart_failed_env_runnersto match other similar settings and ray's mainmax_num_restartsvocabulary.Why are these changes needed?
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.