-
Notifications
You must be signed in to change notification settings - Fork 28.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-4771][Docs] Document standalone cluster supervise mode #3627
Conversation
@@ -257,7 +257,7 @@ To run an interactive Spark shell against the cluster, run the following command | |||
|
|||
You can also pass an option `--total-executor-cores <numCores>` to control the number of cores that spark-shell uses on the cluster. | |||
|
|||
# Launching Compiled Spark Applications | |||
# Launching Spark Applications |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked that there are no references to this section
Test build #24197 has started for PR 3627 at commit
|
Test build #24197 has finished for PR 3627 at commit
|
Test PASSed. |
@@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. ` | |||
To control the application's configuration or execution environment, see | |||
[Spark Configuration](configuration.html). | |||
|
|||
Additionally, standalone `cluster` mode supports restarting your application on failure. To use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you define what is meant by failure? How is failure detected? Non-zero exit code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additionally, the cluster
mode can be optionally asked to automatically restart your application if it fails with non-zero exit code.
Test build #24284 has started for PR 3627 at commit
|
@@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. ` | |||
To control the application's configuration or execution environment, see | |||
[Spark Configuration](configuration.html). | |||
|
|||
Additionally, standalone `cluster` mode supports restarting your application automatically if it |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: my point was also that it is known that we are talking about "standalone" so you can just say "cluster mode". But its okay.
LGTM. |
Test build #24284 has finished for PR 3627 at commit
|
Test PASSed. |
Ok merging into master thanks. I'll add a note to back port this later. |
tdas looks like streaming already refers to the supervise mode. The link from there is broken though. Author: Andrew Or <andrew@databricks.com> Closes #3627 from andrewor14/document-supervise and squashes the following commits: 9ca0908 [Andrew Or] Wording changes 2b55ed2 [Andrew Or] Document standalone cluster supervise mode
JK I just back ported this into branch-1.2 and 1.1. |
tdas looks like streaming already refers to the supervise mode. The link from there is broken though. Author: Andrew Or <andrew@databricks.com> Closes #3627 from andrewor14/document-supervise and squashes the following commits: 9ca0908 [Andrew Or] Wording changes 2b55ed2 [Andrew Or] Document standalone cluster supervise mode
@tdas looks like streaming already refers to the supervise mode. The link from there is broken though.