[SPARK-4839][YARN] Adding a documentation about dynamic resource allocation. #3689
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Can one of the admins verify this patch? |
oza
changed the title
| <td><code>spark.dynamicAllocation.enabled</code></td> | ||
| <td>false</td> | ||
| <td> | ||
| Enabling dynamic resource allocation. |
There was a problem hiding this comment.
Can you add more explanation of what dynamic resource allocation is here? As a user unfamiliar with the feature, this description doesn't tell me enough to know whether I want to turn it on.
|
/cc @aarondav @andrewor14 |
|
Thanks for your comment, I'm updating it. |
Signed-off-by: Tsuyoshi Ozawa <ozawa.tsuyoshi@gmail.com>
Signed-off-by: Tsuyoshi Ozawa <ozawa.tsuyoshi@gmail.com>
|
Updated a description of dynamic resource allocation based on a comment by @ash211. |
|
Hey @oza thanks for doing this. I actually intend to structure this a little differently in its own separate page and there is another JIRA (SPARK-4140) that I filed earlier for this. I'll take this up and work on it in the next day or two. In the mean time would you mind closing this PR? |
|
@andrewor14 OK, feel free to use this PR. Closing this as duplicated one. |
asfgit
pushed a commit
that referenced
this pull request
Dec 20, 2014
Once the external shuffle service is also documented, the dynamic allocation section will link to it. Let me know if the whole dynamic allocation should be moved to its separate page; I personally think the organization might be cleaner that way. This patch builds on top of oza's work in #3689. aarondav pwendell Author: Andrew Or <andrew@databricks.com> Author: Tsuyoshi Ozawa <ozawa.tsuyoshi@gmail.com> Closes #3731 from andrewor14/document-dynamic-allocation and squashes the following commits: 1281447 [Andrew Or] Address a few comments b9843f2 [Andrew Or] Document the configs as well 246fb44 [Andrew Or] Merge branch 'SPARK-4839' of github.com:oza/spark into document-dynamic-allocation 8c64004 [Andrew Or] Add documentation for dynamic allocation (without configs) 6827b56 [Tsuyoshi Ozawa] Fixing a documentation of spark.dynamicAllocation.enabled. 53cff58 [Tsuyoshi Ozawa] Adding a documentation about dynamic resource allocation. (cherry picked from commit 15c03e1) Signed-off-by: Andrew Or <andrew@databricks.com>
asfgit
pushed a commit
that referenced
this pull request
Dec 20, 2014
Once the external shuffle service is also documented, the dynamic allocation section will link to it. Let me know if the whole dynamic allocation should be moved to its separate page; I personally think the organization might be cleaner that way. This patch builds on top of oza's work in #3689. aarondav pwendell Author: Andrew Or <andrew@databricks.com> Author: Tsuyoshi Ozawa <ozawa.tsuyoshi@gmail.com> Closes #3731 from andrewor14/document-dynamic-allocation and squashes the following commits: 1281447 [Andrew Or] Address a few comments b9843f2 [Andrew Or] Document the configs as well 246fb44 [Andrew Or] Merge branch 'SPARK-4839' of github.com:oza/spark into document-dynamic-allocation 8c64004 [Andrew Or] Add documentation for dynamic allocation (without configs) 6827b56 [Tsuyoshi Ozawa] Fixing a documentation of spark.dynamicAllocation.enabled. 53cff58 [Tsuyoshi Ozawa] Adding a documentation about dynamic resource allocation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Tsuyoshi Ozawa ozawa.tsuyoshi@gmail.com