Skip to content

Commit

Permalink
[SPARK-4652][DOCS] Add docs about spark-git-repo option
Browse files Browse the repository at this point in the history
There might be some cases when WIPS spark version need to be run
on EC2 cluster. In order to setup this type of cluster more easily,
add --spark-git-repo option description to ec2 documentation.

Author: lewuathe <lewuathe@me.com>
Author: Josh Rosen <joshrosen@databricks.com>

Closes apache#3513 from Lewuathe/doc-for-development-spark-cluster and squashes the following commits:

6dae8ee [lewuathe] Wrap consistent with other descriptions
cfaf9be [lewuathe] Add docs about spark-git-repo option

(Editing / cleanup by Josh Rosen)
  • Loading branch information
Lewuathe authored and JoshRosen committed Dec 4, 2014
1 parent 743a889 commit ab8177d
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions docs/ec2-scripts.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,11 @@ another.
specified version of Spark. The `<version>` can be a version number
(e.g. "0.7.3") or a specific git hash. By default, a recent
version will be used.
- `--spark-git-repo=<repository url>` will let you run a custom version of
Spark that is built from the given git repository. By default, the
[Apache Github mirror](https://github.com/apache/spark) will be used.
When using a custom Spark version, `--spark-version` must be set to git
commit hash, such as 317e114, instead of a version number.
- If one of your launches fails due to e.g. not having the right
permissions on your private key file, you can run `launch` with the
`--resume` option to restart the setup process on an existing cluster.
Expand Down

0 comments on commit ab8177d

Please sign in to comment.