[SPARK-5434] [EC2] Preserve spaces in EC2 path

[nchammas]

Fixes SPARK-5434.

Simple demonstration of the problem and the fix:

$ spacey_path="/path/with some/spaces"
$ dirname $spacey_path
usage: dirname path
$ echo $?
1
$ dirname "$spacey_path"
/path/with some
$ echo $?
0

[nchammas]

cc @JoshRosen @shivaram

A minor issue, but we might want to backport this. Shall I create a JIRA?

[SparkQA]

Test build #26169 has started for PR 4224 at commit 960711a.

• This patch merges cleanly.

[SparkQA]

Test build #26169 has finished for PR 4224 at commit 960711a.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26169/
Test PASSed.

[shivaram]

LGTM. Would be good to create a JIRA especially if we want to backport. @JoshRosen might have more ideas on backporting

[pwendell]

jw - what does it mean to nest quotes inside of other quotes like this?

[nchammas]

$() invokes a subshell, so the quotes inside the subshell are processed independently of those outside.

So here we're making sure dirname gets exactly 1 argument by quoting the $0, in case the value of $0 has spaces.

Then, we're making sure the output of dirname in turn is captured as 1 argument when passed to SPARK_EC2_DIR by quoting the whole subshell, in case that output has spaces.

Welcome to the most deceptive feature of Bash: Word Splitting.

[nchammas]

@shivaram JIRA created.

[andrewor14]

Ok I'm merging this into master and marking it as backport needed (since we're in the middle of a 1.2.1 release)

[srowen]

I backported this to 1.2