Skip to content

Commit

Permalink
[SPARK-4983] Insert waiting time before tagging EC2 instances
Browse files Browse the repository at this point in the history
The boto API doesn't support tag EC2 instances in the same call that launches them.
We add a five-second wait so EC2 has enough time to propagate the information so that
the tagging can succeed.

Author: GenTang <gen.tang86@gmail.com>
Author: Gen TANG <gen.tang86@gmail.com>

Closes #3986 from GenTang/spark-4983 and squashes the following commits:

13e257d [Gen TANG] modification of comments
47f0675 [GenTang] print the information
ab7a931 [GenTang] solve the issus spark-4983 by inserting waiting time
3179737 [GenTang] Revert "handling exceptions about adding tags to ec2"
6a8b53b [GenTang] Revert "the improvement of exception handling"
13e97a6 [GenTang] Revert "typo"
63fd360 [GenTang] typo
692fc2b [GenTang] the improvement of exception handling
6adcf6d [GenTang] handling exceptions about adding tags to ec2

(cherry picked from commit 0f3a360)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
  • Loading branch information
GenTang authored and JoshRosen committed Feb 6, 2015
1 parent 09da688 commit 36f70de
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions ec2/spark_ec2.py
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,9 @@ def launch_cluster(conn, opts, cluster_name):
master_nodes = master_res.instances
print "Launched master in %s, regid = %s" % (zone, master_res.id)

# This wait time corresponds to SPARK-4983
print "Waiting for AWS to propagate instance metadata..."
time.sleep(5)
# Give the instances descriptive names
for master in master_nodes:
master.add_tag(
Expand Down

0 comments on commit 36f70de

Please sign in to comment.