Skip to content

Commit

Permalink
[SPARK-4745] Fix get_existing_cluster() function with multiple securi…
Browse files Browse the repository at this point in the history
…ty groups

The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups == cluster_name + "-master" (or "-slaves"). This fix allows for multiple security groups by checking if the cluster_name + "-master" security group is in the list of groups for a particular instance.

Author: alexdebrie <alexdebrie1@gmail.com>

Closes #3596 from alexdebrie/master and squashes the following commits:

9d51232 [alexdebrie] Fix get_existing_cluster() function with multiple security groups
  • Loading branch information
alexdebrie authored and JoshRosen committed Dec 4, 2014
1 parent 8dae26f commit 794f3ae
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions ec2/spark_ec2.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,9 +504,9 @@ def get_existing_cluster(conn, opts, cluster_name, die_on_error=True):
active = [i for i in res.instances if is_active(i)]
for inst in active:
group_names = [g.name for g in inst.groups]
if group_names == [cluster_name + "-master"]:
if (cluster_name + "-master") in group_names:
master_nodes.append(inst)
elif group_names == [cluster_name + "-slaves"]:
elif (cluster_name + "-slaves") in group_names:
slave_nodes.append(inst)
if any((master_nodes, slave_nodes)):
print "Found %d master(s), %d slaves" % (len(master_nodes), len(slave_nodes))
Expand Down

0 comments on commit 794f3ae

Please sign in to comment.