Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issue85 #87

Merged
merged 2 commits into from
Dec 20, 2021
Merged

Fix issue85 #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
30a4b0c
Add KarlKeller to contributing list
Nov 30, 2021
52a4a60
Adds AWS subnet, vpc, IAMRole, and zone support.
Dec 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
See https://github.com/OpenDroneMap/documents/blob/master/CONTRIBUTING.md

KarlKeller

15 changes: 12 additions & 3 deletions docs/aws.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ In order to use ClusterODM with AWS:

* Create an Amazon Web Services Account
* Select a region to run your instances in - `us-east-2` typically has the cheapest instance costs.
* Create a security group in this region which allows inbound access from your ClusterODM master instance on TCP port 3000.
* Note the zone selected for the region, e.g. 'a'. This will appear appended to the region in 'availability zone'.
* Select/Create a VPC in which the resources will operate.
* Select/Create a subnet within the VPC.
* Create a security group in this region, VPC, and subnet which allows inbound access from your ClusterODM master instance on TCP port 3000. Note the name of the security group not the ID.
* Create an S3 bucket in this region to handle results. Don't configure this bucket to block public access.
* Select an AMI (machine image) to run - Ubuntu has a [handy AMI finder](https://cloud-images.ubuntu.com/locator/ec2/).
* Create an IAM account for ClusterODM to use, which has EC2 and S3 permissions.
Expand All @@ -30,12 +33,15 @@ the on-demand instance cost - you'll always pay the current market price, not yo
"endpoint": "s3.us-west-2.amazonaws.com",
"bucket": "bucketname"
},
"vpc": "",
"subnet": "",
"securityGroup": "CHANGEME!",

"monitoring": false,
"maxRuntime": -1,
"maxUploadTime": -1,
"region": "us-west-2",
"zone": "a",
"tags": ["type,clusterodm"],

"ami": "ami-07b4f3c02c7f83d59",
Expand Down Expand Up @@ -63,14 +69,17 @@ the on-demand instance cost - you'll always pay the current market price, not yo
| accessKey | AWS Access Key |
| secretKey | AWS Secret Key |
| s3 | S3 bucket configuration. Note that the bucket should *not* be configured to block public access. |
| vpc | The virtual private cloud in which the instances operate. Not providing this assumes a default setting for VPC within the AWS environment. |
| subnet | The subnet supporting the instances. Not providing this assumes a default setting for the subnet within the AWS environment. |
| securityGroup | AWS Security Group name (not ID). Must exist and allow incoming connections from your ClusterODM host on port TCP/3000. |
| createRetries | Number of attempts to create a droplet before giving up. Defaults to 1.
| createRetries | Number of attempts to create a droplet before giving up. Defaults to 1. |
| maxRuntime | Maximum number of seconds an instance is allowed to run ever. Set to -1 for no limit. |
| maxUploadTime | Maximum number of seconds an instance is allowed to receive file uploads. Set to -1 for no limit. |
| monitoring | Set to true to enable detailed Cloudwatch monitoring for the instance. |
| region | Region identifier where the instances should be created. |
| zone | Zone identifier where the instances should be created. |
| ami | The AMI (machine image) to launch this instance from. |
| tags | Comma-separated list of key,value tags to associate to the instance. |
| tags | Comma-separated list of key,value tags to associate to the instance. |
| spot | Whether to request spot instances. If this is true, a `spotPrice` needs to be provided in the `imageSizeMapping`. |
| imageSizeMapping | Max images count to instance size mapping. (See below.) |
| addSwap | Optionally add this much swap space to the instance as a factor of total RAM (`RAM * addSwap`). A value of `1` sets a swapfile equal to the available RAM. |
Expand Down
28 changes: 26 additions & 2 deletions libs/asr-providers/aws.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,13 +30,15 @@ module.exports = class AWSAsrProvider extends AbstractASRProvider{
"endpoint": "CHANGEME!",
"bucket": "CHANGEME!"
},

"vpc": "",
"subnet": "",
"securityGroup": "CHANGEME!",
"maxRuntime": -1,
"maxUploadTime": -1,
"instanceLimit": -1,
"createRetries": 1,
"region": "us-west-2",
"zone": "",
"monitoring": false,
"tags": ["clusterodm"],
"ami": "ami-07b4f3c02c7f83d59",
Expand All @@ -47,7 +49,8 @@ module.exports = class AWSAsrProvider extends AbstractASRProvider{
],

"addSwap": 1,
"dockerImage": "opendronemap/nodeodm"
"dockerImage": "opendronemap/nodeodm",
"iamrole": ""
}, userConfig);
}

Expand Down Expand Up @@ -180,6 +183,27 @@ module.exports = class AWSAsrProvider extends AbstractASRProvider{
args.push(this.getConfig("engineInstallUrl"));
}

if (this.getConfig("zone").length > 0){
args.push("--amazonec2-zone")
args.push(this.getConfig("zone"));
}

if (this.getConfig("vpc").length > 0){
args.push("--amazonec2-vpc-id")
args.push(this.getConfig("vpc"));
}

if (this.getConfig("subnet").length > 0){
args.push("--amazonec2-subnet-id")
args.push(this.getConfig("subnet"));
}


if (this.getConfig("iamrole").length > 0){
args.push("--amazonec2-iam-instance-profile")
args.push(this.getConfig("iamrole"));
}

return args;
}
};