Lesson 15: Setting up AWS

AWS Setup

Amazon Web Services (AWS) allows you to easily launch computing resources in the cloud. It provides many services to handle storage (EBS or S3) and computing (EC2 or EMR). With Ec2 we can launch virtual machines that will run in data center. These will generally be more powerful machines than our local computers and we can launch one large machine or many machines in a cluster (machines that do distributed computing leveraging a framework like Hadoop or Spark). AWS also overs a simpler UI to write MapReduce code for Hadoop through their EMR offering. This allows you to not have to maintain a cluster, but just submit a task to an existing cluster.

There are a few steps necessary to get going on AWS.

1. Creating an AWS account
2. Retrieving you access keys
3. Generating a keypair for EC2

Creating an account

Please visit AWS. In the upper right hand side you'll see a login box, you can login here with your Amazon credentials.

Once you've logged in, above the dashboard you'll see a box that lists a region (i.e. U.S. East Virginia or U.S. West Oregon). This tells us which Amazon datacenter our services will launch in. This may be useful for optimizing performance as we will likely want to ensure that our data and computing resources as well as different nodes in our system are in the same region.

Please select U.S. East Virginia if it is not selected.

Retrieving your keypairs

• In the right corner select > IMA
• Now on the right hand side select Users

Here you can create a user by selecting Create User. Create a user with your name. When prompted download the credentials.csv file

In the bottom panel select Security > Add Policy. We will give this user administrator privileges.

Creating a a KeyPair for EC2

• From the main dashboard, select EC2
• On the right hand side, select KeyPairs, and then create a new one. You can name it and save the .pem locally wherever you'd like.
• Once you've downloaded it, find it locally and run chmod 600 <keypair_file>.pem