Easy automated configuration and deployment of Minecraft servers on AWS spot instances, with features such as instance shutdown monitoring and automatic backups and restorations using S3.
Amazon sells spare EC2 instances that aren’t currently reserved, called "spot instances", for a fraction of the cost of a reserving them. If you can mitigate their volatility risk, they’re an excellent deal and powerful enough to host demanding game servers.
Advantages:
-
More control over everything. SSH into the box, troubleshoot, adjust anything you like on the fly.
-
Better bang for your buck. More powerful systems than most hosting services offer.
-
Pay for exactly what you use. Spot instances are billed by the second, so if your server isn’t going to be used for a few days, or even overnight, just turn it off.
-
Linux-based. This might be a disadvantage, depending on your comfort level. Windows instances are not only more of a pain, they’re also far more expensive.
-
Works with any AWS region, so you can minimize latency for your player base.
Disadvantages:
-
Spot instances can be terminated at any time. This is mitigated by both the emergency shutdown monitoring script, but in practice reasonable bids for spot instances result in excellent uptime.
-
Requires initial work to setup. This isn’t a push-button solution, but once you’re done with the setup day-to-day interactions are extremely easy.
I’ve provided a couple samples of pricing based on different instance types below. When deciding what to use, cross reference your requirements against the spot instance pricing and the EC2 instance types. Be aware that spot instance prices can differ not only between AWS regions, but between availability zones as well.
All calculations below are based on an instance in us-east-1, and assume a
31-day month.
|
Note
|
AWS charges for absolutely everything. It’s critical that you monitor your bill and be aware of where you’re incurring charges. |
An i3.large is a quite powerful instance with ~16GB of RAM and a 500GB NVMe drive, meaning an EBS (beyond the 8GB root volume) isn’t required. Price estimates for the various moving pieces:
-
Current spot instance price for an i3.large is $0.0371 per hour at present.
-
An 8GB EBS is used as the root volume for the instance, at $0.10 per GB-month, or roughly $0.00013 an hour. This is deleted on instance shutdown, so no charges are incurred when the server is off.
-
Elastic IPs are free when assigned to a running instance, and $0.005 per hour when unassigned.
-
AWS data transfer is pretty expensive, at $0.09 per GB. I use about 1.4GB per day, but my server has a lot of time when it’s completely idle. For the sake of round numbers, 2GB of bandwidth a day winds up being $0.0075 an hour. This cost is very subjective.
-
S3 is actually quite cheap, especially with lifecycle management rules cleaning up old backups. 20GB of combined server archives and rotating backups is $0.023 per GB, $0.46 a month, or $0.0006 an hour.
From these numbers, when my server is running, I’m paying roughly $0.045 an hour or $33.73 a month to run it full-time. When my server is off, but I want to preserved my backups and elastic IP address, I’m paying $4.17 a month. Just preserving backups is only $0.46, which is very reasonable.
Options with less RAM more be suited to smaller loads. One example is a t2.medium, which has only 4GB of RAM and no SSD, which means it requires an EBS volume. The upside is that it only costs $0.0134 an hour. If we use a 30GB ephemeral EBS volume that’s deleted on instance termination, that adds roughly $0.0004 an hour.
Using this less powerful, less expensive instance brings considerable savings: $0.0255 an hour, or $18.97 when run for an entire month.
Start by checking out this repository, and navigating to your checkout-out directory. Sample bash commands provided are from the root directory unless otherwise noted.
You’ll need to download and install the following requirements to get started (versions I’ve tested with are in parentheses):
-
pip(20.1) -
terraform(0.12.24)
You’ll also need:
-
An appropriate version of the JDK/JVM to run the server locally at least once, allowing it to generate the necessary configuration files.
-
A fully configured AWS developer account; you’ll be connecting to it to set up the required infrastructure.
You’ll need your AWS credentials available for most of these operations, under
the minecraft profile. ~/.aws/credentials will look like:
[minecraft]
aws_access_key_id = <your_access_key_here>
aws_secret_access_key = <your_secret_key_here>To tell Ansible to use this profile, you can do one of the following:
-
export AWS_PROFILE=minecraftin the shell you’ll be running the Ansible commands in. -
Prefix each command with
AWS_PROFILE=minecraft, which sets the environment for that particular invocation. -
Change the
minecraftsection title todefault(or duplicate it). This will cause those credentials to be tried by default.
You’ll need a key pair for accessing your instance. Generate a public-private
key pair. As an example, you can do this with ssh-keygen:
ssh-keygen -t rsa -b 4096 -C "AWS"In the EC2 console, select Import Key Pair on the NETWORK & SECURITY → Key Pairs page. Upload your public key, and name it "aws-public". The launch configuration Terraform creates includes this key, allowing SSH access to Ansible (and for troubleshooting!)
You’ll need to choose one of the following methods to access your instance:
-
Use the public IP automatically assigned to your spot instance. This requires no setup on your part, but the address will be different each time the instance is started. The IP of your instance can be found under EC2 → Instances in the "IPv4 Public IP" column.
-
Use an elastic IP. This associates a known IP with your instance each time it’s started. Setup for this must be done by hand. Elastic IPs also incur a cost while they’re not in use, as well as additional charges if you make too many assignments per month. (DEPRECATED)
-
Use Route 53 to host an owned domain or subdomain, pointed at your server. Requires a domain and minor manual setup with your registrar.
You’ll need to create an elastic IP for association with your instance, providing a convenient public-facing IP. In the AWS console, do the following:
-
Enter the EC2 service.
-
Click on Elastic IPs, under the NETWORK & SECURITY menu on the left-hand side of the screen.
-
Click Allocate new address.
-
Leave the scope as "VPC", and click close.
-
You should see your new elastic IP in the list. Save the Allocation ID for later use during the setup.
Once a server has been spun up, this elastic IP will be attached to it. An allocated elastic IP is included in the price of a running instance, but you will be billed for any unassigned EIPs by the hour. For this reason, if you plan to stop your Minecraft server for long periods of time, be sure to delete your EIPs and create new ones when you’re ready to begin hosting again.
Proceed through the rest of the setup, specifying a server hostname. Once you’ve applied your Terraform config, return to do this section and do the following:
-
Enter the AWS console, and navigate to the Route 53 service.
-
Select the hosted zone for the hostname you chose.
-
You should see an
NSrecord with four hosts. -
Add a corresponding
NSrecord to your domain for each host. I leave this as an exercise for the reader. -
Now, when your server is booted, it will automatically add an
Arecord with a short TTL pointed at your server’s public IP.
Using pip, install the necessary Python 2.7 requirements. I recommend using virtualenv and virtualenvwrapper. Running the following installs Ansible, the AWS command-line interface, and libraries required for interacting with AWS programmatically.
$ mkvirtualenv minecraft
(minecraft) $ pip install -r requirements.txtYou’ll need to create a Minecraft server archive to be pulled onto your
instance each time the box is spun up. In this example, I’ll be creating an
archive for my Feed the Beast server named daftcyborg.
$ # Create a base directory named after your server name
$ mkdir daftcyborg
$ cd daftcyborg
$ # Get your base server pack. In my case, I've already downloaded the FTB server
$ ls
FTBRevelationServer_1.0.0.zip
$ unzip FTBRevelationServer_1.0.0.zip
$ # Install the server requirements, if the pack requires it
$ sh ./FTBInstall.sh
$ # Create and populate the bootstrap script - mining camp will use this to
$ # launch your server. Copy from _utitilies/_, and update with memory limits
$ # and the `.jar` you're using.
$ cp <repo-checkout-directory>/utilities/server-start.sh .
$ # Launch the server. You'll need to do this twice, once to create the
$ # eula.txt and once to generate the base
$ sh ./server-start.sh
Missing eula.txt. Startup will fail and eula.txt will be created
Make sure to read eula.txt before playing!
To continue press <enter>Open eula.txt, and agree (or don’t) to the terms and conditions.
Launch the server again, and wait for it to complete. This will generate the world base, and any settings and properties files necessary. Quit the server, and do the following as desired:
-
Remove the
worlddirectory, which is the world directory name used by default and which will (assuming you update theserver.propertiesfile) be named differently when your server is run. -
Edit
server.propertiesas desired. It is important that the server-port be left as 25565, otherwise you’ll need to adjust the Terraform configuration. Fields I recommend changing are level-name, level-seed, and motd. -
Add yourself and any other players desired to
ops.json. -
Update
server-icon.pngto a custom icon.
Copy server.properties to ansible/files/server.properties, which Ansible will
install every time over the top of the properties file in the archive, allowing
easy configuration changes.
Now, clean up your leftover base archive, since you don’t need it anymore:
$ rm FTBRevelationServer_1.0.0.zipNavigate up a level, and create a gzipped tarball:
$ cd ..
$ tar -cvzf daftcyborg-server-12-20-2017.tgz daftcyborg/Lastly, push the archive to S3:
$ # The parameterized command is 'aws s3 cp <server_file> s3://<bucket_name>/<server_name>/'
$ # My version looks like:
$ aws s3 cp daftcyborg-server-12-20-2017.tgz s3://josh-minecraft/daftcyborg/Lastly, save the full name, including file extension, of the archive you generated; it will be required when you run the setup wizard.
The recommended way to configure the system is to run the setup wizard from the root repo directory, like so:
$ ./utilities/setup.pyThis guides you through each required setting, offering default values if
available. It then takes your input and renders out terraform/variables.tf
and ansible/group_vars/all from corresponding *.j2 templates. If you like,
you can populate those templates by hand as well.
-
It’s important you choose the right aws_availability_zone, since spot prices can vary substantially from zone to zone.
-
Maximum spot price determines the maximum price you’re willing to pay per hour. Setting this wisely will prevent you from being surprised by a large bill at the end of the month.
Terraform allows you to easily setup EC2 and S3 to match your needs. To apply the terraform configuration, run:
terraform apply terraform/Once this has successfully completed, your AWS configuration is done. Unless you change your configuration, you won’t need to run this again.
The first time you do the configurations, you’ll need to bundle your local Ansible configuration and push it to your S3 bucket, under the same directory as your server is stored. A provided playbook takes care of this for you:
cd ansible/
ansible bundle.yml|
Note
|
You’ll need to repeat this step each time you adjust any server settings. |
Jump to the ansible directory, and run the start.yml playbook to configure
the instance and launch the minecraft server:
cd ansible
ansible-playbook start.ymlThis merely increases the size of the auto-scaling group from 0 → 1; the server won’t be available for a few more minutes as it provisions.
Shutting down your server is very similar, with a few extra command options that allow Ansible to SSH into the host and save the server state before killing the box itself.
cd ansible
ansible-playbook -i ec2.py --private-key=~/.ssh/aws -u ubuntu -c ssh stop.ymlWhen this playbook finishes, your instance will be gone, but the state of the server will have been preserved and pushed to S3, ready for the next time you launch it.
|
Note
|
If the auto-inventory script is taking too long, you can update
ansible/ec2.ini's regions entry with the particular AWS region you’re using.
|
|
Note
|
If using an older version of Ansible, the Paramiko library used by
default may run into errors when gathering facts from the remote host. If this
happens, add -c ssh to the ansible-playbook command above.
|
Logs from the initial provision are available via the console as well as in
/var/log/user-data.
Tests are currently available for the Prospector tool. You’ll need to install the requirements in the test directory in order to run them. From the root, with your virtual environment active:
(minecraft) $ pip install -r utilities/tests/requirements.txtNow you can launch the test suite:
(minecraft) $ python -m unittest -v utilities.tests.test_prospector