A Boilerplate for developing applications with Node

This is a boilerplate application for developing using Node.js, with Bookshelf and Express. Although this can work as an api server for any frontend, it was specifically developed to be used with the ember-boilerplate application I've created. The idea is that using these boilerplates as a starting point, you can go from 0 to a deployed (on AWS), production, user ready application in 1-2 hours.

Requirements

• Git (http://git-scm.com/)
• Node (https://nodejs.org/en/)
• npm (https://www.npmjs.com/)
• postgres (http://www.postgresql.org/)
• redis (http://redis.io/) (optional)

Getting started

• Clone the repository git clone
• Install packages npm install
• Modify the config files (development, test, production) - See this section
• Make sure the db is created in postgres createdb my_db_name
• Migrate the db NODE_ENV='development' knex migrate:latest
• Start the server with NODE_ENV='development' grunt server

A Note on Redis

I originally built this app using the lightning deploy method for ember, which deploys the index.html file to Redis. This is the preferred method, however I found out that Redis is not supported under the free tier for AWS. Therefore the option to use redis is still available (commented out). You'll want to modify package.json to keep ember-cli-deploy and ember-cli-deploy-ssh-tunnel, but all other deploy plugins can be replaced by the lightning pack.

The alternative is to mimic the lightning deploy by deploying the index.html file to the postgres db, and reading the value from there. In all other respects it works very similarly, it is just a little less clean, and a little less fast. It is free though.

Modifying config files

In order to provide config files for three different environments (development, test, and production), there are three different files that should have identical keys in them. The values stored in those keys should reflect the desired configuration for that environment.

• companyName should be the same accross environments
• rootUrl should be http://localhost:4200 locally, and the url to your web application for production
• For the local database, host should be 127.0.0.1, user should correspond to your computer user, password can be empty, and database should match the name of the database created using createdb
• For the test database, host should be 127.0.0.1, user should correspond to your computer user, password can be empty, and database should match the name of the database created using createdb. You should create a separate db for testing
• For the production database, this information will be filled in once you set up a DB on your AWS account
• There should be no need to change anything for redis locally, but once you set up a cluster on AWS the redis endpoint will need to be added (if using the redis option)
• If not using redis, the indexFileTableName should be specified. The default is {frontend app name (underscored)}_bootstrap
• jwtSecret and sessionSecret are secret strings to use for the jwtKey and express session
• I like using mailchimp to manage an email list. In order to take advantage of this you will need to register a free mailchimp account, generate an apiKey, and a mailing list with a specific ID. I also made it so that you can create different groups in your mailchimp list, to determine who signed up exclusively for the mailing list, and who registered as a full user
• This app uses nodemailer with the gmail service. In order to configure this you simply need to sign up for a gmail account and set the email and password in the username and password fields
• Set the feedbackEmail to whatever email you would like feedback from the site sent to
• Set the contactEmail to whatever you would like contact from users sent to

Setting up resources and deploying

Set up the AWS resources for deployment to production

Create a gmail account

Start by creating a gmail account (or use an existing one) that you will be able to use for AWS, nodemailer, mailchimp, etc... Gmail Signup

Create a mailchimp account

Just go to the mailchimp signup page and sign up for an account.

Generate an apiKey

Mailchimp docs

• Click on your name (upper right) > Account > Extras > API Keys
• Scroll down and click Create A Key to generate your key
• Copy and paste the key into your environment variables

Create a list

Mailchimp docs

• On the navbar, click Lists > Create a List
• Fill in your details and click Save

Create Groups

Mailchimp docs

• For your list, click on the dropdown (on the right) > Manage Subscribers
• Click Groups > Create Groups
• Enter a title (Like 'Signup Choice') and Different Choices (Like 'Registered User' and 'Mailing List Only')

Get your Mailchimp API ID's and fill in some of production.js

To get the the Mailchimp List Id and Group Ids, you need to go to the Mailchimp API Playground

• Input your API Key, and select Lists > select your List
  • The id field has your mailingListId
  • Click the subresources dropdown > select interest-categories > select the Group Category you created
    • The id field has you categoryId
    • Click the subresources dropdown > select interests > choose each group in turn
      • For each group, the id field has the registeredUserId (for Registered User) and mailingListId (for Mailing List Only)

Create an AWS account

The free AWS account lasts for 12 months and will give you basic features. Sign up here After it is created and ready, sign in to the console

Create an IAM user

• In the top left, click on Services > Security and Identity > IAM
• Click on Groups (on the left)
• Click on Create New Group
  • Create a group called FullAdmin
  • Filter by 'FullAccess'
  • Select AmazonRDSFullAccess, AmazonEC2FullAccess, IAMFullAccess, AmazonElastiCacheFullAccess, AmazonS3FullAccess, and PowerUserAccess
  • Finish creating the group
• Click on Users (on the left)
• Click on Create New Users
  • Enter in a username for yourself, making sure to generate an access key for each user
  • Create the user, and download the CSV with the access key
• Click on Groups again
• Click on your FullAdmin group, and then Add Users to Group
• Select your IAM profile, and then Add Users to add yourself to the group

Create an s3 bucket

• Amazon Docs
• In the top left, click on Services > S3
• Click Create Bucket, and enter a name for your bucket (e.g. {app-name}-bucket)
• It's best to use the default region provided, but make sure to remember it and use it for the reset of your AWS services (because they all need to exist within the same Virtual Private Cloud)

Create a security group

Amazon Docs

• In the top left, click on Services > VPC
• On the left, scroll down to Security Groups
• Click Create Security Group
• Give it a name and description, and click Create

(Optional) Create a redis elasticache cluster

Amazon Docs

• In the top left, click on Services > Elasticache
• Click on Get Started Now
• Choose Redis
• Choose your preferred option for the cluster specification. Note that redis is not covered under the free tier for AWS
• For Configuration, choose an appropriate name and description. The rest of the fields can be left as defaults

Create an RDS instance

• Amazon Docs
• In the top left, click on Services > RDS
• Click Get Started Now
• Choose PostgreSQL
• Select Dev/Test
• Select the db.t2.micro DB instance
• Select No for Multi-AZ Deployment
• Select a unique DB Instance Identifier
• Select a username and password and remember them for the connection
• Click on Next Step
• Make sure Publicly Accessible is set to No
• Use the default VPC security group
• Select a database name - it will be used for the connection
• Click Launch DB Instance
• Additional resources - Working with RDS in a VPC

Deploy using elastic beanstalk

• Deploying to EB is very easy with the EB CLI
• The docs do an excellent job explaining how to install the EB CLI and configure it for your project
• When running eb init, make sure to select the same region that you selected for your elasticache and RDS instances
• When prompted to set up SSH, do so. This will be necessary to deploy the ember app. Unfortunately, ember-cli-deploy-ssh-tunnel does not support password encrypted key-pairs, so leave the password blank!

Make sure security group rules are in sync

• I've found that for the resources just created (elasticache, RDS, EC2 (through elasticbeanstalk)), the best way to ensure you don't have trip ups is to make sure they are all on the same security group, configured to accept traffic between them
• If you created a security group in the beginning, and assigned it to your RDS and elasticache clusters, they should already have it
• If not
  • You can change the security group on your RDS instance by visiting the RDS Console > Instances > Click on Instance Actions > Modify
  • You can change the security group on your ElastiCache cluster by visiting the ElstiCache Console > Replication Groups > Click on your Replication Group > Click Modify
• You will need to add your custom security group to your EC2 Instance
  • Go to the EC2 Console > Instances > Select your Instance > Actions > Networking > Change Security Groups. From there make sure your custom group is selected and click Assign Security Groups
• Then, for your custom security group, make sure it has the following rules
  • Go to VPC Console > Security Groups > Select your custom security group > Click on Inbound Rules (at the bottom)
  • Copy that security groups groupId (sg-{something}), and then click Edit in the Inbound rules section
    • First, add a rule for Postgres > Select PostgreSQL from Type, and add your groupID in the source field (docs). Click Add another rule
    • Then, add a rule for connecting to Redis > Select Custom TCP Rule from Type, and add your groupID in the source field. Click Add another rule
    • Finally, add a rule to SSH into your VPC to access your EC2 app > Select SSH from Type, and in the source add your own public IP address (docs). Click Save

Test it out

• First, let's SSH into your EC2 instance (docs)
  • First, find the public IP address for your EC2 instance (Listed under Public IP on the EC2 Instance Dashboard)
  • Open a console, add type the command ssh -i {path to key (without .pub)} ec2-user@{EC2 public IP}
  • It should work!
• Once you are in your EC2 instance, connect to your DB
  • run sudo yum install postgresql-server to install postgres
  • Find your RDS Endpoint > Go to RDS Console > Instances > Select Instance > Find the Endpoint field > Get the endpoint without the port tacked on the end
  • run psql -h {Endpoint (Host)} -p {Port (should be 5432)} -U {username} -W {dbname} (ref)
  • It should prompt you for your password. After you provide that you should be in! Type \q to get out
• Now connect to your Redis (docs)
  • Type sudo yum install telnet to install telnet
  • Find your Elasticache Endpoint > Go to Elasticache Console > Replication Groups > Select Group > Scroll down to Primary Endpoint > Get the endpoint without the port tacked on the end (also, see docs)
  • In your console within your EC2 instance (SSHed in), type telnet {Elasticache Endpoint} {Port (should be 6379)}
  • You can type KEYS * to verify it worked, and then quit to exit
• Type exit to exit from the SSH

Configure your environment

First, open up your config/environments/production.js file

• Enter in your database host (make sure to strip port of the end), port, username (user), password, and the name of your database
• Enter in your redis host (Endpoint) (strip port) and redisPrefix. The redisPrefix should be {ember app name}:index:
• Enter your indexFileTableName
• Enter in the mailchimp apiKey, mailingListId, categoryId, registeredUserId, and mailingListId
• Enter in your gmail information for nodemailer
• And fill in the other fields with the desired configuration (jwtSecret and sessionSecret can be any secret string) Then, in your Ember App, be sure to fill in the necessary information for deployment in config/env-vars.js
• appName should match your ember app name
• accessKeyId is the accessKeyId that is pegged to your IAM user
• secretAccessKey is the secretAccessKEy that is pegged to your IAM user
• s3Bucket is the name of your S3 bucket that you created
• s3Region is the name of the region your s3 is located in
• cdnUrl is the prefix that will be prepended to static assets. One way to find this is to add a file to your s3 bucket, open it, and see the url that is listed before the asset. In general, it follows the form https://s3-{s3 region}.amazonaws.com/{bucket name}/{app name}/
• Fill in your postgres information for dev and production. This is the poor man's version of the lightning deploy - using postgres as a key value store instead of redis.
• tunnelConfig is used to ssh into the ec2 instance, and connect to redis from there to deploy index.html
  • username should be 'ec2-user'
  • host should be the elasticbeanstalk ec2 instance public IP address
  • privateKeyPath should be the path in your system to your private key to ssh into the ec2 instance
  • dstHost should be the Endpoint for your RDS instance (if using the postgres deploy) or the Endpoint for your elasticache redis instance (if using redis)
  • If using the postgres deploy, the port should be 5432, with redis remove this field

Deploy the app!

First, go through the ember README and follow the instructions to install node modules and bower components

• Then, in your ember app directory, just ember deploy 'prod' and if all the configuration is set it should just work
  • Follow the instructions in response to activate the revision, ember deploy:activate prod --revision={revision hash}
  • You can verify it did by connecting to your Redis instance, typing KEYS *, and verifying that the redis key for your revision is there
  • Also check to make sure it dumped static assets in your s3 bucket - it should be there under {app name}
  • Once your app is in s3, right click on it and select make public, so that it can serve it's assets to your application
• Before deploying with elastic beanstalk, first go to your elastic beanstalk app Services (upper left) > Elastic Beanstalk > Configuration (left) > Software Configuration (click the gear) > scroll to the bottom, and under environment properties, type NODE_ENV under Property Name and production under Property Value > and click the + and apply
  • This sets your environment variables to use production
• In your node app directory, type eb deploy (make sure you've committed all changes to git first)
  • Once the app is done deploying, it should be working! type eb open to open the app

Note on environment config

For both the Node and Ember apps, there are environment configuration variables that are set within the development.js, test.js, and production.js (and env-vars.js) files. These are obviously sensitive, and should be disseminated with care. Do not commit these to a public repo!

Disclaimer

I do not claim to be an expert programmer, and I am sure there are many bugs and flaws in this application. As you notice them, please open issues and point them out to me, so that we can keep improving it. Note in particular any security flaws that you see, please make sure we can address them. Thanks!

Contributing

I would love for you to contribute, suggestions, PR's, everything is welcome to make this an awesome boilerplate app

License

MIT