Problems with Environment Variables

Problem 1: Lack of Complex Values

Because environment variables can only be strings, even something as simple as a flag in your code means that you have to jump through conversion gymnastics every time it's used.

if ENV['MY_NEW_FEATURE_ENABLED'] == 'true'
  # My feature stuff
end

Alternatively, we could check for truthiness:

if ENV['MY_NEW_FEATURE_ENABLED']
  # My feature stuff
end

and that works great if it's true, but what if we want to set it to false? Since, in this case, the only way we can set an environment variable to false is if it's not set, then how do we know:

• it was intentionally left blank to demonstrate false
• it was intentionally left blank because it didn't need to be set
• it was unintentionally left blank due to an oversight

So either a) we're doing conversions on every check, or b) we have ambiguity on what an unset environment variable really means.

Solution 1

Chamber gives you YAML data types in your settings:

• Strings
• Integers
• Floats
• Booleans
• Arrays
• Hashes
• Dates
• Null
• Regexes
• Symbols

plus environment variable coercions for when you absolutely have to use an environment variable to override your YAML settings.

So the above example turns into:

if Chamber.dig!('my_feature', 'enabled')
  # My feature stuff
end

Problem 2: Organization

Even when working with a small-sized application, being able to organize your settings is important. Take a look at the number of settings that I had in one of my applications after just a couple weeks. We've only just begun working on this app and already it's hard to look at.

Not only that, but keeping multiple versions of that list (one for each environment and/or deploy and/or host) may be "scalable" (as 12-factor states it), but it's not very manageable.

Solution 2

Chamber lets you organize your settings in multiple ways:

Single File with Nesting

In this (the simplest) case, you can nest your settings as deep as you'd like so that different sections are easy to scan.

# settings.yml

smtp:
  username: "my_username"
  password: "my_password"

Single File with Namespaces

As your app grows, you may find the need to set settings differently based on things like environment or host. Chamber lets you easily do that by nesting your settings under the namespace in which it applies.

# settings.yml

development:
  smtp:
    username: my_dev_username
    password: my_dev_password
 
test:
  smtp:
    username: my_test_username
    password: my_test_password
 
staging:
  smtp:
    username: my_staging_username
    password: my_staging_password

Multiple Files with Namespace

If you'd like to completely separate out a namespace into other files, you can do that by appending the namespace to your filename. This allows you to not only manage them separately, but also gitignore settings for an entire namespace.

# settings.yml

development:
  smtp:
    username: my_dev_username
    password: my_dev_password
 
test:
  smtp:
    username: my_test_username
    password: my_test_password
 
staging:
  smtp:
    username: my_staging_username
    password: my_staging_password

# settings-production.yml
 
smtp:
  username: my_prod_username
  password: my_prod_password

Different Files By Type

Lastly, Chamber lets you split out arbitrary settings into multiple files. Generally as your app grows, what you tend toward is a different YAML file per type of setting. For example, if our settings are getting rather lengthy, we can split out all of the SMTP settings into their own SMTP YAML files:

# settings/smtp.yml

development:
  smtp:
    username: my_dev_username
    password: my_dev_password
 
test:
  smtp:
    username: my_test_username
    password: my_test_password
 
staging:
  smtp:
    username: my_staging_username
    password: my_staging_password

# settings/smtp-production.yml
 
smtp:
  username: my_prod_username
  password: my_prod_password

Problem 3: Sharing Settings Between Namespaces

If only using environment variables, sharing common settings information between environments can be a pain.

When creating a new environment (or onboarding a new developer) you have to duplicate all settings from every environment. This is the "combinatorial explosion" that was mentioned in the 12-factor manifesto.

# .env-development

EBAY_AFFILIATE_ACCOUNT_ID="my affiliate account id"
EBAY_AFFILIATE_CAMPAIGN_ID="my affiliate campaign id"
EBAY_AFFILIATE_PARTNER_CODE="9"
EBAY_AFFILIATE_USER_ID="my affiliate user id"
EBAY_API_URL="https://api.sandbox.ebay.com/ws/api.dll"
EBAY_CATEGORY_CALLS_PER_HOUR="60"
EBAY_CERT_ID="my dev cert id"
EBAY_DEV_ID="my dev dev id"
EBAY_ENVIRONMENT="sandbox"
EBAY_LIMIT_API_CALLS="true"
EBAY_SITE_ID="0"

# .env-production

EBAY_AFFILIATE_ACCOUNT_ID="my affiliate account id"
EBAY_AFFILIATE_CAMPAIGN_ID="my affiliate campaign id"
EBAY_AFFILIATE_PARTNER_CODE="9"
EBAY_AFFILIATE_USER_ID="my affiliate user id"
EBAY_API_URL="https://api.ebay.com/ws/api.dll"
EBAY_CATEGORY_CALLS_PER_HOUR="720"
EBAY_CERT_ID="my prod cert id"
EBAY_DEV_ID="my prod dev id"
EBAY_ENVIRONMENT="production"
EBAY_LIMIT_API_CALLS="false"
EBAY_SITE_ID="0"

When using environment variables to store settings, you have some options which are different:

# .env-development

EBAY_API_URL="https://api.sandbox.ebay.com/ws/api.dll"
EBAY_CATEGORY_CALLS_PER_HOUR="60"
EBAY_CERT_ID="my dev cert id"
EBAY_DEV_ID="my dev dev id"
EBAY_ENVIRONMENT="sandbox"
EBAY_LIMIT_API_CALLS="true"

# .env-production

EBAY_API_URL="https://api.ebay.com/ws/api.dll"
EBAY_CATEGORY_CALLS_PER_HOUR="720"
EBAY_CERT_ID="my prod cert id"
EBAY_DEV_ID="my prod dev id"
EBAY_ENVIRONMENT="production"
EBAY_LIMIT_API_CALLS="false"

And you have other options, which are the same:

# .env-development

EBAY_AFFILIATE_ACCOUNT_ID="my affiliate account id"
EBAY_AFFILIATE_CAMPAIGN_ID="my affiliate campaign id"
EBAY_AFFILIATE_PARTNER_CODE="9"
EBAY_AFFILIATE_USER_ID="my affiliate user id"
EBAY_SITE_ID="0"

# .env-production

EBAY_AFFILIATE_ACCOUNT_ID="my affiliate account id"
EBAY_AFFILIATE_CAMPAIGN_ID="my affiliate campaign id"
EBAY_AFFILIATE_PARTNER_CODE="9"
EBAY_AFFILIATE_USER_ID="my affiliate user id"
EBAY_SITE_ID="0"

So if some of the options are the same, why aren't we just hard-coding them?

Why Not To Hard-Code Common Options

They Can Still Be Sensitive

The EBAY_AFFILIATE_ACCOUNT_ID is the same between development and production, but it's still something I want to be secret.

DRYing Things Up

And even if it's not secret, it doesn't mean that I'm not (or won't in the future) use it in more than one place.

One Source of Truth

And even if I'm only using it in one spot, I don't want to have multiple places that I have to look for settings for a given thing.

I don't want to have to look in (for example) a Rails initializer as well as my YAML files.

Solution 3

Because Chamber uses YAML, instead of this:

development:
  smtp:
    server:          "smtp.sendgrid.net"
    port:            587
    tls:             true
    authentication:  "login"
    delivery_method: "smtp"
    username:        "my_dev_username"
    password:        "my_dev_password"

test:
  smtp:
    server:          "smtp.sendgrid.net"
    port:            587
    tls:             true
    authentication:  "login"
    delivery_method: "smtp"
    username:        "my_test_username"
    password:        "my_test_password"

production:
  smtp:
    server:          "smtp.sendgrid.net"
    port:            587
    tls:             true
    authentication:  "login"
    delivery_method: "smtp"
    username:        "my_prod_username"
    password:        "my_prod_password"

We would use YAML's anchors and references to do something like this:

defaults:
  smtp: &defaults
    server:          "smtp.sendgrid.net"
    port:            587
    tls:             true
    authentication:  "login"
    delivery_method: "smtp"

development:
  smtp:
    <<: *defaults
    username:        "my_dev_username"
    password:        "my_dev_password"

test:
  smtp:
    <<: *defaults
    delivery_method: "test"
    username:        "my_test_username"
    password:        "my_test_password"

production:
  smtp:
    <<: *defaults
    username:        "my_prod_username"
    password:        "my_prod_password"

Here, by utilizing YAML's anchors and references to use portions of settings in multiple places. We're also able to set a default for delivery_method and just override it in the one environment where we need something different.

Notice how much easier it is to see the things that are similar (and different) between each environment, than if we were looking at simply a list of environment variables.

# .env-development

SMTP_SERVER="smtp.sendgrid.net"
SMTP_PORT="587"
SMTP_TLS="true"
SMTP_AUTHENTICATION="login"
SMTP_DELIVERY_METHOD="smtp"
SMTP_USERNAME="my_dev_username"
SMTP_PASSWORD="my_dev_password"

# .env-test

SMTP_SERVER="smtp.sendgrid.net"
SMTP_PORT="587"
SMTP_TLS="true"
SMTP_AUTHENTICATION="login"
SMTP_DELIVERY_METHOD="test"
SMTP_USERNAME="my_test_username"
SMTP_PASSWORD="my_test_password"

# .env-production

SMTP_SERVER="smtp.sendgrid.net"
SMTP_PORT="587"
SMTP_TLS="true"
SMTP_AUTHENTICATION="login"
SMTP_DELIVERY_METHOD="smtp"
SMTP_USERNAME="my_prod_username"
SMTP_PASSWORD="my_prod_password"

Problem 4: Syncing Settings Changes

Whether it's from Jane to Joe, or Sam to Victor, or Gloria to Gayle, making sure settings stay in sync between the development team can be a pain. The more developers you have, the more people need to have the settings synced between them... talk about a "combinatorial explosion".

And lastly, what about the environment-specific files like .env-production and .env-staging? If a developer makes a change to those, how do they get synced to the rest of the team?

It's a hard problem. Generally one of three things happens:

Nothing

Some teams are moving "too fast" or are small enough that they just deal with this pain.

Dropbox / Google Drive

Some teams set up a Dropbox or Google Drive shared folder and then symlink the settings files from that share into their project.

But neither of these solutions is designed with developers in mind. As well as the fact that if you're not into checking in your settings, how is this any different?

Separate Git Repository

Some teams will choose to create a completely separate git repository. This repository will contain only the settings for their app. Then, like with Dropbox, files will be symlinked into the project directory.

This is a better solution, but is still a pain. And again, for those who don't ever want to check sensitive settings into the repository, even this won't work.

Solution 4

Chamber uses Public Key Cryptography to allow every setting to be committable to the repository. Some settings, which are not sensitive, can be committed as plain text. Other settings, which are, can be encrypted with a project-wide private key.

But what about decrypting? Do you have to decrypt every time you need to use a setting?

Absolutely not. Here are the steps:

chamber init

Running:

chamber init

will generate a public/private key pair for your project (this only need to be done once).

Add _secure_ Prefixes to Settings

Any settings you'd like to be encrypted need to have a _secure_ prefix in front of their names.

defaults:
  smtp: &defaults
    server:           "smtp.sendgrid.net"
    port:             587
    tls:              true
    authentication:   "login"
    delivery_method:  "smtp"

# ...Snip...

production:
  smtp:
    <<: *defaults
    _secure_username: "my_prod_username"
    _secure_password: "my_prod_password"

chamber secure

Running chamber secure will use your private key, encrypt all your secure settings and update your file to look something like this:

defaults:
  smtp: &defaults
    server:           "smtp.sendgrid.net"
    port:             587
    tls:              true
    authentication:   "login"
    delivery_method:  "smtp"

# ...Snip...

production:
  smtp:
    <<: *defaults
    _secure_username: wVOqkGbZWfzgLINJlXtjg8H9k3q/SxbNQtgYqEy5uLp7FRbsM79MeIsoStEiv/ZqbpXZQCVmtvFAhlCAT77l1HpGTBFWd9WFIjhO0t0GoI+ewYTug21kOJI18I4zzZUPnsnm9Ml5vjRPleHsI745HhJ8lGHc3bHcGlIit34/KT6vmKdqksbMne6kUCHxt472S92OIhrkx6cY8uHj6Cs8Q3tb3JcfsVgjNAaBzpxaUw8MA8OEEk8vr6YUkgGT3k55GXzM58atwSvGxHTzbCnNJsfqY0Dqj4uUQo07OaIAYXbJ7jn43brhQEq30EoniZ2RFXjgHbQCUmsM+Rskh/l8LA==
    _secure_password: reHGp1YKoAW7nD64szmKbDo2hLi69G4Upi9DlcYYNJ8BsztzooDFQfUfOWcWFw//3wQ2DbC3ff8ZE2M3Ym8W/K4dFxdC+YftNB5hQpwjlW+d2SPqL081MMyb4/wWJNefIQ2sN7Xj0Y7MQcgzJ9ctQv6qB93GbgzbdI3e/cKl2r4ON+JfvdTQ4KA9Ik2FRhpJuQk26OAr+2jr+4yTpNSAf+2y24TodRSO2TiPHoy6LJWEfAjpvvdl26R+uK7oqtrXjdlss1WqPfrmLyXpODicxhiO1LWOyESfD7V3ta1TS9PtcACbw1vI7zyofjDJwGZtAeoKoXRgj9qJfL7CcLR+SA==

But when you want to access your settings, all you have to do is reference the key (without the _secure_ prefix) and it'll decrypt it for you automatically.

Chamber.dig!('smtp.username')
# => "my_prod_username"

Problem 5: Keeping Environments In Sync

Even if you'd managed to solve the above problem without Chamber, you still have another issue. When you change your development settings and (for example) added a setting. If this setting is required in production, it's quite easy for a developer to forget to set it.

This can cause production to go down.

Solution 5: Part 1

Chamber provides a way to be able to set one environment variable in each environment one time and each time the app is deployed, all the settings that existed on the developer's machine will exist on the environment. No syncing required.

Solution 5: Part 2

But even with the above tools, you still need to remember to run it and that really only needs to happen if a YAML file changes. And even then, you're always running the same comparisons (eg comparing development to test to production).

Chamber provides a convenient solution in the form of a git pre-commit hook.

This hook will run chamber compare against a configurable set of namespaces any time a settings file is changed. If the settings don't match, the commit will abort.

Additionally, since Chamber 3.0, any attempts to access to non-existent key will result in a runtime error making any missed misconfigurations easy to spot.

Problem 6: Inconsistent Local Settings

Because your settings aren't supposed to be checked in, the standard practice is to have something like a .env-sample which is checked in. This file typically has development settings as they are considered not to be sensitive. Nevermind the fact that this is not a correct assumption, it still poses another problem.

Even if you put .env-sample in your repo, nothing forces your developers to remember that when that file changes, they need to update their local copies. And on the flipside, there's nothing to make a developer remember that when they add something locally, they need to remember to add it to .env-sample.

So let's say we have a fresh checkout. At which point we do:

cp .env-sample .env

# .env-sample

OTHER_EXISTING_SETTINGS="here"
COMBINATION_TO_MY_LUGGAGE="12345"

# .env

OTHER_EXISTING_SETTINGS="here"
COMBINATION_TO_MY_LUGGAGE="12345"

Now assuming that no changes need to be made to this file (another typically incorrect assumption) to get it to run locally, we now have two identical files.

But at some point we need to change a value locally. That's reasonable:

# .env-sample

OTHER_EXISTING_SETTINGS="here"
COMBINATION_TO_MY_LUGGAGE="12345"

# .env

- OTHER_EXISTING_SETTINGS="here"
+ OTHER_EXISTING_SETTINGS="somewhere"
  COMBINATION_TO_MY_LUGGAGE="12345"

The next thing that happens is that another developer comes along and adds a variable which is required for a feature they're working on.

# .env-sample

  OTHER_EXISTING_SETTINGS="here"
  COMBINATION_TO_MY_LUGGAGE="12345"
+ NEW_FEATURE_SETTING="so sweet"

# .env

OTHER_EXISTING_SETTINGS="somewhere"
COMBINATION_TO_MY_LUGGAGE="12345"

Now, on our next pull, we need to notice that the .env-sample file changed and manually merge in the changes.

It'd be great if we could just say:

Give me everything in the shared development settings and then override specific settings with my local values.

Solution 6

Chamber's namespaces allow you to have host-based settings which are based on your hostname. Chamber will load everything for your environment, then, if it sees settings for your host, it will override the environment settings with the host settings.

Problem 7: Tracking Settings Changes

When you make changes to your settings, those are as important as changes to your code. It seems odd to me that we make a big deal about writing good commit messages so that we can communicate effectively with the rest of the development team (or ourselves in 6 months) yet the reason we had to change the SMTP host name in production is unimportant enough that it doesn't need to be tracked.

Solution 7

Since any developer can commit secured settings (because the public key is committed to the repository) and because those secured settings can be committed to the repository, then good commit message can be written regarding the why and how of the change.