Compiled config vs runtime config

[mars]

💚 The custom environment variables feature.

---

Update February 2017: We devised a solution for runtime config on Heroku. create-react-app-buildpack now supports runtime environment variables by way of injecting values into the javascript bundle at runtime. This is still an open question for other runtimes and deployment techniques.

---

This feature leverages Webpack's DefinePlugin which is explained as:

…global constants which can be configured at compile time.

The issue is not all configuration should be compiled into an application.

Things that are stable between environments (dev, staging, production) of an app make sense to compile-in:

• version number
• commit sha/number
• browser support flags

Things that change between environments would be better provided dynamically from the current runtime:

• URLs of an APIs (may change for each environment)
• secret tokens (may change for each user or request)

Ideally a bundle could be tested in CI, then promoted through environments to production, but compiling all of these values into the bundle means that every promotion requires rebuild.

I've used Node to serve single-page apps in the past with a base HTML template that gets environment variables set via inline <script> at runtime; even used a server-side Redux reducer to capture process.env values into an initial state.

Now, I'm trying to find a good solution to the conflation of runtime configuration values being compiled into the bundle.

create-react-app is clearly focused on producing a bundle that can be used anywhere.

Is this project open to solutions/hooks for injecting runtime variables, or is this wandering into npm run eject territory?

[gaearon]

An ad hoc solution would be to put some placeholder into HTML and then replace it with a script tag initializing a global before serving the app. This is however outside of CRA concerns IMO.

[glenngillen]

I got directed to this thread via Twitter, so need to qualify my input with I've limited experience re best practice for React & single page apps. My interest is around 12factor/Heroku specific aspects.

So looking at the problem as it pertains to Heroku implementation of 12factor, the issue is really that the notion of a slug and release end up conflated for single page apps because they environment is baked in at build/compile time rather than runtime. I understand the appeal in doing that, but I wonder what the options are for trying to avoid it.

An ad hoc solution would be to put some placeholder into HTML and then replace it with a script tag initializing a global before serving the app

Could that be encapsulated into some higher order and reusable concept like process.env on the server side? If we can converge on a single object to return the values as a best practice, then it seems the problem reduces down to "where does that object get the values from?". To which I'd suggest the following as an order of preference, which could be pluggable/configured by the developer:

• Fetched from a another service
  • Pros: Can actually get runtime specific values at runtime. Changes are reflected immediately. No need to rebuild for each environment. No need to rebuild for each value change
  • Cons: Another thing to run/use. If you're running it yourself now potentially need to run a server somewhere to lookup & return these values.
• All values stored in nested object hierarchy, runtime is able to determine which one to use
  • Pros: No need to rebuild for each environment.
  • Cons: Need to rebuild for each value change. Values for all environments are exposed in source to client, is that a potential security issue?
• Values interpolated in at build time:
  • Pros: Client only exposes what is relevant for a given environment
  • Cons: Need to rebuild for each environment. Need to whenever a value changes.

[tibdex]

The Twelve-Factor App guide mentioned by @mars has a section about "build, release, run" cycle. It explains that the build stage is only about creating an executable that could run anywhere. It is the release stage that should be responsible for combining this executable with the the deployment environment configuration.

create-react-app is really close to being able to generate builds deployable anywhere!

Indeed, if merged, this PR #1489 will relieve apps not using the HTML5 pushState history API to specify the URL on which they will be deployed.

Then, the only remaining task to make create-react-app builds compliant with the Twelve-Factor App recommendations is to let the deployment environment specify some part of the configuration. In fact, create-react-app documents how to use environment variables but they only work at compile time.

One of the solutions to add support for runtime environment variables that is really simple and lightweight would be to add an env.js file inside the public directory and load it in a script tag in index.html just before the main.js bundle.

env.js could have the following content:

// This file will not end up inside the main application JavaScript bundle.
// Instead, it will simply be copied inside the build folder.
// The generated "index.html" will require it just before this main bundle.
// You can thus use it to define some environment variables that will
// be made available synchronously in all your JS modules under "src". 
//
// Warning: this file will not be transpiled by Babel and cannot contain
// any syntax that is not yet supported by your targeted browsers.

window.env = {
  // This option can be retrieved in "src/index.js" with "window.env.API_URL".
  API_URL: 'http://localhost:9090'
};

The body of index.html would be:

<body>
    <div id="root"></div>
    <!--
      Load "env.js" script before the main JavaScript bundle.
      You can use it to define "runtime" environment variables.
      Please open this file for more details.
    -->
    <script src="%PUBLIC_URL%/env.js"></script>
  </body>

After running npm run build, you would find your env.js file untounched under build/env.js. You would then be able to edit this file manually directly on the deployment machine or through a small bash script to change API_URL to https://api.my-app.com for instance.

Should I submit a PR documenting this technique or even adding this env.js file to the public folder of the boilerplate ?

[jayhuang75]

@tibdex I like this solution, but did you test via the npm run test with the jest. seems will have some error and it can not get in the test level. any advice?

[mars]

Writing configuration values into a file is antithetical to 12-factor. Instead, always use runtime environment variables for configuration.

create-react-app [CRA] does not provide a production runtime. It only builds a static bundle. Runtime environment is not something solvable by CRA, unless it becomes prescriptive about deployment which is unlikely given its wide range of applications.

Solving this problem is about finding a way to set those variable values in the React app at runtime instead of buildtime. I know of two ways to accomplish this:

1. Render runtime environment values into global variables in a <script> element in index.html with a Node/Express server. Then, use those globals within your React app. (This would need to be resolved for local development too.)
2. Deploy to Heroku using create-react-app-buildpack which supports runtime environment variables by way of injecting values into the javascript bundle at runtime.

This is a challenging topic 🤓 Bravo for trying to solve it 👏👏👏

[glenngillen]

I've been working on a somewhat related effort to manage the configuration/credentials at runtime problem. And I'd be interested in helping come up with a standard approach of doing it for SPAs like this. The basic approach is:

• Expose a single key in build phase that can be included in the build in some fashion
• Have a client in the runtime that can use that key to query an API (in this case https://api.voltos.io), and receive back whatever config the runtime needs
• Expose that additional config in a way that can be re-used by the client/rest of the SPA

If we can reach agreement on what the API response should look like, how to define what the env var API host/url should be, and how/where it should be exposed I'd be happy to do the heavy lifting and try to write a shim. I can foresee some API-like wrapper that lets people store their secrets direct on S3 and the API just exposes them in the right format.

What do y'all think?

[mars]

@glenngillen from my perspective, configuration should be in-place when the javascript app loads in the browser. Requiring remote fetch of config values from an external service:

• taxes performance (how does the app render before those config values are present?)
• increases complexity (how do you account for the async config dependency in any given javascript app?)
• decreases reliability (what happen if the config service is inaccessible?)

The fundamental problem is that SPAs can be deployed many many different ways: static web sites, GitHub sites, instant deploy services (Heroku, Now, Netlify, etc), embedded in blogs (Wordpress, Drupal), composed with web app servers (Node, Rails, Django, etc), and the runtime is where this problem must inevitably be solved.

[glenngillen]

@mars agreed, but... as you've mentioned "runtime" in this context includes things like static web/hosting directly off s3 and GitHub pages. Short of fetching config on demand I see no alternative for these apps other than pushing the config back into the build/compile phase.

[jayhuang75]

@tibdex Finally I get the unit test code fixed. Basically we have to mock the window.env in the jest mock folder
@mars Thanks for the encourage! :) and our use case, its build once deploy everywhere with the docker image to the PAAS. Once the docker image deploy to the PAAS, we can just change the environment variable to point the different API EndPoint.
Disclaimer: we just export the public accessible API endpoint. If you want to add your secret key, etc. please DO NOT use the solution.

[tibdex]

@jayhuang75, I think a better place to provide the configuration values to the tests might be src/setupTests.js.

@mars, I agree that ideally the config should not be stored in a file. The env.js file could however be generated by the server on which the static bundle is deployed. The server would then be able to use it's own OS environment variables to generate this file. I believe it's a good and simple way to solve runtime config if you have control over the server ;)

[mars]

@tibdex your technique of adding a <script src> include to index.html and separately writing an env.js file during deployment seems sound for the purely static use-case. Preserving the local dev & testing experience is going to be the challenge.

Perhaps using a module-based approach like I did for Heroku runtime env would allow you to capture logic for dev/test vs production runtime and graceful fallback/error-messaging when the env is not setup correctly?

[ivosabev]

I am having the same problem. I need to change a the index.html based on a dynamic value from the database. This is easy to do when serving a prebuilt CRA via NodeJS with a simple string search and replace, but becomes impossible in dev mode without gutting the CRA inside out. The more I think about it I do not see an easy way for doing this dynamically in both environments. Any plans to support this or any ideas how to implement it with the least CRA changes possible?

[rmoorman]

@tibdex you mean something along the lines of this (regarding the server on which the static bundle is deployed)? Seems to be working allright :-)

const express = require('express');
const path = require('path');
const serialize = require('serialize-javascript');

const env = {
  'API_URL': process.env.API_URL,
  'DYNO': process.env.DYNO || 'Not running on a dyno',
}

const app = express();

app.use(express.static(path.join(__dirname, 'build')));

app.get('/env.js', function (req, res) {
  res.set('Content-Type', 'application/javascript');
  res.send('var env = ' + serialize(env));
});

app.get('/*', function (req, res) {
  res.sendFile(path.join(__dirname, 'build', 'index.html'));
});

app.listen(process.env.PORT);

[tibdex]

@rmoorman this would work indeed when you use a custom express app to serve your create-react-app build directory. But the technique I suggested in #578 (comment) was server agnostic and would work with Caddy, nginx or even serve since it only requires serving static files.

[rmoorman]

@tibdex thank you for the clarification. The small example is indeed meant for the case when one can (and wants to) use a custom express app for serving the static files (the same could also be achieved using something else for the app part of course).
This can indeed also be handled at container/app startup by writing to the env.js file :-)

[neverfox]

@tibdex Indeed it would but it wouldn't be 12-factor, if that's important to someone, because it is technically using a static config file. In other words, it solves the problem of moving the config out of the build, but it doesn't solve the problem of injecting the config from the environment. That is going to require a system like Node that can read from the environment (maybe there are ways to do this in NGINX, but I'm not sure).

[pbouzakis]

@neverfox or for anyone else on this thread....

Maybe we need to take a step back in regards to why a config file is not recommended for 12-factor. I could be wrong, but my understanding is that they are referring to a server side environment. Let's review the goals:

The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

If you create a service say on a subdomain (meaning it doesn't have to be hosted with the static build directory), and it all does it serve responses for the environment config (/env.js) AND that server handler simply looks up configuration from environment variables, I feel you are achieving the goals mentioned above.

What's important is the server side is not using a config file. This allows the server to easily change settings w/o relying on source code change / new deploy. There are also no checked in environment configs in the repo.

From the client side you wouldn't be able know the difference.

IMO, it passes the 12-factor app guidelines.

My only leftover concern is relying on global variables. It's possibly for other 3rd party scripts to be on your page that could alter these values (This might be an issue for some who use tag manager scripts that load more 3rd party scripts onto your site).

You could also use something Content-Security-Policy to whitelist domains so you only 3rd parties you trust, or you use some sort of iframe solution to sandbox.

[gaearon]

I'll close as I don't see an actionable item for us here. If you disagree please file a new issue.

I feel like a part of this discussion is a bit too philosophical. We shouldn't need to blindly apply the guidelines somebody else wrote, but to think critically. Building client-side apps is not the same problem as building server-side apps, and some invariants are just not true (e.g. there are no "secrets" in client-side apps).

As for injecting runtime variables, I feel like doing this via some sort of templating is reasonable and I think it falls out of scope of CRA itself which isn't concerned with what you do after deployment.

[n-sviridenko]

@gaearon regarding this comment #578 (comment), I'm agree to do this to manage environment variables, but it won't work for the PUBLIC_URL. Environments, at least qa/staging and production, have different URL's, so it forces to bundle the application 2+ times only to have different public url's. Otherwise, with "hoc solution", only adding a <script /> tag won't help, cause it's also present in .js files. Is there any other suggestions?

[gaearon]

@n-sviridenko

I don't see what else you could do.

The tradeoff is:

• Either you bundle twice, and the HTML refers to an absolute path (which is what you want for client-side apps).
• Or you bundle once, but the HTML refers to a relative path (which is usually not what you want for client-side apps).

Option 1 is default in CRA.

Option 2 is what you get if you specify "homepage": ".".

What am I missing? Is there a third option here?

[n-sviridenko]

@gaearon you're right. But the javascript part can depend on a global variable which can be easily overridden in a higher level instead of replacing process.env.PUBLIC_URL to hardcoded strings. The html part can be manually modified every time before serving the application. I think, a nice solution can be replacing process.env.<name> to global.<some prefix>_<name> || <value of process.env.<name>> where name is REACT_APP_* or PUBLIC_URL when bundling the application.

In other languages, in most of the cases, you always can pass environment variables in runtime.

[mobimation]

I settled for a solution based on this post by @tibdex where in my case a script

• runs a production build
• replaces the build folder index.html with a new version that runs Javascript 'config code' before app launch. The resulting build folder is deployed as a tar file to sites. In my case pre-launch code consists of a config.js that imports a file env.js containing server URLs that is post edited per deployment site with the aim to only distribute one production bundle for use on multiple sites and let site administrators edit that config file for actual server URLs on their site.

config.js stores the URL settings in browser sessionStorage, see w3.schools doc making the values accessible globally by Javascript throughout the app once launched, with browser storage lifetime for the current session.
I was not able to get the @tibdex suggested window.env storage to work, likely because the app isn't at all launched at time of configuring. So sessionStorage came handy.

[forrest-rival]

@tibdex curious if you know of an equivalent solution for react-native (no index.html)? I'm using react-native-web, so the script imported config works for the web build, but wouldn't be viable for native builds.

[tibdex]

Nope, I have never used react-native 😏

[jamesmfriedman]

I landed on the exact solution above about having config variables written into a dynamic /environment.js file. One addition I made is to patch these onto webpacks process.env at runtime so that the config can still be part of the build and simply overwritten be remote configuration.

/**
 * Remote configuration can be attached from window.process.env
 * Take the remote config and patch it onto webpacks process.env
 * This is a runtime side effect that should be run before anything else
 */
const patchLocalConfig = (
  localConfig: { [key: string]: ?string },
  remoteConfig: ?{ [key: string]: ?string },
  debug?: boolean,
) => {
  if (!remoteConfig) {
    if (debug) {
      console.warn(
        'No remote config found. Set window.process.env to an object to apply a remote configuration.',
      );
    }

    return;
  }

  if (debug) {
    console.log('Remote Config:', JSON.parse(JSON.stringify(remoteConfig)));
    console.log('Local Config:', JSON.parse(JSON.stringify(localConfig)));
  }

  Object.entries(remoteConfig).forEach(([key, value]) => {
    if (!(key in localConfig)) {
      throw new Error(
        `Remote configuration var '${key}' was not found in local configuration.`,
      );
    } else if (
      typeof value === 'string' ||
      value === null ||
      value === undefined
    ) {
      localConfig[key] = value;
    }
  });

  if (debug) {
    console.log('Merged Config:', JSON.parse(JSON.stringify(localConfig)));
  }
};

And to use it, just make sure to run it at the top of your main entry point before the rest of your code runs.

const remoteConfig =
    'process' in window &&
    window.process.env &&
    typeof window.process.env === 'object'
      ? window.process.env
      : null;
  patchLocalConfig(
    process.env,
    remoteConfig,
    true,
  );

[FahdW]

@rmoorman Do you see any issues with using essentially an endpoint for that env.js? Since it is loading it async but once its loaded it exists through the entire app right?

[rmoorman]

@FahdW what do you mean with endpoint? An http handler which serves up the env.js or a webpack entrypoint? For what it's worth, I wouldn't want to run webpack during container startup. In my opinion the webpack build process belongs to the docker build phase of a container (also running a container should be rather instantaneous and not involve a build process potentionally taking ages to complete and consuming a lot of RAM).

As mentioned before, I would indeed opt for an env.js file either served dynamically (just because it is easy enough) through some http server where you can code up the handlers yourself (e.g. something something nodejs/express, python/flask ... pick your poison).
Or one could have the docker entry point write the env.js file based on the container's env vars at startup, which is a bit more efficient of course as the env.js file would be built only once and served just like the rest of the app's files (and env vars won't change anyway during container runtime).
I don't see many drawbacks actually (it's more like ... required complexity for the app implementation at hand). Of course a dynamic http handler thing could also break, but so could the env.js writing script for example. Do you prefer something like nginx to serve all app files? Or do you need a scriptable webserver within the container anyway? It's all about tradeoffs. Pick what feels better for your use case. In case it does not work out as expected, try something else.

But I cannot judge what would be the best fit for your situation. Generating the env.js file dynamically can also be of use in case some configuration for the react app would change dynamically and independently from the docker container's environment (could also be handled by overwriting a static env.js file at runtime to some extend of course).

Either way, the env.js file is to be loaded along with the app itself (through index.html), providing configuration for the app. And since the env.js is basically assigning things to the global window object, your whole app can access it from there.

Lastly, if there are a lot of dynamic configuration options to consider for your app, or they change a lot through the apps lifetime within a browser session, it could also be a good thing to have some (REST/graphQL/whatever) API satisfy the react app's needs.

[FahdW]

@rmoorman I was wondering more along the lines of you generate your express env.js endpoint. Meaning your webapp needs to read from it, isn't this read async? Does it only happen once, i am just trying to understand the ramifications of serving env.js. So far i have not seen any issues with my webapp getting the values, but just wanted to know of any potential side effects that may occur due to fetching env.js from the express server that our webapp is deployed with. I believe the env.js is an async action but once the webapp responds with it, it persists forever?

[forrest-rival]

@FahdW I have this setup right now; this is how it works. When releasing an artifact to an environment, we create the env.json from values managed in AWS Parameter Store and deploy it sibling to the app (served from S3). You're right that requesting the config is a second async request triggered after the app starts. To ensure nothing depending on config values is ran too soon all modules with setup/initializing code have an init method that is called from a Startup module after config.json has been received. This is triggered by wrapping the root React component with a config fetcher that simply initiates the request for config.json, then mounts its children and calls the startup module once config.json is received.

[rmoorman]

@FahdW I now have a better picture of what your question was about. In case of using a javascript file for the environment variables (env.js) you would use a script tag within the index.html file from CRA much like this:

  <body>
    <noscript>
      You need to enable JavaScript to run this app.
    </noscript>
    <div id="root"></div>

    <script src="%PUBLIC_URL%/env.js"></script>
  </body>

which ends up like this in the build output

<body><noscript>You need to enable JavaScript to run this app.</noscript><div id="root"></div><script src="./env.js"></script><script type="text/javascript" src="./static/js/main.0cf5ce69.js"></script></body>

Loading the env.js this way is completely synchronous. env.js will be loaded before the main bundle and you can just assume the configuration is there.

You could however (as @forrest-rival also pointed out) use an asynchronous request to fetch some json (env.json) too right when your application is loading. As mentioned, you would have to take care that the configuration is loaded before showing the app (by wrapping your app as mentioned).

I have been using both approaches in the past and personally, I found a synchronously loaded env.js file to be sufficient a lot of times. Loading the configuration dynamically gives you some flexibility with regards to configuration loading but comes with a bit complexity.
You can also combine both approaches of course.

In any case, at some point you might have to consider what to do with the app loaded up in your user's browsers when configuration changes. How to signal configuration changes to the browsers? Will you poll for changes? Have a socket open? SSE? And in case your configuration changed, is it enough to just load the new values or do you need to reload the whole application?
When you reach that point, some dynamically loaded configuration can really be a good way to solve the problems at hand.

@forrest-rival, nice idea to use aws parameter store and s3 for this :-)