Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global Environment Variables from Root or via a "Package" #3928

Closed
anricoj1 opened this issue Feb 23, 2023 · 15 comments
Closed

Global Environment Variables from Root or via a "Package" #3928

anricoj1 opened this issue Feb 23, 2023 · 15 comments
Labels
area: docs Improvements or additions to documentation needs: investigation The bug / idea needs more investigation

Comments

@anricoj1
Copy link

Which project is this feature idea for?

Turborepo

Describe the feature you'd like to request

I feel as though many users would greatly benefit from more helpful docs on "Using environment variables".

The only suggestion is to install dotenv-cli and add it infront of the script like "dev": "dotenv -- turbo dev".

I'm currently working on adding prisma to to my monorepo for a shared db client. The turborepo docs on adding prisma are helpful but not in regards to environment variables like the DATABASE_URL.

Describe the solution you'd like

I feel like more suggestions for having a shared .env config in your monorepo would be ideal.

Regardless of how this is handled via a "package" in the monorepo or using a .env file from root without having to manage dotenv-cli especially when having multiple .env files like

  • .env.local
  • .env.local.development
  • .env.local.production

This would be a great out of the box feature since people usually need some .env managment ESPECIALLY in a monorepo.

Describe alternatives you've considered

I feel as though my [Describe the solution you'd like] section hits this point already.

I've tried using the recommended dotenv-cli approach from the docs and it doesn't work for my use case.

This is the approach I tried using my .env file from root with prisma and turbo:
In root package.json:
"db:migrate": "dotenv -e ./env -- turbo db:migrate"

In packages/database/package.json:
"db:migrate": "prisma migrate dev --name init"

Running npm run db:migrate from root gives me this error message:

database:db:migrate: error: Environment variable not found: DATABASE_URL.

I understand that this could potentially be labeled as an issue with prisma. But, the question still stands for managing environment variables throughout your turborepo

@anricoj1 anricoj1 added needs: triage New issues get this label. Remove it after triage story labels Feb 23, 2023
@mehulkar
Copy link
Contributor

Thanks for filing @anricoj1. Just to make sure I understand the request: are you looking for more guidance on how to load environment variables in your monorepo? Or are you thinking that Turborepo can do something automatically when you do turbo run build (or some other task)? For example, are you running db:migrate via the turbo CLI and hoping that it would make DATABASE_URL available where it's needed?

@mehulkar mehulkar added needs: author input and removed needs: triage New issues get this label. Remove it after triage labels Feb 23, 2023
@anricoj1
Copy link
Author

@mehulkar Correct I was hoping running db:migrate via the turbo CLI and it would make DATABASE_URL available where it's needed.

I think adding more guidance in the docs on managing environment variables across your turborepo would be very helpful.

I'm using my current issue as an example for managing environment variables throughout my turbo repo. I have a .env.local.development file in the root of my project. I would like to be able to use it throughout all of my projects in my turborepo.

Is it better practice to:

  1. Give each app their own .env files.
  2. Create a env package inside the packages/ dir.
  3. Use dotenv-cli Infront of every turbo command.

It would be great if turbo could handle this out of the box or at least provide an opt in via npx create-turbo@latest to configure dotenv-cli for you.

As the template already provides you with example eslint tsconfig and ui packages it would be cool to see an example env package

@mehulkar mehulkar added needs: team input Filter for core team meetings and removed needs: author input labels Feb 23, 2023
@bchilcott
Copy link

bchilcott commented Apr 4, 2023

+1: It's also not clear in the with-prisma example that the env solution there won't work on windows (unless I've done something horribly wrong).

The best alternative seems to be adding dotenv -e ../../.env to each script, which feels like a bit of a hack, especially when you have a lot.

If there is a better way it doesn't seem to be documented, and if there isn't it seems like some kind of automatic solution would be a valuable improvement.

@mehulkar
Copy link
Contributor

mehulkar commented Apr 4, 2023

Does #4406 go far enough for you? I'd be happy to review a PR for where you think some extra material could go if you want to start one up! I'm guessing somewhere in the Monorepo Handbook is a good idea.

@bchilcott
Copy link

Does #4406 go far enough for you?

Ah, didn't realise that had been merged at the same time I was having the problem.. It looks like the ../../.env thing is the best option we've got atm so having that documented is good. If I think of something better I'll open a PR for it, but handling global env vars should be the monorepo tool's job ultimately, imo.

@andrevenancio
Copy link

Just wanted to share my thoughts. I think having some common environment variables to the whole monorepo is useful, however, as you know, in a monorepo with multiple applications where we can specify an app and a vercel project its more useful to have environment variables local to the application itself. I've tried to add .env.local to my apps/my-application and I can't get the variables to work. I could add all my environment variables to the root of the monorepo but that's not ideal. some env variables might be called the same and I dont want to be filtering which variables I'm interested.

Is there a solution I can use to avoid going down the path of

// .env
APPLICATION1_GRAPHQL=...
APPLICATION2_GRAPHQL=...
APPLICATION3_GRAPHQL=...

@chihab
Copy link

chihab commented Jun 10, 2023

@andrevenancio could this package help? @dotenv-run/cli

/workspace
 apps
    frontend1
       .env.local # API_USERS=http://localhost:3001/users
       src/
.env.dev # API_BASE=https://dotenv-run.dev
.env.prod # API_BASE=https://dotenv-run.app
.env # API_USERS=$API_BASE/api/v1/users API_AUTH=https://$API_BASE/auth
package.json
turbo.json
$> cd /workspace/apps/frontend1
$> NODE_ENV=dev dotenv-run -- bash -c 'printf "✨API_USERS $API_USERS\n✨ API_AUTH $API_AUTH"'
✔ /workspace/apps/frontend1/.env.local
✔ /workspace/.env.dev
✔ /workspace/.env
✨ API_USERS http://localhost:3001/users
✨ API_AUTH https://dotenv-run.dev/api/v1/auth

@xmlking
Copy link

xmlking commented Nov 20, 2023

@dotenv-run/cli had nice feature needed for turbo repo:

Supports hierarchical cascading configuration in monorepo projects (Nx, Turbo, etc.) apps/next-app/.env > apps/.env > .env

but missing support for multiple custom .env files

chihab/dotenv-run#15

@andrevenancio
Copy link

andrevenancio commented Nov 21, 2023

I think the whole way turborepo is handling environment variables along side the vercel cli is not well though through.

So lets consider this, we have a monorepo with 2 applications app-a and app-b. On vercel we create 2 separate applications and we link them accordingly:

apps/app-a for app-a and
apps/app-b for app-b

We open vercel app-a application dashboard, add a few env variables and then we change to the app-b and add a few other variables.

Before I can run vercel env pull on my local machine I need to do vercel link. When we link we need to choose what application we're linking to. Either app-a or app-b. At this point running vercel env pull pulls the environment variables to a .env.local on the root of the monorepo and those variables are refering to app-a which is the app we linked. Those environment variables defined on .env.local can be propagated to all other apps and packages on turborepo if we add

"globalDotEnv": [".env.local"],

to the turbo.json file.

Now this means that if we're testing app-a we have our env variables, but if we are testing app-b we're out of luck and you will have to do vercel link again, pointing to app-b, and then do vercel env pull to get the env variables used on app-b.

I would expect maybe that doing vercel env pull will pull a specific .env.local to each of the apps that have been added as Projects on vercel So I would have

apps
  app-a
    .env.local
  app-b
    .env.local
packages
  ...

Instead of

apps
  app-a
  app-b
packages
.env.local
  ...

@xmlking
Copy link

xmlking commented Nov 22, 2023

"globalDotEnv": [".env.local"],
Not sure if turbo command currently can load above env variables into process.env when run:

turbo run dev —filter=apps/web

other idea is to have common .env at project root level and per-subproject .env in apps/web level.
When we rung turbo command , it should merge hierarchical .env files and make them available to process.env

@andrevenancio
Copy link

Hi @xmlking thanks for your reply.

So would you do the vercel link inside each of the different apps you want to pull the .env from?

Following my structure in the example above

cd apps/app-a && vercel link
// follow the steps to link to the app on vercel called "app-a"

And then do

cd ..
cd app-b && vercel link
// follow the steps to link to the app on vercel called "app-b"

And that "links" you to the 2 applications so you can run vercel env pull from inside each folder and get the environment variables relating to that app only?

@andrevenancio
Copy link

Got an update, not sure if its helpful but please have a read here

@gsoltis gsoltis added needs: investigation The bug / idea needs more investigation and removed needs: team input Filter for core team meetings labels Dec 18, 2023
@xmlking
Copy link

xmlking commented Jan 10, 2024

@chihab generously added turborepo support with some additional features I requested: https://dotenv.run/
this is how I am using it in apps/console/package.json

{
	"scripts": {
		"dev": "dotenv-run -p .env,.secrets -- vite dev",
		"build": "dotenv-run -p .env,.secrets -- vite build",
		"preview": "dotenv-run -p .env,.secrets -- vite preview",
	}
}

in debug mode, it loads env files like this

image

@anthonyshew anthonyshew removed the story label Feb 22, 2024
@anthonyshew anthonyshew added the area: docs Improvements or additions to documentation label Apr 2, 2024
@anthonyshew
Copy link
Contributor

We've updated our environment variable guidance in new documentation: https://turbo.build/repo/docs/crafting-your-repository/using-environment-variables

@sajadmh
Copy link

sajadmh commented Jun 25, 2024

The new documentation is helpful but easy to miss when searching for Vercel build error messages that don't occur in local dev.

For example:

web:build: Error: WORKOS_REDIRECT_URI environment variable is not set
web:build:     at r (/vercel/path0/apps/web/.next/server/chunks/2071.js:20:32761)
web:build:     at 88988 (/vercel/path0/apps/web/.next/server/chunks/2071.js:20:33174)
web:build:     at t (/vercel/path0/apps/web/.next/server/webpack-runtime.js:1:128)
web:build:     at 12686 (/vercel/path0/apps/web/.next/server/chunks/2071.js:23:6987)
web:build:     at t (/vercel/path0/apps/web/.next/server/webpack-runtime.js:1:128)
web:build:     at 45211 (/vercel/path0/apps/web/.next/server/chunks/2071.js:20:30800)
web:build:     at t (/vercel/path0/apps/web/.next/server/webpack-runtime.js:1:128)
web:build:     at 72071 (/vercel/path0/apps/web/.next/server/chunks/2071.js:23:2607)
web:build:     at t (/vercel/path0/apps/web/.next/server/webpack-runtime.js:1:128)
web:build:     at 42212 (/vercel/path0/apps/web/.next/server/chunks/9550.js:17:13171)
web:build: 
web:build: > Build error occurred
web:build: Error: Failed to collect page data for /analytics
web:build:     at /vercel/path0/node_modules/.pnpm/next@14.2.4_@babel+core@7.24.7_@opentelemetry+api@1.8.0_react-dom@18.3.1_react@18.3.1__react@18.3.1_sass@1.77.6/node_modules/next/dist/build/utils.js:1268:15
web:build:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {

The environment variable is set in the Vercel project and locally in the app/package, but it wasn't clear that 'environment variable is not set' referred to the root turbo.json and needed to be added under 'globalEnv' or 'env'.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: docs Improvements or additions to documentation needs: investigation The bug / idea needs more investigation
Projects
None yet
Development

No branches or pull requests

9 participants