src: support multi-line values for .env file #51289

IlyasShabi · 2023-12-26T12:53:05Z

Following the follow-up, we currently do not handle multi-line values in the .env file.
This PR aims to add this feature based on the implementation in dotenv

IlyasShabi · 2023-12-26T12:54:17Z

cc @anonrig

anonrig · 2023-12-26T13:03:37Z

@IlyasShabi Can you fix the C++ linting errors?

github-actions · 2023-12-26T13:03:55Z

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @anonrig.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

src/node_dotenv.cc

test/fixtures/dotenv/valid.env

test/parallel/test-dotenv.js

RafaelGSS

Can you update the docs too? It should include the version .env start supporting multiline strings.

test/fixtures/dotenv/valid.env

anonrig · 2023-12-26T15:20:30Z

src/node_dotenv.cc

-  while (!key.empty() && std::isspace(key.back())) key.remove_suffix(1);
+    // Expand newlines if double quoted
+    if (maybeQuote == '"') {
+      value = std::regex_replace(value, std::regex("\\\\n"), "\n");


I believe string erase and remove would be faster and more readable than regex replace. The same applies for the next line as well.

str.erase(remove(str.begin(), str.end(), ch), str.end());

test/fixtures/dotenv/valid.env

src/node_dotenv.cc

test/fixtures/dotenv/valid.env

src/node_dotenv.cc

tniessen · 2023-12-27T11:11:17Z

src/node_dotenv.cc

@@ -8,6 +9,17 @@ namespace node {
 using v8::NewStringType;
 using v8::String;

+/**
+ * The inspiration for this implementation comes from the original dotenv code,


Does this mean that the parser is exactly compatible with dotenv? #49148 (comment) sounds like dotenv supports multi-line values, and if it does, this implementation should probably be compatible.

@tniessen The goal of this PR is to support multi-line values same as dotenv

IlyasShabi · 2024-01-07T18:06:56Z

test/parallel/test-dotenv.js

+assert.strictEqual(process.env.MULTI_BACKTICKED, 'THIS\nIS\nA\n"MULTILINE\'S"\nSTRING');
+assert.strictEqual(process.env.MULTI_NOT_VALID_QUOTE, '"');
+assert.strictEqual(process.env.MULTI_NOT_VALID, 'THIS');
+assert.strictEqual(process.env.EXAMPLE, 'ignore export');


This PR also adds support for the export keyword. According to dotenv, there may be an export keyword before the key, this keyword is ignored.
Unfortunately, I did not find a test for the export feature in the dotenv test suite.

IlyasShabi · 2024-01-07T18:26:13Z

@RafaelGSS @anonrig @tniessen Could you please review it again ?

himself65 · 2024-01-07T21:37:31Z

src/node_dotenv.cc

+ * The inspiration for this implementation comes from the original dotenv code,
+ * available at https://github.com/motdotla/dotenv
+ */
+const std::regex LINE(


Ref: https://github.com/motdotla/dotenv/blob/2eed66b107229eb662a7edfe1ef31597c8d53023/lib/main.js#L9

src/node_dotenv.cc

UlisesGascon · 2024-01-30T19:36:06Z

It seems all macOS devices are down nodejs/build#3616

@anonrig the macos machines are available again, so they are processing the CI pending queue. It can take a while

nodejs-github-bot · 2024-01-31T13:09:15Z

Landed in 8b28366

PR-URL: nodejs#51289 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>

PR-URL: #51289 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>

PR-URL: nodejs#51289 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>

Notable changes: build: * (SEMVER-MINOR) build opt to set local location of headers (Michael Dawson) #51525 crypto: * (SEMVER-MINOR) implement crypto.hash() (Joyee Cheung) #51044 * update root certificates to NSS 3.98 (Node.js GitHub Bot) #51794 doc: * add zcbenz to collaborators (Cheng Zhao) #51812 * add lemire to collaborators (Daniel Lemire) #51572 http2: * (SEMVER-MINOR) add h2 compat support for appendHeader (Tim Perry) #51412 * (SEMVER-MINOR) add server handshake utility (snek) #51172 * (SEMVER-MINOR) receive customsettings (Marten Richter) #51323 lib: * (SEMVER-MINOR) move encodingsMap to internal/util (Joyee Cheung) #51044 sea: * (SEMVER-MINOR) support sea.getRawAsset() (Joyee Cheung) #50960 * (SEMVER-MINOR) support embedding assets (Joyee Cheung) #50960 src: * (SEMVER-MINOR) print string content better in BlobDeserializer (Joyee Cheung) #50960 * (SEMVER-MINOR) support multi-line values for .env file (IlyasShabi) #51289 * (SEMVER-MINOR) add `process.loadEnvFile` and `util.parseEnv` (Yagiz Nizipli) #51476 * (SEMVER-MINOR) do not coerce dotenv paths (Tobias Nießen) #51425 stream: * (SEMVER-MINOR) implement `min` option for `ReadableStreamBYOBReader.read` (Mattias Buelens) #50888 util: * (SEMVER-MINOR) add styleText API to text formatting (Rafael Gonzaga) #51850 vm: * (SEMVER-MINOR) support using the default loader to handle dynamic import() (Joyee Cheung) #51244 PR-URL: TODO

Notable changes: build: * (SEMVER-MINOR) build opt to set local location of headers (Michael Dawson) #51525 crypto: * (SEMVER-MINOR) implement crypto.hash() (Joyee Cheung) #51044 * update root certificates to NSS 3.98 (Node.js GitHub Bot) #51794 doc: * add zcbenz to collaborators (Cheng Zhao) #51812 * add lemire to collaborators (Daniel Lemire) #51572 http2: * (SEMVER-MINOR) add h2 compat support for appendHeader (Tim Perry) #51412 * (SEMVER-MINOR) add server handshake utility (snek) #51172 * (SEMVER-MINOR) receive customsettings (Marten Richter) #51323 lib: * (SEMVER-MINOR) move encodingsMap to internal/util (Joyee Cheung) #51044 sea: * (SEMVER-MINOR) support sea.getRawAsset() (Joyee Cheung) #50960 * (SEMVER-MINOR) support embedding assets (Joyee Cheung) #50960 src: * (SEMVER-MINOR) print string content better in BlobDeserializer (Joyee Cheung) #50960 * (SEMVER-MINOR) support multi-line values for .env file (IlyasShabi) #51289 * (SEMVER-MINOR) add `process.loadEnvFile` and `util.parseEnv` (Yagiz Nizipli) #51476 * (SEMVER-MINOR) do not coerce dotenv paths (Tobias Nießen) #51425 stream: * (SEMVER-MINOR) implement `min` option for `ReadableStreamBYOBReader.read` (Mattias Buelens) #50888 util: * (SEMVER-MINOR) add styleText API to text formatting (Rafael Gonzaga) #51850 vm: * (SEMVER-MINOR) support using the default loader to handle dynamic import() (Joyee Cheung) #51244 PR-URL: #51932

PR-URL: #51289 Reviewed-By: Yagiz Nizipli <yagiz.nizipli@sentry.io> Reviewed-By: Zeyu "Alex" Yang <himself65@outlook.com> Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com> Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>

Notable changes: build: * (SEMVER-MINOR) build opt to set local location of headers (Michael Dawson) nodejs#51525 crypto: * (SEMVER-MINOR) implement crypto.hash() (Joyee Cheung) nodejs#51044 * update root certificates to NSS 3.98 (Node.js GitHub Bot) nodejs#51794 doc: * add zcbenz to collaborators (Cheng Zhao) nodejs#51812 * add lemire to collaborators (Daniel Lemire) nodejs#51572 http2: * (SEMVER-MINOR) add h2 compat support for appendHeader (Tim Perry) nodejs#51412 * (SEMVER-MINOR) add server handshake utility (snek) nodejs#51172 * (SEMVER-MINOR) receive customsettings (Marten Richter) nodejs#51323 lib: * (SEMVER-MINOR) move encodingsMap to internal/util (Joyee Cheung) nodejs#51044 sea: * (SEMVER-MINOR) support sea.getRawAsset() (Joyee Cheung) nodejs#50960 * (SEMVER-MINOR) support embedding assets (Joyee Cheung) nodejs#50960 src: * (SEMVER-MINOR) print string content better in BlobDeserializer (Joyee Cheung) nodejs#50960 * (SEMVER-MINOR) support multi-line values for .env file (IlyasShabi) nodejs#51289 * (SEMVER-MINOR) add `process.loadEnvFile` and `util.parseEnv` (Yagiz Nizipli) nodejs#51476 * (SEMVER-MINOR) do not coerce dotenv paths (Tobias Nießen) nodejs#51425 stream: * (SEMVER-MINOR) implement `min` option for `ReadableStreamBYOBReader.read` (Mattias Buelens) nodejs#50888 util: * (SEMVER-MINOR) add styleText API to text formatting (Rafael Gonzaga) nodejs#51850 vm: * (SEMVER-MINOR) support using the default loader to handle dynamic import() (Joyee Cheung) nodejs#51244 PR-URL: nodejs#51932

nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Dec 26, 2023

IlyasShabi force-pushed the dot-env-multi-lines branch from f6d0899 to 35eb89c Compare December 26, 2023 12:58

anonrig self-requested a review December 26, 2023 13:00

anonrig added the notable-change PRs with changes that should be highlighted in changelogs. label Dec 26, 2023

anonrig reviewed Dec 26, 2023

View reviewed changes

src/node_dotenv.cc Outdated Show resolved Hide resolved

src/node_dotenv.cc Outdated Show resolved Hide resolved

test/fixtures/dotenv/valid.env Outdated Show resolved Hide resolved

test/parallel/test-dotenv.js Outdated Show resolved Hide resolved

IlyasShabi force-pushed the dot-env-multi-lines branch 3 times, most recently from 3ca8625 to 876aef7 Compare December 26, 2023 13:14

RafaelGSS reviewed Dec 26, 2023

View reviewed changes

test/fixtures/dotenv/valid.env Show resolved Hide resolved

IlyasShabi force-pushed the dot-env-multi-lines branch from 876aef7 to 7089fd7 Compare December 26, 2023 14:56

anonrig reviewed Dec 26, 2023

View reviewed changes

tniessen reviewed Dec 27, 2023

View reviewed changes

IlyasShabi force-pushed the dot-env-multi-lines branch 3 times, most recently from 99d1f88 to e7fe39d Compare December 31, 2023 09:00

IlyasShabi force-pushed the dot-env-multi-lines branch 5 times, most recently from 3b3c645 to de3d605 Compare January 7, 2024 09:51

IlyasShabi commented Jan 7, 2024

View reviewed changes

IlyasShabi force-pushed the dot-env-multi-lines branch from 0e6d3ca to 5ed6c5b Compare January 7, 2024 18:32

himself65 approved these changes Jan 7, 2024

View reviewed changes

anonrig requested changes Jan 7, 2024

View reviewed changes

src/node_dotenv.cc Outdated Show resolved Hide resolved

src/node_dotenv.cc Outdated Show resolved Hide resolved

anonrig added the commit-queue Add this label to land a pull request using GitHub Actions. label Jan 31, 2024

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jan 31, 2024

nodejs-github-bot merged commit 8b28366 into nodejs:main Jan 31, 2024
55 checks passed

IlyasShabi deleted the dot-env-multi-lines branch January 31, 2024 13:26

anonrig mentioned this pull request Jan 31, 2024

.env file support issue tracker #49148

Open

8 tasks

victorgomes mentioned this pull request Feb 14, 2024

src: fix empty string access in dotenv #51757

Closed

marco-ippolito mentioned this pull request Mar 1, 2024

v21.7.0 proposal #51932

Merged

IlyasShabi mentioned this pull request Mar 14, 2024

--env-file parser overrides variables with commented out values #52084

Closed

richardlau mentioned this pull request Mar 25, 2024

v20.12.0 proposal #52212

Merged

kibertoad mentioned this pull request Mar 28, 2024

Loading long env value from a file is crashing Node on Windows #52248

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

src: support multi-line values for .env file #51289

src: support multi-line values for .env file #51289

IlyasShabi commented Dec 26, 2023

IlyasShabi commented Dec 26, 2023

anonrig commented Dec 26, 2023

github-actions bot commented Dec 26, 2023

RafaelGSS left a comment

anonrig Dec 26, 2023

tniessen Dec 27, 2023

IlyasShabi Dec 30, 2023

IlyasShabi Jan 7, 2024 •

edited

Loading

IlyasShabi commented Jan 7, 2024

himself65 Jan 7, 2024

UlisesGascon commented Jan 30, 2024

nodejs-github-bot commented Jan 31, 2024

src: support multi-line values for .env file #51289

src: support multi-line values for .env file #51289

Conversation

IlyasShabi commented Dec 26, 2023

IlyasShabi commented Dec 26, 2023

anonrig commented Dec 26, 2023

github-actions bot commented Dec 26, 2023

RafaelGSS left a comment

Choose a reason for hiding this comment

anonrig Dec 26, 2023

Choose a reason for hiding this comment

tniessen Dec 27, 2023

Choose a reason for hiding this comment

IlyasShabi Dec 30, 2023

Choose a reason for hiding this comment

IlyasShabi Jan 7, 2024 • edited Loading

Choose a reason for hiding this comment

IlyasShabi commented Jan 7, 2024

himself65 Jan 7, 2024

Choose a reason for hiding this comment

UlisesGascon commented Jan 30, 2024

nodejs-github-bot commented Jan 31, 2024

IlyasShabi Jan 7, 2024 •

edited

Loading