-
Notifications
You must be signed in to change notification settings - Fork 134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When to make --default-type=module
the Node.js default
#1445
Comments
For those wondering, this is what the migration paths look like:
Or, of course, they could refactor code from CommonJS to ESM, but I’m not sure that needs to be presented as an option. This is nothing like Python 2 to 3. The other significant migration is of packages that rely on monkey-patching the CommonJS loader. The authors of major instrumentation libraries and of Yarn Plug-n-Play have been very active engaging with the loaders team in ensuring that the module customization hooks can be used to support their use cases. Those APIs are already in “release candidate” stage, just baking to see if anyone reports any bugs before we declare them stable. Regardless of the |
Doesn’t this mean that users writing a library would have different default behavior for themselves than for when they’re installed in someone else’s node_modules? |
Yes. I think changing that is going to a be a massive breaking change that would break everybody. Most modules would need to keep working with the flag. Other alternatives are
I would personally prefer to ship this in 23, having v24 the first LTS with the change, but I'm not opposed to an accelerated timeline. If we decide not to flip the switch in v22, we could ship some form of warning in v22 for typeless packages outside of node_modules, taking a similar "iteration" approach that we used in the past. Something to consider is the coordination with the TS team, as this will impact them and their resolution algorithms. Of course, Loaders should be stable by then and no critical bugs should be there (or in ESM) to do the switch. |
Let’s investigate this, shall we? Let’s go through some scenarios:
So basically, package authors would need to learn to add the |
Excellent summary @GeoffreyBooth that really clarified how migration would look and it sounds very reasonable. We should also contact npm and ask for:
@nodejs/npm does that sound reasonable? Edit: opened an issue with npm: npm/cli#6855 |
For reference, adding |
Am I following this correctly, that every package that's currently published will stop working once this goes into effect, and only newer versions of those packages that have this field defined will work? |
@wraithgar no, the issue is with the default changing outside node_modules, the issue is when people publish new versions of packages without specifying |
I think this means if you have a simple script like this: // test.js
require('fs') This would stop working once the flag is flipped, unless you change the file name or add a |
Also, wouldn't this break most of our tests unless we do something? (and to that extent, it could probably break a lot of other tests in other projects that are run in a similar "loose" fashion). It could also force all projects not yet migrated to do something, while we already know that there are a lot of simple projects that are not actively maintained but are still heavily depended upon in the ecosystem - or the Node.js ecosystem once favored very simple packages like this, so if an author has tons of very simple packages that do not need much maintenance since Node.js v0.x days (because who would've thought that Node.js is going to break such simple use cases), they will now be forced to update all these small packages. And many of the old reproductions of confirmed bugs would also need updating... |
The “do something” is what I wrote above as the migration path for CommonJS projects with no People seem to keep pointing at Python 2-3 as the only example of a migration, but this is nothing like that: that required everyone to refactor their code, whereas this requires no one to refactor their code. They can if they want to, but this is a configuration change not a syntax change. Yes the “shell scripts” case is the most awkward, because people often lack |
Requiring everyone to add a new file or change file name is no less a problem, I would say, especially considering:
First of all I don't think whatever we put down on that web page is regarded as our principal. There are also a lot of things on that web page is out of sync with how Node.js is used in the wild. Also on that note, the hello world on that page is still using ComomnJS. Second of all, claiming that "Node.js is designed to build scalable network applications." does not mean Node.js is only designed for that and we should disregard other use cases, for example, tooling. Also, can one lose the ability to build a scalable network application just because the default entry type stays the way it is since >10 years ago? I doubt how much difference the default entry type makes if that's the only thing we ever care about.
I think above all Node.js prioritizes stability and not breaking existing code unless absolutely necessary.
Running a |
Yes, many guides to Node.js were written a long time ago. Most people don’t develop “Node apps” per se anymore, they develop in frameworks that build on Node, or with things like TypeScript that use Node. If you look at those docs, most use ESM syntax. @mhdawson do we have any survey data to put some numbers on how people use Node, or what they use it with, or what their preferences are regarding modules? Here’s what happens if you use ESM in a CommonJS context today: mkdir test && cd test
echo 'import "fs"' > entry.js
node entry.js
(node:51228) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension. This would be the inverse experience when we flip the default, in that the message would say something like “Warning: To load a CommonJS module, set
Because the new mode doesn’t affect anything under |
I don't think we should base our decision of such a wide-spread breakage on pure impressions. We need evidence that this is not going to break, say, more than 5% of Node.js users (no matter they are writing code in a framework, or just using Node.js as a runner/tool, or using Node.js inside another environment e.g. cloud provider, desktop environment, or using Node.js to run mod/plugin script), before we can break an API contract that has been by design for >10 years.
By dependencies, I mean that the Node.js application is used as a runner/build tool/a component as part of the project, while developers/users do not necessarily know how to deal with it (e.g. our |
I found the data: https://github.com/nodejs/next-10/tree/main/surveys/2023-04 And in https://github.com/nodejs/next-10/blob/main/surveys/2023-04/Node.js%20Survey%20open%20ended%20answers.pdf, question 18 is “If you wish to use ESM in an existing application, what have been the pain points or blockers preventing you from doing so (if |
I don't think the comparison to Python is fair at all. This is like Python 2/3 if Python 2 code kept working. Namely:
Python 2->3 would have been if we removed CJS entirely, removed all the non-promise callback APIs, removed all deprecated APIs and made breaking changes to promise ones and then created an incompatible Node.js binary effectively making "two Nodes". This on the other hand is just changing the default outside node_modules |
I do not think the survey is created in a way that gathers data about how a widespread breakage in the existing code matter for users. It is true that people may want and do work on transitions when they have time, but it is also true that widespread breakages are disruptive and must be done with care. The right questions should be:
Even in the survey, you can see that more than 30% of the respondents do not plan to migrate to ESM in the near future. I doubt how many of our users would be happy to trade a breakage in a widespread way of running CommonJS in order to speed up migration to ESM. It's not impossible to do forever, but I think we are still very far away from that point. If this only breaks running loose CommonJS files, and if that was really just a niche use case as you described, how much value would this actually bring to the speedup of ESM migration and why is it so urgent that it need to be done in a year? Either this is very widespread and moving on would be an important milestone, or this is just a niche use case so this is one of the last thing that needs to be taken care of. I don't think it can be both niche and an important milestone. |
If you check the search results, many of them are either written or updated within the last two years.
If that's really the case, why would it matter how we run the loose scripts by default? The frameworks should take care of this for them and how the entry point is interpreted by default is the last thing they need to care about. This is also contradicting what the OP says:
Either most users get started by running a loose script and need to configure something to make it run as ESM by default, or they get started by writing code using some framework and the configuration should be done by the framework. I very much doubt how many JavaScript tutorials would teach readers to run ESM code in Node.js without any extra configuration and let them assume that it would just work, but we have evidence that most top Node.js tutorials are still explicitly teaching people to write a CommonJS entry point and run it without any additional configuration. I don't think we should invalidate existing working tutorials to prioritize non-working tutorials that may or may not even exist. Following a tutorial and finding out that what it says can run does not actually run is much more frustrating than trying out code in an environmnet that the tutorials do not teach (because it's in fact not runnable) and finding out that it doesn't work. |
If you want to, you can make any conclusions you want from that survey data. You could look at the 65-35 split of users already using ESM and say that more than two-thirds of our users are already using ESM, why shouldn’t it be the default? Or you could look at the 35% and say but this is still a large group, too large to inconvenience, etc. I think one fair conclusion that can be drawn is that this data does prove that far and away our users are application developers. They may run some loose shell scripts, but by and large I think it’s safe to assume that the vast majority of our users are probably doing most of their Node development in apps with a
I think the framing of “widespread breakage” or “Would you accept a change that breaks” or “Would you accept other breakages” is very leading. Not all breaks are created equal. As @benjamingr wrote, there’s a big difference between adding Every breaking change, like any change, has costs and benefits. The question for us is whether the benefits of this one outweigh its costs. I think they do: I think it really matters that Node join the rest of the ecosystem in prioritizing the standard module format, that we provide the best experience for users in that mode, and that we make it easier to write standards-compliant JavaScript. I think the requirement of a one-line JSON file change, or adding a flag, is a very minimal cost to impose on a minority of users in order to realize these benefits.
Well then that would imply that they’re still maintained, and if we announce that this change is coming, they’ll be updated before April 2024. Regardless, we can add a link to our own tutorial in the error message that someone would get when they run CommonJS syntax in ESM mode. I don’t view this as a blocker. |
I don't think "we are not going to do a large-scale survey" is a good response for the lack of data to support an important breakage like this. If we can't run a survey anytime soon, then we should wait until we are ready to run another survey, gather data about how breaking the breakages are, before we make a decision to break something that's been working for more than 10 years and in most top Node.js tutorials out there.
If the weight of this breakage is enough in the prioritization, then it should be a very commonly used thing that would break a lot of people, hence a huge cost. If it only breaks a minority of users, then it should not have much weight in the prioritization and it's not really worth breaking. I don't think it can only have a minimal cost while also bring a lot of weight in the migration. |
I think this is a blocker. We need to at least make sure that the top Node.js tutorials out there are teaching people to write ESM by default before we create a sense of emergency, and for example, >90% of our users are ready for the flip. If that doesn't happen, then we should postpone the flip. It's fine to announce that a change is coming, but I think a one-year notice is just too short for breaking something that's been working and assumed everywhere for more than 10 years. |
I think what you’re saying is that this is a big deal because it affects so many users; up to a third of our user base. What I’m saying is that it’s a small deal because all it takes is a minute of effort on those users’ part to update their projects to work in the new mode. Both things are true. If there’s a way to run another survey, sure, that would be great. @mhdawson? But I think we need to write the 21 announcement soon, so I find it hard to imagine we’ll have results in time to inform what we write. Even if we wanted to I don’t think we can promise anything in particular, because no one can predict what PRs get blocked, but I think we can say that we intend to propose flipping the default of |
I think "we'll flip in 22" is a message that shows how little we care about the stability of this software. This is not a niche API that only a fraction of users know about. It's something that has been working and assumed everywhere for more than 10 years, and it's something that almost all Node.js beginner tutorials out there has been teaching. A one-year notice for that is just too hasty. The message we should give is that "we'll flip when there's sufficient evidence that less than X percent of our users would be affected by the breakage" and ideally send a survey gathering data about the breakage (basically, "if you run the Node.js code you publish or maintain with |
This kind of statement has no acknowledgement that we’re just changing a default configuration setting, and that it’s easy to continue working in the previous mode if desired. I read phrases like “affected by the breakage” as implying that affected users are just irreparably broken, like they can never upgrade. This is not that. It does imply that we strongly prefer stability and backward compatibility, which we do, and I think that’s one of Node’s strengths. But a statement that says that we need to wait years to catch up with the ecosystem, to embrace a standard that was released in 2015, makes Node seem hopelessly stagnant and stuck in the past. I’d like to think that we’re a more forward-looking and faster-moving project than that. I think we can draft a statement along the lines of this:
|
Even browsers do not support
I would say this statement only looks appropriate when the flag is no longer experimental. It still looks careless to release a timeline for such a widespread breakage based on an experimental feature that just gets out and is not yet even tested in the wild. |
(WITH ALL RESPECT...) This is why options like Bun/Deno exist. Even stupid things like this don't get fixes in Node. How is that in 2024, beginners who want to write Node code, they have to deal with dumb thing like this? Ohhh, you need to add Just get over it. Node isn't the browser. It can introduce breaking change. By not introducing it, it just adds more complications and annoyance. How many conversations have happened over this??? The community will move over. It's not a big deal. If Node drops CJS, community packages will do the same, and at some point, it will be over. People even write migration scripts... Take a look at the state of JavaScript, people say that it's because of backwards compatibility, but that's BS because browsers can easily detect what version of whatever something is by reading the response or HTML attributes or whatever. Things should be easier for future developers, not harder/annoying. Feel free to like/dislike. I also want to say that I'm happy to see that Node team doesn't lock issues (as far as I've seen). Unfortunately, it happens a lot with other open-source projects (with the claims that they listen to community, lol). |
@babakfp Node can detect the module format by default from v22.7.0 |
Thank you for letting me know. This solves the problem with:
Which is great. But:
I'll open a new discussion... |
I understand your frustration. But you can't just start a reply with "all respect" and be anything but respectful. We're all in for open and respectful discussions here. |
For real? |
const {
getPort,
checkPort,
getRandomPort,
waitForPort,
} = require("get-port-please")
const port = await getPort() Getting this:
With this {
"dependencies": {
"express": "4.21.0",
"get-port-please": "3.1.2"
},
"devDependencies": {
"@types/express": "5.0.0"
}
} LOL. ( |
@babakfp please create a new issue. |
Now that nodejs/node#49869 has landed, and we need to figure out what to say about it for the 21.0.0 announcement, we should agree on a tentative plan for when the default value of
--default-type
flips fromcommonjs
tomodule
. That would be a semver-major change.Obviously we can’t announce a certain date or version, since we don’t know what issues may arise during 21.x as people test out
--experimental-default-type=module
and when we might consider it stable, but assuming things go fairly smoothly then we could conceivably make the flip in 22.0.0 in April 2024. I think that would probably be preferable to 23.0.0 in October 2024, which would mean that the first LTS line with ESM by default wouldn’t begin until 24.x in April 2025, which is a long way off.I think for the health of the project and the greater JavaScript ecosystem it’s important for Node.js to shift to an ESM-first mindset. We will continue to support CommonJS as a first-class module system, but it becomes opt-in rather than opt-out, conforming Node with other runtimes and modern tools.
This is primarily a change for new users. Most users learning JavaScript today are learning it with
import
/export
syntax and get frustrated when Node doesn’t run their code right away without messing with configuration settings. I think it’s important that we prioritize the UX of these users. The users still writing CommonJS today are among our most sophisticated and experienced, and will be able to figure out how to opt into the old mode via--default-type
or the"type"
field.The text was updated successfully, but these errors were encountered: