Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug(esm): TypeScript is not an ECMAScript superset post-ES2015 #50501

Closed
ctjlewis opened this issue Aug 29, 2022 · 69 comments
Closed

bug(esm): TypeScript is not an ECMAScript superset post-ES2015 #50501

ctjlewis opened this issue Aug 29, 2022 · 69 comments
Labels
Discussion Issues which may not have code impact

Comments

@ctjlewis
Copy link
Contributor

ctjlewis commented Aug 29, 2022

Bug Report

This bug report will show that TypeScript is no longer an ECMAScript subset as of ES2015 due to its refusal to support import specifier rewrites for contrived and misplaced reasons.

Preface

This issue is related to previously reported issues, but expands on them and clarifies the scope of the problem.

This team, mostly Andrew and Ryan, have repeatedly shut down these issues in the past. I ask that this issue not be closed on a kneejerk reaction because I intend to speak with others at Microsoft about it.

TS is designed to be an ES superset

TypeScript doubtless has many design goals, and I understand how hard it is to balance all of them. However, undeniably, its core design goal - what TypeScript exists to do - is stated in this repository's description and also throughout the TS docs:

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

We should all agree that TS, first and foremost, is supposed to be an ECMAScript superset. There is no amount of mental gymnastics that gets us out of this. We will proceed with this in mind as we cover the scope of this issue and some of the technical excuses that have been offered to justify not addressing the problem.

What is a superset?

Originally a set theory concept, in terms of language it was best stated by @sepp2k in this StackOverflow answer:

Syntactically a language A is a subset of a language B if every program that is valid in language A is also valid in language B. Semantically it is a subset if it is a syntactic subset and each valid A program also exhibits the same behavior in language B.

To formalize this, Sebastian offers the following definition:

If P_A is the set of all valid programs in language A and P_B is the set of all valid programs in language B, then language A is a syntactic subset of language B exactly if P_A is a subset of P_B.

We will not need to address the semantic subset issue as TS fails the first test for arbitrarily many input programs1, which we will get into.

For convenience, and to emphasize the point that our subset programs are actually emitted JS versions of the input TS program, we will refer to a program written in the superset language as "program T" and the corresponding output in the subset language as "program T'".

Additionally, because TS output depends on your compiler configuration, we will consider TypeScript to pass the semantic test if there is some configuration for which you can transform a valid T into T'.

1 This is something this team is already well aware of, and have simply chosen to ignore. In my judgment, the only possible explanation for violating the core purpose of this technology for such a large portion of its lifecycle is organizational deficiency, and not any practical technical reason.

Is TypeScript a JS superset?

Sometimes. Let's walk through some examples (with approximated and simplified, not literal, output):

  1. ✅ Program T

    const a: string = "hello world!"
    console.log(a);

    ✅ Program T'

    const a = "hello world!"
    console.log(a);

    The input program is valid and the output program is valid. The runtime behavior of T and T' are identical. This example passes both the syntactic and semantic superset tests.

  2. ✅ Program T (using relative extensionless TS import)

    import { a } from "./a"
    console.log(a);

    ❌ Program T' (TS-style imports left untransformed)

    import { a } from "./a"
    console.log(a);

    The input program is valid, but there is no configuration for which you will get a valid ECMAScript module output. The output program T' will throw according to the ES spec. This example fails the syntactic and semantic superset tests. Our perfectly valid TypeScript program cannot be transformed by the compiler into a valid ECMAScript program, and this is easily shown.

    While we can construct a different input program for which we will receive valid output (import { a } from "./a.js"), making that suggestion is proof positive that TS is no longer an ES superset, and there are infinitely many valid input programs which will not emit valid output. Nothing about this is subjective or within the realm of personal judgment.

Bearing this in mind, the last version of ES for which TypeScript was a superset would be ES5 (2009), before the introduction of ES modules.

Why does this matter?

It matters because our industry has struggled to adopt ESM for a variety of reasons, largely related to interop with downstream CJS consumers and also integration with TypeScript. It should be fairly obvious the damage that is done to the ecosystem when the TypeScript compiler very literally cannot generate valid ESM programs without special syntax.

The idea of having to use a special syntax to achieve a correct output program is inherently antithetical to the "TS is an ES superset" principle on which TypeScript is based. It takes a severe amount of technical dysfunction to turn a blind eye to this for years and years, and a serious amount of contempt for users and software quality to continually shut down user feedback on this issue.

Inadequacy of existing solutions and team response to user feedback

I have clarified several times that I am chalking this up to organizational dysfunction and attempting to kick it up the chain myself, as many many many users have raised this issue in isolation only to be shut down and/or directly gaslit by members of this team.

Let's go over some of these issues. I will start with the closest suggestion to a fix currently under consideration, and then go through the rest in chronological order.

(!) 2020: allow voluntary .ts suffix for import paths (#37582) (by @timreichen)

I will start with this suggestion as it comes the closest to fixing the problem, while also offering a two-birds-one-stone benefit with regard to Deno, Bun, etc. support:

let
import a from "path/to/a.ts"
behave the same as
import a from "path/to/a"

As specified, this partially solves our problem. Because import "./a.ts" could only ever refer to a TypeScript program, you cannot conceive of any input .ts import statement which will not correctly build to a .js import and execute equivalently at runtime.

However:

  1. Because it is apparently "core design goal" that the imports never get rewritten, the most important part of this suggestion cannot be honored.

  2. (Very important) This demands you rewrite your existing, valid import "./a" statements to ./a.ts, and your valid ./a imports still do not build to valid ES import statements.

Thus, however we might choose to emit the statements from input, we must either:

  1. Resolve extensionless relative specifiers which are valid TS; OR
  2. Deprecate extensionless relative specifiers in TypeScript, and make them invalid TS, to bring T and T' to parity.

The current approach by this team has literally just been to do nothing and deny that there is a conflict here. This conflict and its relationship to TS as an ES superset is the core focus of this issue.

2017: TypeScript and <script type="module"></script> (#13422) (by @cyrilletuzi)

Because the problem is, again, that TypeScript ESM input programs cannot build to valid output, naturally <script type="module"> goes right out the window.

Same pattern: User opens thread, years of comments and confusion, denialism from team members, summarily closed.

2017: Provide a way to add the '.js' file extension to the end of module specifiers (#16577) (by @quantuminformation)

This issue looks like the others we will see: A user raising the issue that their valid TypeScript program cannot compile to a valid ES program, a large debate between users and various TS maintainers, and eventually the issue being closed, locked, and buried.

As usual, users point out that TS will act like their imports are valid while refusing to resolve them at compile-time, highlighting the tradeoff I explained. Relative extensionless specifiers must either be allowed and transformed, OR disallowed entirely. Doing both only violates the TS superset principle and confuses users because they depend on their output programs being isomorphic with their inputs.

Also, to generalize this issue a bit, I don't think it's actually about adding a .js extension, but resolving the module specifier to an actual path, whatever the extension is.

This comment has 50 likes. The parent issue has nearly 300. Minimizing comments from the team were met with a negative reaction as usual.

2020: Compiled JavaScript import is missing file extension (#40878) (by @jameshfisher)

This is an interaction, and a restatement of the same problem, which is effectively the perfect example of not just the problem and the common sense solution to it, but also the community's investment in this being solved, and the textbook denialism by maintainers. The bot response announcing the closure as "working as intended" has 65 downvotes.

Mr. Fisher raises the issue that a program containing valid TypeScript imports will not emit a valid corresponding ECMAScript program. As we will see, this issue is met with approval and enthusiasm by people who found it trying to understand why their programs will not build (after several hours, they will presumably put two and two together and realize it's just not possible). OP is then promptly sandbagged by members of this team and the issue is erroneously closed.

After the issue is posted, Ryan quickly responds to this post by minimizing OP's complaint and telling him to kick rocks:

TypeScript doesn't modify import paths as part of compilation - you should always write the path you want to appear in the emitted JS, and configure the project as necessary to make those paths resolve the way you want during compilation

Again, as we've covered, this is not how supersets work. Simply telling someone to rewrite every already-valid TS import statement into a different one which will compile should seem absurd on its face given what we've gone over. This comment received about 20 downvotes.

Mr. Fisher responds noting that he was shocked to find referring to TS source files with .js (an extremely confusing and hacky solution that is unbecoming of even a temporary workaround, let alone a permanent solution) actually does resolve his problem. He then lays out a few of the many reasons why this is confusing and hacky. His comment received 50 positive reactions.

Ryan responds with another heavily-downvoted dismissal of this issue:

TS never takes an existing valid JS construct and emits something different, so this is just the default behavior for all JS you could write. Module resolution tries to make all of this work "like you would expect"

You would expect a valid TS import to become a valid JS import because it's supposed to be a superset. It is not a discussion or an issue of perspective; TS claims to be a superset, if it is, the valid imports would remain valid at runtime. Yet they do not, and we have highly experienced and talented engineers effectively lying about what is and is not expected runtime behavior.

@RevealedFrom clarifies this at further length, though it should be pretty obvious already why there are several problems with this, least of all being the superset issue:

@RyanCavanaugh Writing "./dep.js" doesn't sound logical. The file dep.js does not exist in the Typescript universe. This approach requires the coder to know the exact complied output and be fully aware of the compiled environment. It's like having to know the CIL outputted and modify it here and there in order to code in C# successfully. Isn't the whole idea of Typescript to abstract away Javascript?

import { foo } from "./dep" is legitimate Typescript, and it provides the information for Typescript to resolve all that is needed to type check and make the code compile successfully. So, the compiled output should work. Typescript should not be generating syntactically incorrect Javascript.

IMHO, this issue should be a bug.

Ultimately, it is a bug, and we know it is a bug for the reasons we've covered. Ryan responds again with a minimizing and heavily-downvoted statement I would characterize as dishonest at best, and an outright lazy lie at worst:

The whole idea of TypeScript is to add static types on top of JavaScript, not to be a higher-level language that builds to JS.

We have enough background to know this is wrong, though we are starting to get insight into the executive dysfunction that is causing this issue. Core maintainers are making up what the purpose of this technology is as they go along, apparently now privately rejecting that TS is or ever was meant to be a JS superset.

A user in this thread (@Gin-Quin) had to actually implement a bundler to work around this, something I also had to do:

I will add that I indeed created a library called tsc-esm as a workaround for this bug, and even though it works quite fine every time I use it (ie 100% of the times I need to create a library written in Typescript) I feel I should not have to patch the output of the Typescript compiler like I do.

And another user (@djfm) also:

@silassare I think probably a thousand person, me included, have written similar scripts. This is just absurd.

And the last comment from this thread I will cover, and emphasize, is this one by Mr Felber (@PatrickFelber):

This whole discussion is absurd. Valid typescript obviously transpiles to invalid ES6 code. I do not understand in which world this can be classified as "works as intended" behavior. Adding ".js" to import statements which actually refer to ".ts" files as workaround is too wild for me. For my current project I have free choice, therefore I'm gonna switch over to CommonJS target module code. CommonJS works without file extensions. In case one can live with CommonJS target module, this is an option to get around this problem.

The only people who claim to not see the problem are members of this team. Note that this user's only choice was to give up and stay on CJS - this is what I am referring to when I say that this is likely the most major blockade to ESM adoption under Why does this matter?.

2020: TypeScript cannot emit valid ES modules due to file extension issue (#42151) (by me)

This issue was opened by me to clarify that the compiler cannot emit valid ES modules. It received positive feedback from users, but was naturally accompanied by denialism from Ryan and eventually closed and locked. A comment by @richardkazuomiller points out the ridiculousness of the justification offered for not fixing this issue ("we can't rewrite import specifiers"):

Exactly, could someone explain like I'm 5 why it's OK for TypeScript to convert import ... from './file' to const ... = require('./file') changing ./file to ./file.js is not OK?

2021: One year later: TypeScript still cannot emit valid ES modules (#47270) (by me)

This is a rephrasing of the original issue, posted so as to remind this team that this problem has not been resolved. The single response from Ryan was amusing given the context of this issue:

We've explained the rationale behind not modifying import paths on emit many times, and don't see value in rehashing that discussion since it relates to a core design goal. [...] Please file something concrete in the future.

You must decide to prioritize a TypeScript "core design goal", but you can only choose one:

  1. Never rewriting import specifiers, even though the compiler does this for regular import statements
  2. Ensuring TS is an ES superset like it says it is throughout its documentation

Apparently this is not an obvious answer to TS maintainers, though to users it apparently is. Furthermore this issue is an attempt to follow-up explicitly on the second half of Ryan's request to file something concrete going forward, though I imagine this issue will also be summarily buried as this team appears to have only one tool in its toolbox, namely denial for the sake of convenience.

2020: Appending .js extension on relative import statements during Typescript compilation (ES6 modules) (StackOverflow)

Same situation we've seen: User has a valid input program, can't get a valid output program. The answerer prudently notes:

If the compiler simply generated extension-less output files it would also solve the issue. But, would that also somehow violate the design principle regarding URI rewrites? Certainly, in that case there could exist other design principles to defend the position! But wouldn't such stubbornness only help to further validate the adamancy or ignorance of the TS team on this issue?

2022: "module": "node16" should support extension rewriting (#49083) (by @arendjr)

This is the most recent issue that covers this problem. Mr. van Beelen is certainly a lot more generous and forgiving with his wording than I am, though he is equally rewarded with stark denialism from Ryan.

We don't rewrite target-legal JavaScript code to be different JavaScript code from what it started with, no matter the configuration.

We won't even get into the fact that import "./a" is not an ES2015 "target-legal" import statement, as it's just an egregiously incompetent claim. It received about 60 downvotes.

Mr. van Beelen left several more long, thoughtful comments, generating over 100 positive responses, explaining much of the same things that I have covered here, that valid TS programs should become valid JS programs, though he did so with much more forgiving phrasing. I recommend everyone read his comments, and I found them very considerate - likely impractically so.

This is another thread that contains hundreds of comments, hundreds of positive reactions for the sensible position taken by users, and hundreds of negative reactions for this team's excuses. It is impossible to read it all, but it is worth skimming over and noticing that it, like every other issue raised regarding TS's inability to build valid output programs, was summarily closed for the sake of maintainer sanity to the detriment of several million codebases relying on TS.

2022: Add Compileroption to add Extensions to relative imports (#47436) (by @jogibear9988)

A PR to add non-breaking, opt-in compilerOption for adding .js extension to extensionless relative TS imports, which received about 30 positive reactions. Of course, it was summarily closed without a real review. No amount of attempts to contribute by the community have been well-received by this team.

🔎 Search Terms

  • superset
  • ESM
  • imports

🕗 Version & Regression Information

TypeScript was initially released in 2012, and it has not been an ES superset since the release of ES2015, which require imports to have file extensions if they are not named modules.

⏯ Playground Link

Cannot repro in playground, requires multiple entry points. Minimum repro: https://replit.com/@ctjlewis/ts-esm-superset

Throws ERR_MODULE_NOT_FOUND because ./a is not a valid ES import specifier.

💻 Code

N/A, see repro and notes in this issue.

🙁 Actual behavior

The valid TS-ESM program T builds to an invalid program T'.

🙂 Expected behavior

The valid TS-ESM program T should build to a valid program T'. Valid TS imports should always compile to valid ES imports for the given target.

@fatcerberus
Copy link

I stopped reading here:

The output program T' will throw according to the ES spec.

The ECMAScript specification says nothing about the meaning of module specifiers. Module resolution is the host’s responsibility and is therefore implementation-defined. If the TS compiler can resolve an import but your runtime can’t, that just means your project is incorrectly configured for the target runtime environment and doesn’t say anything about ES compliance.

If you want to argue this behavior is suboptimal, that’s fine, but you can’t use spec compliance as a bludgeon here.

@Josh-Cena
Copy link
Contributor

Josh-Cena commented Aug 29, 2022

Sigh so many people thinking that explicit extensions and full paths are ES requirements is really Node's fault. They could well keep the CJS resolution mode and make everyone happy. (Same goes for JSON modules—soon people will think assert { type: "json" } is required for JSON modules, just because Node is paranoid about aligning with browsers these days.)

Also, if you use ts-node, you could well make your code run with .js extension imports via the resolver—it's host-defined behavior at the end of the day.

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Aug 29, 2022

Module resolution is the host’s responsibility and is therefore implementation-defined

This is technically true (though in practice, pedantic).

The ECMAScript specification says nothing about the meaning of module specifiers.

This is not true:

Multiple different referencingScriptOrModule, specifier pairs may map to the same Module Record instance. The actual mapping semantic is host-defined but typically a normalization process is applied to specifier as part of the mapping process. A typical normalization process would include actions such as alphabetic case folding and expansion of relative and abbreviated path specifiers.

ECMAScript 2023 § 16.2.1.7

We can go round and round on this if you want, but whether you're in the browser, you're in Node, or any other major host of note, in pure ES, import "./file.js" will be valid output and import "./file" will not be.

@Josh-Cena
Copy link
Contributor

"Include actions such as" is extremely handwavy. It's neither normative nor actually specifying any algorithmic steps. It says nothing about import "./file" being invalid or import "./file.js" being valid.

@fatcerberus
Copy link

An example of “typical” use does not constitute normative text.

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Aug 29, 2022

Stockholm Syndrome ITT.

@ctjlewis
Copy link
Contributor Author

We've had thousands of people confused and upset over this for the last 5 years because their valid input programs end up failing to generate valid output.

On the basis of that alone you're wasting your time, you're not going to convince me that doing nothing is the appropriate solution, thanks for the comments though.

@fatcerberus
Copy link

fatcerberus commented Aug 29, 2022

On the basis of that alone you're wasting your time, you're not going to convince me that doing nothing is the appropriate solution, thanks for the comments though.

Like I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here.

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Aug 29, 2022

Like I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here.

OK, fine, I take your point with the limited note that it doesn't say nothing about it, and also point to implementations in practice. There's enough dishonesty in this whole issue to go around. But yes I will grant you that the spec does not explicitly specify this - we'd probably agree that it could and should.

Though, to be clear, import "./a" is not even target-legal for Node target. So I mostly dispute that there's a substantive issue with my statement.

@quantuminformation
Copy link

quantuminformation commented Aug 29, 2022 via email

@jcalz
Copy link
Contributor

jcalz commented Aug 29, 2022

I'm probably going to regret wading into this, but I wouldn't frame this issue in terms of language supersets, if the goal is to change how import statements are emitted.

"TS is a superset of JS" means that every valid JS program is also a valid TS program, in a particular sense. It basically just means that a valid JS program should be emitted as-is (or possibly downleveled). It's quite a weak statement. The only violations of this I'm aware of are things like parsing edge cases as in #33639.

It doesn't imply that every valid TS program will be emitted as a valid JS program, although obviously such behavior is important. It might not be desirable to emit some line of TS as-is to JS, but doing so doesn't violate "TS is a superset of JS".

@RyanCavanaugh RyanCavanaugh added the Discussion Issues which may not have code impact label Aug 29, 2022
@magic-akari

This comment was marked as duplicate.

@SerLizar
Copy link

Like I said, if you want to argue the current behavior is suboptimal, that's fine and I'd even be likely to agree with you. But trying to using the specification as a bludgeon is dishonest here.

Let's forget about the specification then, given that those using TS dislike the current behaviour, and you are likely to accept it is suboptimal. How about we finally change it!

I really don't care if I have to use a config file and CLI argument at the same time to make it work.

It doesn't imply that every valid TS program will be emitted as a valid JS program, although obviously such behavior is important.

If you describe TS as a superset of JS, this is correct (in TS is JS with extra stuff, some of that could make it incompatible). However, the problem is that:

TypeScript adds optional types to JavaScript that support tools for large-scale JavaScript applications for any browser, for any host, on any OS. TypeScript compiles to readable, standards-based JavaScript.

If it should work in any browser, changing './foo' to './foo.js' is desirable. Unless you want to take the battle to the browsers, changing the current behaviour of TS looks like the way to go.

@avin-kavish
Copy link

The whole idea of TypeScript is to add static types on top of JavaScript, not to be a higher-level language that builds to JS.

Not a fan of this backtracking either, 5 years ago this was not the rhetoric that brought me onto TS. TS offered much more, it provided design constructs that were lacking in js and also more expressive syntax (which eventually got added to js). But mostly, the design constructs of access modifiers, statics, interfaces and classes made it possible to design large systems without breaking a sweat. Maybe the differences in the ts implementation and eventual es standard is now deterring the team from adding new features to the language? Case in point, # vs private. legacy decorators. different field initializers. So it looks like the goal post was (silently) moved? Maybe it is also to appeal to the masses of javascripters? I don't believe this is entirely good. TS used to show what JS could be. Now only babel can do that, but new syntax that compiles through babel is not compatible with the ts language server. So users have to chose between new features with babel on js vs ts with current features. Also babel is limited to whatever is proposed to Tc39, which is a very small subset of what a language can be. Just look to kotlin, swift and even recent versions of c# for inspiration. Languages have evolved a lot in the past few years and TS hasn't moved much in terms of non-type features. But type features dissapear at compile time and don't add any value at runtime. So I think excessive type-safety is overrated, moderate type-safety is good. As one wise medium user once said to me, "a sound type system does not maketh a language". TS is not forward looking anymore, it's only making what works in js typesafe. I think that was important for full adoption. But I don't think it should be the only goal. I think it should provide a superset of js.

Regarding this particular mechanic, I mean we have a flag that says "allowESModuleInterop" that transforms imports. So I don't see why adding another flag, "suffixESImportsOrWhatever" that transforms imports in a slightly different way would be too much of a stretch to imagine. Whether it can accurately do that though, while remapping things like index imports, I don't know. I think this whole add-.js-at-the-end fiasco is completely unnecessary trouble. Why did we go from nice imports to ugly imports again? Sounds like a step back for mankind.

I'm solutions-oriented though, most times the fastest path to a solution is to write your own code, like below.

Solution

Problem: I want TypeScript to automatically add .js extensions to my imports.

In cases like this I use ttypescript, it basically plugs into the compiler api and lets you run arbitary code as it visits nodes in your AST. Scroll down the readme and you will find a plugin that does exactly what you want, i.e. add .js extensions to your relative imports.

Also, I wrote this hacky but working script to do a one time migration of my code base.

import glob from 'glob'
import { readFileSync, writeFileSync } from 'fs'

glob.glob('src/**/*.ts', (err, matches) =>
  matches.forEach(match =>
    writeFileSync(
      match,
      readFileSync(match, { encoding: 'utf-8' })
        .split('\n')
        .map(l =>
          l.startsWith('import') && l.includes("'.") && !l.endsWith(".js'")
            ? l.replace(/'$/, ".js'")
            : l,
        )
        .join('\n'),
      { encoding: 'utf-8' },
    ),
  ),
)

(js is so expressive these days, you can do all that in a single statement.)

Also, shoutout to tst-reflect which through ttypescript gives much better reflection than anything else we have "officially" atm.

@ctjlewis
Copy link
Contributor Author

So I don't see why adding another flag, "suffixESImportsOrWhatever" that transforms imports in a slightly different way would be too much of a stretch to imagine.

Exactly correct. There is no obstacle to adding non-breaking, opt-in support for this.

Noted in the issue, someone actually added this flag and sent in a PR, but it was summarily closed. It is a striking example of organizational deficiency in software. I suspect a single decision-maker or a small group of them are the only obstacles to un-breaking the compiler in this respect.

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Aug 29, 2022

@jcalz And it remains a superset if a mapping from the superset to a subset is not a valid subset?

Really, that's where we're going with this to justify emitting broken programs?

@jogibear9988
Copy link

@ctjlewis i added an option in my pull req: #47436

@ctjlewis
Copy link
Contributor Author

@ctjlewis i added an option in my pull req: #47436

I know, I mentioned and linked it - thanks for sending that in, sorry it got buried. That was a good contribution and I appreciate it.

@jogibear9988
Copy link

and it could also be solved with custom transformers, but they are not public. don't know why they exist at all when they are not exposed

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Aug 29, 2022

Worth highlighting Ryan's comment from the linked response to that PR too:

Maybe we have failed, I don't know, but we're going to keep going on this path either way.

Awesome. We're plugging our ears and yelling "imports are pure JS! imports are pure JS!" even though we can write TypeScript-specific, non-target-legal ./relative imports which are not pure JS and do need to be transformed.

Yes, you are failing, but the beauty of software is we can always fix it. I'm still holding my breath.

@jcalz
Copy link
Contributor

jcalz commented Aug 29, 2022

@jcalz And it remains a superset if a mapping from the superset to a subset is not a valid subset?

Yes.

Really, that's where we're going with this to justify emitting broken programs?

I’m not really trying to justify anything. I’m pointing out that this issue isn’t about what the title is claiming to be about. And that talk of supersets is a distraction from your actual concern here.

By the way, I’m not a member of the TS team or anything, so I certainly wouldn’t be in a position to authoritatively justify TS emitting broken programs, working programs, or mp3s of flugelhorn music.

@nicolo-ribaudo
Copy link

nicolo-ribaudo commented Aug 30, 2022

You know that import { foo } from "./foo" works perfectly in the browser, right? Your server just has to respond to GET /path/to/foo with a file that has text/javascript mime type. This is identical to how it has to respond to GET /path/to/foo.js with a file that has text/javascript mime type when using import { foo } from "./foo.js".

@richardkazuomiller
Copy link

I shared my opinion about how I would have liked TypeScript ESM imports to work on other issues and I didn't get everything I wanted, but what we have now works and many projects are finally able to migrate to ESM so I'm happy nonetheless.

@richardkazuomiller
Copy link

Actually, my original complaint was that TypeScript didn't throw any errors or show warnings if you wrote imports the CJS way and compiled to ESM. Now it does, so I kind of did get everything that I wanted regarding things that are most important in my opinion.

@egasimus
Copy link

egasimus commented Sep 17, 2022

Okay, let's say I have three files, foo.ts, bar.ts, and baz.ts.

// foo.ts
import { baz } from './bar.js'
// bar.ts
export { baz } from './baz.js'
// baz.ts
export function baz () { /* ... */ }

Let's also say that I use a framework which compiles and loads TypeScript on demand, without ever emitting JavaScript.
So, the files bar.js and baz.js never exist. In other words, for things to work, I need to import from non-existent file.

  • Generally speaking, to whom does this make sense at all?
  • More specifically, is TypeScript smart enough to know that when I write bar.js I mean bar.ts, even though bar.js never exists at all? And why would it have to work in this completely counterintuitive and convoluted manner?

I understand that Node is somewhat at fault here for nerfing ESM/CJS backwards compatibility in a way that looks very much the worst of both worlds. Why doesn't TypeScript:

  • (a) fix the broken behavior (by adding extensions in node16 mode)

instead of:

  • (b) making it doubly broken (by enforcing the wrong extension in node16 mode)
  • and (c) to add insult to injury, handwaving it all away (with the giant "you'll get used to it" that is the official documentation on the subject?)

Now let's say I have the above three files, but I import without extensions (import { baz } from './bar', etc). The framework loads everything correctly, watches files, reloads on edit, even does nifty in-place hot reloading, etc.

Now let's say I want to publish part of my repo as a library in the form of a portable ES module on NPM. I run TSC, which emits foo.js, bar.js, baz.js. Now these files exist but contain the (invalid according to Node) extensionless imports. So I diligently add the weird .js extensions in the original TS source and what happens?

👏 Reloading 👏 stops 👏

Now I have to re-run TSC on every change, where I previously didn't, because bar.ts and baz.ts have fallen off the dependency graph and the dev server only sees the .js variants which will go out of date the first time I change the original sources.


Again, please point out the person to whom this affront makes sense, and let them explain how this is not an arbitrary speed bump introduced into the otherwise generally sane modularity and packaging workflow that the JS community had largely settled on before you bought your pseudo-superset a place the ecosystem. I agree that what the spec says is beside the point, because TypeScript is not only non-compliant, it simply generates code that doesn't work and, to make it work, requires the programmer to write code that doesn't make sense.

And it requires developers to wrap their heads around numerous implementation details that simply did not exist previously. In my experience, the average JS dev I've met already has a hard time wrapping their head around how Promises and async/await are the same thing, you're just making this harder on people and businesses trying to solve problems by using your platform, which of course is widely lauded as the next best thing since sliced bread. So you can't even choose not to use it. Congrats, you've "won".

Yall really overdid it with this one, folks. This particular abstraction is so leaky that if it was a ship it'd be sinking. I'd like to say that it ruined my day - only it's not just one day and it's not just mine. You know what they say, "don't look for malice where things can be explained by incompetence". To which I ask - what is malice, in our industry, other than organized incompetence?

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Sep 17, 2022

@egasimus Go off king. Also, sadly, you'll find several people ITT willing to suck up and make excuses for this problem that could've been resolved years ago. Stockholm Syndrome is a hell of a drug.

@ctjlewis
Copy link
Contributor Author

ctjlewis commented Sep 17, 2022

You know that import { foo } from "./foo" works perfectly in the browser, right? Your server just has to respond to GET /path/to/foo with a file that has text/javascript mime type. This is identical to how it has to respond to GET /path/to/foo.js with a file that has text/javascript mime type when using import { foo } from "./foo.js".

OK, so you need to customize the behavior of the server rather than serve a simple filesystem. Proved the point for me.

iT wOrKs pErFecTlY iN tHe bRowSer (with my super special server configuration) - lmao.

@jcalz And it remains a superset if a mapping from the superset to a subset is not a valid subset?

Yes.

Does it remain a superset that can map to the subset of it doesn't map to the subset, since TS literally is designed to meet this definition as we have already covered and as is mentioned in all of their documentation?

We can ad hoc it as much as we want, it doesn't meet its technical goals in an ESM context. No amount of cope will change this.

ericvergnaud added a commit to ericvergnaud/antlr4 that referenced this issue Sep 18, 2022
parrt added a commit to antlr/antlr4 that referenced this issue Dec 21, 2022
* Fix CMake syntax for variable expansion

When using variables to compare (like in if clause) the variable
shouldn't be quoted. More details can be found at the link below:

https://cmake.org/cmake/help/latest/command/if.html#variable-expansion

Signed-off-by: HS <hs@apotell.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* initial commit

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* renamed for clarity

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* renamed for clarity

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* able to locate antlr4 runtime using ts-node, missing types

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* progressing

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* able to 'run' a test. It fails but it compiles and resolves!

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* reflect refactored runtime

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* able to run RecursiveLexerRuleRefWithWildcardPlus_1 test locally

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* passes LexerExec tests in IntelliJ

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* make ATN private

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ignore same tests as JavaScript

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* compiles Parser and Lexer bu local run fails

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ParserExec.TokenOffset test successful in IntelliJ !

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Progressing, passing 131 of 348 tests

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* pass 327 tests out of 348

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* more successful tests

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* 333 successful tests out of 348

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* all tests pass except 7 caused by #3868

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update getting-started doc

Signed-off-by: nicksxs <nicksxs@hotmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* add blank github action file for hosted CI

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update getting started document to say java 11

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Revert "update getting started document to say java 11"

This reverts commit 1df58f7.

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* add C# book code links

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Add Jim/Ken to readme

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Update Swift Package to support static library

Add static library distribution in SPM

Signed-off-by: Hell_Ghost <dev.hellghost@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Update Package.swift

Signed-off-by: Hell_Ghost dev.hellghost@gmail.com
Signed-off-by: Hell_Ghost <dev.hellghost@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Add caching support for maven & dependencies. Also, include caching for
cpp builds using actions/ccache.

Builds are more reliable (avoids the archive.apache server which
intermittently reports timeouts) and also significantly improves the
overall builds times (down from 46 mins to 28 mins).

The slowest part of the build now is the Windows+cpp builds because
there is no reliable cache implementation yet. MacOS+cpp (65% cache hit) is
also relatively slow compared to Ubuntu+cpp (99% cache hit).

Signed-off-by: HS <hs@apotell.com>
Signed-off-by: Terence Parr <parrt@antlr.org>

# Conflicts:
#	.github/workflows/hosted.yml
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* use snap to install go 1.19

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* grr...install snap

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ugh. start snap

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ugh. start snap

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ugh. cant get snap to install go

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* try downloading golang with curl

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Issue #3823: Temporarily disable a few tests on CI

The tests are currently failing. The underlying issues have been fixed
on dev and so the builds will be turned back with the next release.

Signed-off-by: HS <hs@apotell.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update actions status badge

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update getting started document to say java 11

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Revert "update getting started document to say java 11"

This reverts commit 3591ee0.

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update getting-started doc

Signed-off-by: nicksxs <nicksxs@hotmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* make getValue visible to external profiler tools.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Fix #3508: Document the $parser attribute and its use in target-agnostic grammars.

Signed-off-by: Ross Patterson <ross.patterson@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Add accessor to IntervalSet for intervals

Signed-off-by: James Taylor <jamestaylor@apache.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Remove libuuid dependency from C++ runtime

libuuid and its headers are not referenced anywhere, so remove it.

Signed-off-by: Bryan Tan <bryantan@technius.net>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Add `@SuppressWarnings("CheckReturnValue")` to prevent error_prone lib errors.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: Fixes for #3718

  o Implement collections with generics to solve hash collisions
  o Fix type casting in LL start parser simulation optimization
  o General minor tidy ups

Acknowledgements to @kaby76 for help with tracing errors

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3718 Revert accidental keyboard error in Java target

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3718 Correct DFAState index in Lexer ATN Simulator

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3718 Fix go runtime test runners

With older versions of go, there was no good way to tell the compiler to use your local
development copy of a particular package instead of the one installed in GOPATH/src/...

However, we are now using modules, which allows us to tell the compiler that instead of
a module downloaded to GOPATH/pkg, to use a local copy on disk.

Hence this change removes the need to copy the whole of the go installation to a
tempoorary location, then put the antlr go runtime in to the go installation as if it was
part of the compiler. Hence the execution time for the go tests is now faster than before.

This works because when the generated code is placed in the temporary location, we create
a go.mod file for it, tell the module to replace the online module for the go runtime with
the local copy on disk, then ro a go mod tidy to add the dependencies from the code (which
avoids network access, so it is instant), which adds the ANTLR dependency itself (which is
then replaced at compile time).

All go runtime tests now pass.

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Rm remote github actions; hosted seems to work

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* install golang with curl; go was missing from dev

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: Rework of all Hash() and Equals() methods - implement generic collections

 - Implement new collections using generics that implement the functionality
   required by the Java runtime in a more idiomatic Go way.
 - Fix Hash() and Equals() for all objects in the runtime
 - Fix getConflictingAlts so that it behaves the same way as Java, using a
   new generic collection
 - Replaces the use of the array2DHashSet, which was causing unneeded memory
   allocations. Replaced with generic collection that allocates minimally
   (though, I think I can improve on that with a little analysis).

Jim Idle - jimi@idle.ws

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3718 Correct DFAState index in Lexer ATN Simulator

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* feat: Reduce initial memory allocations for collections

  - Many small collections are created at runtime, the default allocation for
    maps, even though small, still requires memory. Specifying a very small
    initial allocation prevents unnecesary allocations and has no measurable
    effect on performance. A small incremental change.

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3758 Allow for string being a keyword and fix go template to use escapedName

  - The go template was ignoring the use of escapedName in many places and was
    not consistenet with the Java version.
  - Added 'string' to the list of reserved words for the Go target

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3758 Add go.sum to the repo

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #3758 Ensure that standard runtime extensions are included in go.mod

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #2826  Go template is incorrect for dynamic scopes

closes #2826
obviates PR #3101

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #2016 - Generate correct iGo code for lists in a grammar, such as `label+=arg+`

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* feat: Bump poms to use 4.11 Snapshot

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* turn off Golang test at circleci for now

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Replace smart-quote with single-quote in code examples

Signed-off-by: Tim McCormack <cortex@brainonfire.net>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Augment error message during testing to include full cause of problem.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Augment error message during testing to include full cause of problem. (round 2 to avoid null ptr)

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Cpp: Link to threads library

As detailed in #3708, it is necessary to link against the (p)threads
library in order to be able to use std::call_once without producing
linker errors.

Since this function is frequently used in ANTLR's cpp version, this
commit ensures that the respective library is always linked against in
order to avoid this error, even if downstream users are not explicitly
linking against an appropriate threads library.

Fixes #3708

Signed-off-by: Robert Adam <dev@robert-adam.de>

Co-authored-by: Bryan Tan <Technius@users.noreply.github.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* add test for #2016 and fix java.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ensure all targets have the appropriate argument list for the template causing the problem.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Fix other targets

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix format

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* add check that $args is a list

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* change made by @lingyv-li to fix bug in DART exposed by this test.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix AssertIsList in multiple targets. Go doesn't pass test and has no AssertIsList so I'm dropping that test from the Go test suite.

How did a comment to the C++ runnerFor my future reference as to how to build things from the command line.

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* C++ gets an exception with this test so I'm turning it off. See #3845

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* feat: #3840 Move Go to version v4.11.0

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* feat: #3840 Create the v4 version of the Go runtime

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* feat: Create the v4 runtime layout for the Go runtime, ready for release tagging

Note that the vast majority of the changes here are just copying the runtime file in to
the /v4 subdirectory so that we can support legacy projects that use GOPATH only, as well
as users that can use go modules. At a later release, we will delete the default path, and move
the v4 subdirectory back to the top level. But, we cannot do that on this release.

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Reenable go tests on CircleCI

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Fold constants in generated code for all runtimes

Go: getInlineTestSetWordSize 32 -> 64
Dart: get rid of BigInt
Swift: optimize TestSetInline
Python: fixes #3698
JavaScript: fixes #3699

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Use int literals instead of refs for Python and JavaScript

Update getMultiTokenAlternativeDescriptor test

fixes #3703

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update release doc for Go version numbers

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix: #2016 Fix Go template list reference, go runtime and got test template for list labels

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* feat: Add a deprecation message to the existing v1 module

Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Split tool and runtime tests for GitHub workflow

Build only necessary modules for tests

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Remove not used methods from FileUtils (runtime tests)

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Update dependencies of antlr4-maven-plugin

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Update jUnit: 5.8.2 -> 5.9.0

Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Fixes #3733; update ST4 so it uses proper ANTLR 3

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* tweak doc

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Set to 4.11.0 not 4.11 in poms

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* [maven-release-plugin] prepare release 4.11.0

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* [maven-release-plugin] prepare for next development iteration

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Damn. java target said 4.10.2 not 4.11.0

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* roll back to 4.11.0; made mistake

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* roll back to 4.11.0; made mistake

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* [maven-release-plugin] prepare release antlr4-master-4.11.0

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* [maven-release-plugin] prepare for next development iteration

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* use build and twine to publish source and wheel

Signed-off-by: Qijia Liu <liumeo@pku.edu.cn>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* tweak doc

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* tweak c++ build script to make Mac binaries with cmake/make

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* tweak release doc

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* tweak code / doc related to bad previous release

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* [maven-release-plugin] prepare release 4.11.1

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* [maven-release-plugin] prepare for next development iteration

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* clean up deploy c++ source script

Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* cleanup code generation

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* don't initialize default param values twice

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* add missing field

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* update codegen template for 4.11.1

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* support new param: Parser

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix template for 4.11

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* default export Listener and Visitor

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* also default export parser and lexer

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* all tests pass except 7 caused by #3868

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix issues

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* make it easy to break

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix #3868

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* ALL TESTS PASS!!!!

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* cross fingers with CI

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* try fixing broken go tests

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Try fix typescript CI

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* disable cpp for now

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix broken config

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* try fix macos gh build

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* improve speed by caching node_modules

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* no longer using ts-node

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix all tsc warnings

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* try fix MacOS CI

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* CI errors seem random, reactivate ubuntu

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Disable node_modules caching, which seems to randomly fail in CI

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* don't delete symlink contents on windows (java bug with is SymbolicLink ?)

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* fix broken windows CI

Signed-off-by: ERIC-WINDOWS\ericv <eric.vergnaud@wanadoo.fr>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* verify windows ci

Signed-off-by: ERIC-WINDOWS\ericv <eric.vergnaud@wanadoo.fr>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* Revert "verify windows ci"

This reverts commit 770d821.

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* reinstate full CI

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* manually merged

* manually merge

* fix merge

* fix broken template

* add template for invoking context list

* fix typo

* fix test templates

* Add code of conduct but with a different name since I do not like that name

Signed-off-by: Terence Parr <parrt@antlr.org>

* Update C# release instructions

* Tweak code of conduct

Signed-off-by: Terence Parr <parrt@antlr.org>

* Bring back the Package.swift in the project's root

Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>

* swift-target.md: fix SPM installation instructions

Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>

* the scope (parser or lexer) in @parser::header was dropped, so keep track of it and only include @Header in Listener and Visitor code

Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>

* drop workaround in favor of #3878

* drop cache usage since it fails in CI

* #3878 was missing some scenarios

* fix issue when deleting test folder

* fix warnings

* drop duplicate behavior

* drop alien 'abstractRecognizer' property

* drop alien property 'channels'

* fix various codegen issues

* change import

* restore js extensions, see microsoft/TypeScript#50501

* use consistent inheritance

* more API

* more API stuff

* fix typo

* fix typescript exports

* use ts-node to run typescript tests

* webpack runtime before linking

* fix exec paths on windows

Signed-off-by: ERIC-WINDOWS\ericv <eric.vergnaud@wanadoo.fr>

* fix failing tests

* fix a few import issues

* merge typescript-target with latest dev

* runs Java and JavaScript tests after merging typescript-target

* merge test template

* skip unsupported test

* fix template prototype

* fix missing merge

* bump typescript beta version after rebase

* update docs

* rollback unwanted changes

Signed-off-by: HS <hs@apotell.com>
Signed-off-by: Eric Vergnaud <eric.vergnaud@wanadoo.fr>
Signed-off-by: nicksxs <nicksxs@hotmail.com>
Signed-off-by: Terence Parr <parrt@antlr.org>
Signed-off-by: Hell_Ghost <dev.hellghost@gmail.com>
Signed-off-by: Hell_Ghost dev.hellghost@gmail.com
Signed-off-by: Ross Patterson <ross.patterson@gmail.com>
Signed-off-by: James Taylor <jamestaylor@apache.org>
Signed-off-by: Bryan Tan <bryantan@technius.net>
Signed-off-by: Jim.Idle <jimi@gatherstars.com>
Signed-off-by: Tim McCormack <cortex@brainonfire.net>
Signed-off-by: Ivan Kochurkin <kvanttt@gmail.com>
Signed-off-by: Qijia Liu <liumeo@pku.edu.cn>
Signed-off-by: ERIC-WINDOWS\ericv <eric.vergnaud@wanadoo.fr>
Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>
Co-authored-by: HS <hs@apotell.com>
Co-authored-by: nicksxs <nicksxs@hotmail.com>
Co-authored-by: Terence Parr <parrt@antlr.org>
Co-authored-by: Hell_Ghost <dev.hellghost@gmail.com>
Co-authored-by: Ross Patterson <ross.patterson@gmail.com>
Co-authored-by: James Taylor <jamestaylor@apache.org>
Co-authored-by: Bryan Tan <bryantan@technius.net>
Co-authored-by: Jim.Idle <jimi@gatherstars.com>
Co-authored-by: Tim McCormack <cortex@brainonfire.net>
Co-authored-by: Robert Adam <dev@robert-adam.de>
Co-authored-by: Bryan Tan <Technius@users.noreply.github.com>
Co-authored-by: Ivan Kochurkin <kvanttt@gmail.com>
Co-authored-by: Qijia Liu <liumeo@pku.edu.cn>
Co-authored-by: Nikolay Edigaryev <edigaryev@gmail.com>
@RyanCavanaugh
Copy link
Member

RyanCavanaugh commented Mar 22, 2024

The documentation already explicitly dis-recommends moduleResolution: "node"

@nurse-the-code
Copy link

Thanks for the responses!

The solution I went for was using the create vite vanilla TypeScript template. Under the hood it uses Rollup. In the process, I also discovered pnpm. Overall, I am pretty happy with my initial results.

I have a functioning starter project, and I can easily transpile the following TypeScript:

import Foo from "./foo.ts";

into functional JavaScript that does what I need it to do.

For those interested, here are links to info on Vite, Rollup, and pnpm.

I also checked out @arendjr's suggestions of Deno and Bun. They looked like interesting projects, but I was a bit nervous to move that far away from the npm/node ecosystem. Choosing 'vite', 'rollup' and 'pnpm' seemed like a more moderate choice.

@ekwoka
Copy link

ekwoka commented Mar 23, 2024

Vite actually uses esbuild for the ts transformation, not rollup.

@arendjr
Copy link

arendjr commented Mar 23, 2024

Vite actually uses esbuild for the ts transformation, not rollup.

I think that’s a bit misleading, because they use rollup for module resolution, which is what we’re discussing here. For syntax transpilation it can use either SWC or ESBuild.

@MiirzaBaig
Copy link

Was it able to be resolved?

@RyanCavanaugh
Copy link
Member

See https://github.com/microsoft/TypeScript/wiki/FAQ#module-specifier-rewriting

@ctjlewis
Copy link
Contributor Author

Was it able to be resolved?

The solution is use Bun, and not depend on the compiler built by the TypeScript team.

@RyanCavanaugh RyanCavanaugh closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2024
@guest271314
Copy link

We can go round and round on this if you want, but whether you're in the browser, you're in Node, or any other major host of note, in pure ES, import "./file.js" will be valid output and import "./file" will not be.

This might be of interest to you Intercepting and handling arbitrary static and dynamic Ecmascript import specifiers, protocols and file extensions.

We have WICG Import Maps now. We can do this https://github.com/guest271314/webbundle/blob/browser/index.html#L5-L18

  <script type="importmap">
    {
      "imports": {
        "Buffer": "https://gist.githubusercontent.com/guest271314/08b19ba88c98a465dd09bcd8a04606f6/raw/f7ae1e77fb146843455628042c8fa47aec2644eb/buffer-bun-bundle.js",
        "base32-encode": "https://esm.sh/base32-encode@2.0.0",
        "cborg": "https://esm.sh/cborg@1.10.2",
        "commander": "https://esm.sh/commander@4.1.1",
        "mime": "https://esm.sh/mime@2.6.0",
        "to-data-view": "https://esm.sh/to-data-view@2.0.0",
        "wbn": "https://esm.sh/wbn@0.0.9",
        "wbn-sign-webcrypto": "./wbn-sign.js",
        "zod": "https://esm.sh/zod@3.22.4"
      }
    }
  </script>

and this https://gist.github.com/guest271314/78372b8f3fabb1ecf95d492a028d10dd#the-code

<script type="importmap">
  {
    "imports": {
      "test/file": "./x.json",
      "test/sub/file": "./z.json"
   }
 }
</script>
<script type="module">
  import x from "test/file"
  with {
    type: "json"
  };
  import z from "test/sub/file"
  with {
    type: "json"
  };
  console.log(
    new TextDecoder().decode(new Uint8Array([x, z])), 
    import.meta.resolve("test/file"), 
    import.meta.resolve("test/sub/file")
  );
</script>

and this

deno run --unstable-byonm --import-map=import-map.json -A index.js

The solution is use Bun, and not depend on the compiler built by the TypeScript team.

My solution is to indeed bundle all TypeScript source code I use in my programs, or want to test or break, or use in the browser or just read, to JavaScript with bun build or deno bundle while it's still around https://gist.github.com/guest271314/08b19ba88c98a465dd09bcd8a04606f6, https://gist.github.com/guest271314/1e8fab96bd40dc7711b43f5d7faf239e.

My broader solution to use multiple JavaScript engines and runtimes for development. I don't entertain a preference for any. They all have features that the other doesn't have, and implement things differntly. E.g., Bun.stdin.stream() behaves differently from Bun.file("/dev/stdin").stream(); Deno's dynamic import() is not really dynamic when using static string specifiers; Node.js doesn't support import maps, nor does Bun.

The semantics of what constitutes ECMAscript conformity or not is ultimately a politically hierarchical decision, it's not a democratic process.

If you use TypeScript, use TypeScript. There's only one, that I am aware of, even though

Typescript Specifications version #15711

There are about four people on the planet who can accurately update the spec and all of them are quite busy already.

If you are using JavaScript I would strongly suggest, if only for an experiment, if not a standard practice or policy, which is what I do, trying to run your source code using multiple JavaScript(/TypeScript) runtimes, constantly. There are few dozen, at least A list of JavaScript engines, runtimes, interpreters, to experiment with and develop with simultaneously, without enterI think what you'll notice is that they all (perhaps with the exception of Bellard's QuickJS) tend to omit this or that, extend this or that, not provide this or that capability, and might become resistant to developer feedback from the field sharing feedback the maintainers havn't thought of, or just flatly refuse to implement of see as "on-topic".

As to ECMAscript, and the minutae of what the technical writing means, and doesn't mean, some folks in the filed might have a different perspective, and different ideas. Top-down organizations might think they know best. The developer could decide to do their own thing, that leads to more stuff happening in the field, when the whole thing could be dealt with on a broader scale within the given entity/organization/specification body, where all stakeholders input MUST be included in the design pattern. As options.

I wouldn't mind ECMA-262 specifying reading standard input and standard output, as each engine and runtime implements that basic functionality differently. It's 2024, not 1995. Folks are running JavaScript - and now TypeScript - in all sorts of environments for multiple use cases. You would think STDIO would be interoperable and compatible. It ain't. So, there's diversity not by plan, rather by isolation, opinion, "Popularity".

You folks have a great day. Good luck!

@mrhyde
Copy link

mrhyde commented Jun 20, 2024

An easier way would be to use tsfix to handle imports.

@guest271314
Copy link

An easier way would be to use tsfix to handle imports.

At

compiled JavaScript files are valid ESM

import "file"

without a file extension is valid Ecmascript Module syntax.

The simplest solution that as I see it is using/incorporating WICG Import Maps into TypeScript, which in the browsers Chromium 128 and Firefox 128 implements, and Deno JavaScript/TypeScript runtime respectively; with this caveat Deno dynamic import("./exports") throws module not found for "exports.js" dynamically created in the script for Deno re dynamic imports, which dives into precisely what ECMA-262 writes out as to specifiers for Ecmascript Modules, and what it doesn't.

Then you can set your specifiers to be whatever you want - completely user-defined. Controversy and schism over. I don't think it is possible for anybody other than the core TypeScript maintainers to even speculate about whether or not TypeScript is ECMA-262 conformant.

If you are exclusively in Node.js world Node.js doesn't support import maps. Node.js is still using CommonJS as a default module loader.

@guest271314
Copy link

Re

This bug report will show that TypeScript is no longer an ECMAScript subset

I don't even get that far.

From my perspective TypeScript is an entirely different programming language from JavaScript.

For comparison Bun claims to somehow be comparable to Node.js. I start poking around, testing things until they break, and find Bun to be it's own thing; no HTTP/2; no upload streaming, and so forth. bun does have tooling for TypeScript/JavaScript, but it ain't node. node has issues, too. We have to use --experimental-default-type=module just to get to Ecmascript Modules being the default in the context, out of legacy CommonJS support, and so forth. That doesn't change the fact folks are initiating new projects using Ecmascript Modules, thus this issue.

So, I would use Deno if I was into TypeScript to eliminate the node variable of not supporting import maps or Ecmascript Modules by default. It's the same underlying V8 JavaScript/WebAssembly engine.

@ctjlewis
Copy link
Contributor Author

@RyanCavanaugh - I would probably lock, you guys have made your position on it super clear and this was mostly a troll post to give people a place to vent.

@minecrawler
Copy link

this was mostly a troll post to give people a place to vent.

@ctjlewis while it may be a troll post in your opinion, it's about a problem real people have in real life in real projects, working for real corps, earning their very real money and having to spend so much time working around a problem introduced by convention and definition without any technical basis, which is maddening. For all practical purposes and to lessen everyone's burden, so much could be done here. And just in the recent weeks we've seen actual improvements on different sides of this problem.

Yes, not everything is on Microsoft, but TS is a standard in the industry, and having no solution here - at least a documented way of getting things done - is a problem. It's not being addressed, which is a no-go for such a widely used tech, and many workarounds have sprouted in the community which points to the problem being a pain-point.

Hence, closing this post will just lead to the problem popping up somewhere else in another issue, possibly with similar discussions to what we already have here. Something I'd like to see is an actionable plan to resolve this issue. Which doesn't mean that the MS team should just do whatever internet randoms tell them to, but decide on some steps to solve the problem in a way they see fit. Which at least should include a chapter in their tutorials addressing it and showing a recipe for common scenarios, like making a lib for different runtimes and module systems. A best-practice we can just copy and be done with it.

TypeScript is an entirely different programming language from JavaScript.

@guest271314 yes, TS is a different language than JS. They are still both based on ECMA-262 and TS has the explicit goal of supporting simple transpilation to JS and adding to it. See the official landing page for reference: "TypeScript is a strongly typed programming language that builds on JavaScript." At the same time, TS is an industry standard, and should try to play nice with other parts of the ecosystem. No one cares about perfect integration, however simple recipes on how to deal with common scenarios are a baseline.

@jogibear9988
Copy link

for me there are two solutions. Enable the transformer plugins, (like ttypescript does) or add somethink like I did in my pull: #47436

@guest271314
Copy link

@minecrawler

They are still both based on ECMA-262 and TS has the explicit goal of supporting simple transpilation to JS and adding to it.

ECMA-262 has some omissions. Observable when we dive in to the minutae.

Take Deno as an example. Deno explicitly supports TypeScript. No need for ts-node. If you are in to TypeScript you can run TypeScript out of the box on the command line or in .ts scripts.

Now, let's examine one impact of supporting static TypeScript programming in the dynamic JavaScript programming language that I just happened to encounter while creating a non-Node.js specific version of wbn-sign NPM package. Dynamic import module is not found when created after the application started. #20945 linking to Do not permission prompt for statically analyzable dynamic imports, specifically this comment

I also used to think the same, but it's essentially a given due to how TypeScript supports typings for statically analyzable dynamic imports. Stemming from there it becomes a requirement to preload such imports, which results in things like write("foo.ts"); await import("./foo.ts") being quirky. And it follows that the runtime net access doesn't occur so it shouldn't prompt.

Basically this ship has sailed from the perspective of build tools / static analysis. At this point I think it's viable to carry forward with these behaviours and suggest that users do const s = "./foo.ts"; await import(s) where they want to opt out.

The impact of that decision, per Deno authors based on the internal decision to statically compile dynamic ECMA-262 import() results in the technical fact that we can make Deno's import() consistently throw - where node, bun do not throw under the same conditions Deno dynamic import("./exports") throws module not found for "exports.js" dynamically created in the script

// Deno dynamic import("./exports") throws module not found for "exports.js" dynamically created in the script
// dynamic_import_always_throws.js
// References: https://www.reddit.com/r/Deno/comments/18unb03/comment/kfsszsw/ https://github.com/denoland/deno/issues/20945
// Usage:
// deno run -A dynamic_import_always_throws.js
// bun run dynamic_import_always_throws.js
// node --experimental-default-type=module dynamic_import_always_throws.js
import { open, unlink } from "node:fs/promises";
const runtime = navigator.userAgent;
const encoder = new TextEncoder();
try {
  const script = `export default 1;`;
  // deno
  if (runtime.includes("Deno")) {
    await Deno.writeFile("exports.js", encoder.encode(script));
  }
  // node
  if (runtime.includes("Node")) {
    const dynamic = await open("./exports.js", "w");
    await dynamic.write(script);
    await dynamic.close();
  }
  // bun
  if (runtime.includes("Bun")) {
    await Bun.write("exports.js", encoder.encode(script));
  }
  const { default: module } = await import("./exports.js"); // Raw string specifier
  console.log({ module });
  console.log({ runtime });
} catch (e) {
  console.log("Deno always throws.");
  console.log({ runtime });
  console.trace();
  console.log(e.stack);
} finally {
  console.log("Finally");
  // node, bun
  if (runtime.includes("Node") || runtime.includes("Bun")) {
    await unlink("./exports.js");
  } // deno
  else if (runtime.includes("Deno")) {
    await Deno.remove("./exports.js");
  }
}

Now, let's examine the ECMA-262 language for dynamic import() re specifiers. I don't see anywhere here [13.3.10.1 Runtime Semantics: Evaluation
ImportCall : import ( AssignmentExpression or 13.3.10.1.1 ContinueDynamicImport ( promiseCapability, moduleCompletion ) in the steps at where dynamic import("./path") inside a running script should throw, per the specification. Some might disagree and claim that runtimes are free to do or not do whatever they want for dynamic import() specifiers.

Deno is only doing this, per the above Deno issue, in order to accomodate TypeScript static compilation for Deno-specific bahaviours. The effecttive result is Deno's implementation of ECMA-262 dynamic import() is not dynamic.

Deno does support WICG Import Maps. Node.js doesn't. Import Maps won't help when we are deliberately running dynamic import() inside an active script.

So TypeScript folks have to reconcile that you can't have a completely static scripting language and support dynamic scripting.

Since TypeScript folks are interested solely in static scripting, bring WICG Import Maps into TypeScript, instead of some custom TypeScript solution that departs further from the dynamic JavaScript programming language.

I'll leave it to the reader to determine whether or not Deno is in conformance with ECMA-262.

@nicolo-ribaudo
Copy link

The first bullet point of https://tc39.es/ecma262/multipage/ecmascript-language-scripts-and-modules.html#sec-HostLoadImportedModule ("or a throw completion") is what makes Deno compliant. ECMA-262 doesn't have a concept of URLs, paths, and file systems at all: it's einteirely implementation-defined.

@guest271314
Copy link

@nicolo-ribaudo

The first bullet point of https://tc39.es/ecma262/multipage/ecmascript-language-scripts-and-modules.html#sec-HostLoadImportedModule ("or a throw completion") is what makes Deno compliant.

So that necessarily means in this case Deno behaviour that throws and everybody else's implementation other than Deno that does not throw for raw string specifiers are both compliant?

See what I mean about the vagueness of ECMA-262 in this case?

@nicolo-ribaudo
Copy link

So that necessarily means in this case Deno behaviour that throws and everybody else's implementation other than Deno that does not throw for raw string specifiers are both compliant?

Yes

See what I mean about the vagueness of ECMA-262 in this case?

And there is a good reason for that: ECMAScript is designed to work everywhere, while I/O is platform-specific. The only I/O that ECMA-262 is aware of is datetime and randomness.

ECMAScript can run on platforms that use URLs, that use filesystem paths, or even on platforms that don't have a concept of a hierarchical filesystem. For this reason, what's the meaning of the string you pass to imports (both static and dynamic) is entirely platform-defined, and individual platforms have to decide what's the meaning of that string and if it corresponds to a module or not.

@guest271314
Copy link

Yes

I have never observed any specification, standard where throwing is optional, at implemenmter discretion. Where one implementer deliberately throws in one case, other implementers do no throw and bother are conformant to the specification.

Never seen that on any specification for an actual build plan, either. Where having a 10 foot retaining wall is optional. Never seen that in law, either. Where a statute or administrative regulation has the legislative or administrative intent to fine one person, not fine another person for the same conduct or activity.

Nonetheless people will no doubt argue that one implementer can throw for impoirt("./exports.js") another implementer of the same specification can not throw for import("./exports.js") - and both are equally comformant to the same specification language.

And there is a good reason for that: ECMAScript is designed to work everywhere, while I/O is platform-specific. The only I/O that ECMA-262 is aware of is datetime and randomness.

I actually think that is an omission in ECMA-262. For every JavaScript engine and/or runtime that does implement I/O, each implementatiuon is completely different. There is zero (0) compatibility or interoperability between readline() of V8's d8 and SpiderMonkey's js, nor between Deno, Node.js, QuickJS, txiki.js, et al. https://github.com/guest271314/NativeMessagingHosts - because reading standard input stream and writing to standard output stream, and handling standard error are not specified by ECMA-262.

We have all sorts of exotic objects, realms, signals, dynamic import() that can throw and not throw and both be argued to be conformant to the same language, no basic STDIO written out. That's a huge omission for a general programming language.

Nobody seems to notice because people wind up huddling in their own little JavaScript corners, entertaining preferences, shouting whose package registry is bigger, whose been around the longest, and so forth. But try to run the same code in multiple JavaScript engines or runtimes and see what happens. It takes some work to make that so https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_host.js. When it's a simple matter to specify.

But what difference does it make given since we've opened the books any implementation can throw or not throw for any section of ECMA-262. At least the Chromium folks confessed their implementation of MediaStreamTrack of kind audio is not compliant with W3C Media Capture and Streams, even if they havn't fixed that, yet. It generally takes a few years for Chromium folks to fix stuff, 6 years to get rid of censoring webkitSpeechRecognition(); 5 years to capture system audio on Linux.

TBH if I were in to TypeScript developement and I would modify or abandon that claim that TypeScript is ECMA-262 compliant, or follows the specification. The specification is clearly meaningless when people can interpret throwing and not throwing an error to be equally specification conformant.

Further, it's like Bun claiming it somehow is tryinmg to implement Node.js infrastructure, though we can't full-duplex stream using WHATWG Fwetch in Bun because there's no HTTP/2 support.

Just do your own thing TypeScript folks, without clinging to ECMA-262. Everybody else does their own thing and massages compliance out of throwing an error or not throwing an error.

@guest271314
Copy link

I'll note this, I have seen such ambiguous language in statues and administrative regulations in the domain of law, re how the preceding or following words can be interpreted by jurists. It's called a term of art in law, and if you ever see this language you best pay careful attention because it generally means the legislature ran out of time, or knows there's some bs going on:

"Notwithstanding any provision to the contrary"

Anyway, good luck trying to be in conformance with ECMA-262, statically analyzing dynamic modules in TypeScript world. Cheers.

@jogibear9988
Copy link

Add least adding a setting (wich default is false), like 'addFileExtensionToImportsMissingThemIfTheyAreResolvedWith' would not harm anybody. If you don't need it, keep it false

@guest271314
Copy link

Add least adding a setting (wich default is false), like 'addFileExtensionToImportsMissingThemIfTheyAreResolvedWith' would not harm anybody. If you don't need it, keep it false

We already have that with WICG Import Maps. That ain't ECMA-262, but neither is WHATWG Fetch or Streams.

<script type="importmap">
  {
    "imports": {
      "test/file": "./x.json",
      "test/sub/file": "./z.json"
   }
 }
</script>
<script type="module">
  import x from "test/file"
  with {
    type: "json"
  };
  import z from "test/sub/file"
  with {
    type: "json"
  };
  console.log(
    new TextDecoder().decode(new Uint8Array([x, z])), 
    import.meta.resolve("test/file"), 
    import.meta.resolve("test/sub/file")
  );
</script>

The dynamic import() part I see no solution for in TypeScript. TypeScript folks have to massage "we can throw here if we want to so we can statically analyze dynamic modules" into the mix to support the philosophy of statically analyzing everything. Well, you can be static and dynamic at the same time. That's physically impossible. You'll have to make a decision on that. I suspect you'll adhere to Deno's interpretation that, "we can throw here, there's ambiguity or room to do that so we can support TypeScript".

@djfm
Copy link

djfm commented Jul 2, 2024 via email

@guest271314
Copy link

Are we actually missing out on important stuff by not using imports in the JS code?

CommonJS is not Ecmascript Modules as defined in ECMA-262.

Node.js default loader is CommonJS. There are dozens of JavaScript runtimes that don't use CommonJS.

I'll point out that Bun supports TypeScript out of the box and does not throw as Deno does for dynamic import() with raw string specifier.

@nicolo-ribaudo
Copy link

Out of curiosity, why are we discussing some Deno-specific behavior here and not in the Deno repo? Especially if Bun shows that this is not a restriction from TypeScript.

@guest271314
Copy link

Out of curiosity, why are we discussing some Deno-specific behavior here and not in the Deno repo? Especially if Bun shows that this is not a restriction from TypeScript.

Well, why does tsconfig.json have a module: "commonjs" option if TypeScript is allegedly ECMA-262 conformant?

CommonJS is not specified in ECMA-262.

@guest271314
Copy link

Out of curiosity, why are we discussing some Deno-specific behavior here and not in the Deno repo? Especially if Bun shows that this is not a restriction from TypeScript.

Because for Deno TypeScript is a first-class citizen. And Deno has bent the interpretation of dynamic imort() in ECMA-262 to satisfay TypeScript constituents.

TypeScript has massaged CommonJS into its configuration to satisfy Node.js constituents.

So there's exceptions to rules happening everywhere here, from a neutral perspective.

I test and break multiple JavaScript engines runtimes at the same time without entertaining a preference for any. So I see what's going on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Discussion Issues which may not have code impact
Projects
None yet
Development

No branches or pull requests