-
Notifications
You must be signed in to change notification settings - Fork 12.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expand auto-import to all dependencies in package.json #37812
Comments
Also, thanks to @andrewbranch who explained why this behavior was happening, and who gave me the rundown of what his PRs did in #31893 and #32517 - both which were the inspiration for using the |
First I want to link #36042, which I’ve been using as the canonical issue to reconsider this behavior. I also explained in some detail how auto-imports work today and some ideas (and corresponding pitfalls) about how we could change them to solve this.
I want to expound on this a little bit for the benefit of others who aren’t as familiar with what this means, and to emphasize “potentially breaking” a bit more. What you’re getting at here, which I also explained in #36042, is that today, auto-imports for a particular module are offered if and only if that module is already part of your whole-program compilation, which is determined by:
Any file matched by these criteria are automatically parsed and bound, and type-checked to some degree, depending on the file location and A part of me thinks it would make a lot of sense to use package.json files to guide type definition inclusion, and stop giving special behavior to In that case, the only option is, as you said, adding a language-service-specific implementation that doesn’t affect program semantics, and doesn’t touch tsc at all. So, a couple thoughts on that:
I completely agree with this. If people have packages listed in their package.json, they’re going to import or reference them somehow. I’m not at all concerned about the memory impact of loading types from package.json-listed dependencies; I’m only concerned about their semantic impact on the program. So if we need a way to isolate these types from influencing the core program, how do we do that? Load them into a second program? That would take care of all the parsing, binding, and checking that auto-imports relies on, but it would duplicate a lot of work and types and symbols from common lib files (not to mention the weight of all the closed-over functions in the program/checker)—that’s the concern I have with memory. We should do some experiments and see what the cost of that really is. I also think it’s worth seeing how much we can do with parsing and binding alone. But both of these ideas are pretty complicated in an editor scenario, where the user is continually installing and uninstalling node modules and changing the shape of the core program by adding and removing imports. Takeaways:
|
One of the things that @ahejlsberg mentioned in a conversation was potentially using a second |
I'll need to read through this tomorrow for any lucid feedback, but is there anything we can learn from Roslyn's version of this for C#? I think theirs even recommends things from packages you don't yet have installed, so I imagine they've put work into the appropriate heuristics for this, albeit tuned to their user base. |
Having different files in program from what they are in |
Yeah, I agree. I haven’t seriously considered that as an option. Just to make sure it’s clear, when I say
I’m referring to changing the behavior both for tsc and for the language service (which is why it would be such a significant breaking change). |
Really glad you brought this up - I briefly spoke with @CyrusNajmabadi to understand more of what Roslyn does.
As far as I understand, that's a quick fix, not code completion functionality. We provide an I also think that auto-import completions could be a bit too noisy for npm packages. There's over a million packages on npm, and while Roslyn seems to rely on an index of the most popular NuGet packages, it's partially out of date and we might have a harder time distinguishing between common identifiers, especially functions named This all did make me wonder: instead of digging through |
Not necessarily interested; my point was just that it seemed like a good sign they had substantially thought through this. :)
I think LSIF could definitely be useful for this - in fact, this is kinda what it's for (see lsif-npm). I'm not sure where this information is currently hosted or how often it's generated, though. |
As the person who opened #38176, which was linked to this issue, let me just provide one user's viewpoint. I looked at #32517 and saw that only imports from package.json are included. I can understand needing to limit the scope of what you provide auto-imports for, and if package.json is the limit then I will work with that. That being said, sometimes packages provide functions/etc. that reference types (either as input parameters or return types) from other packages, which may be in their own package.json dependencies. You could consider a scheme that imports from a project's package.json, and then through all dependencies of the packages listed in that package's package.json (1 level deep). It could go 2 levels deep, or even fully recursive. Perhaps the number of levels could be configurable by some property to allow users to chose the balance betwee performance and coverage. I don't pretend to know all the under-the-hood details here, and looking at the linked issues I can see that there has been signficant discussion on this issue to date. And I understand that this is just one use case in a sea of many. So take my input with a grain of salt, and perhaps it will prove useful! |
I have related issue but not sure if it the same.
So when Im trying to import smth from shared into componentLevel3 it goes with import projects/shared while I as joing to see imports from ../../../shared. |
I want to give an update on the state of my experiments here. I’ve been working on an implementation of using one or more auxiliary programs that contain only type definition files from node_modules packages that are listed in a package.json file to see what the memory and performance impact is. A simple testI created an extremely simple approach with lots of room for optimization and tested it on the boilerplate generated by the Angular CLI. (It was a good candidate because it’s realistic, not entirely empty, and includes quite a few unreferenced dependencies that ship their own types, e.g. After the auxiliary program was created, the difference in response time for completions increased by about 50%. I haven’t yet investigated whether there are opportunities for improvements here yet, but I expect that much of the increased time is just the natural cost of processing more modules for auto import—that is, if all these dependencies were in @types, or already imported into the main program, we’d realize that same cost, which we’ve always considered to be worthwhile. As a result of that extra work, we get a better list of completions. Resolving all the package.json dependencies and creating the program took around 1700 ms, so we need to be sure that happens during a non-interactive time. Also, creating the program without the redundant dependencies already in the main program could bring this time down. The approximate weight of the extra program was 56 MiB, which was a 47% increase over the baseline weight. Again, deduplication of dependency inclusion would help here. Upcoming challengesThe big things that need to be done to make this approach viable are
The latter is particularly challenging in the project references scenario (particularly project references in a monorepo context where there are lots of package.jsons and node_modules folders), because there are multiple “main programs,” each with a different constituency of files already included. Suppose we have a project like:
Let’s say that If you open On the other hand, @sheetalkamat pointed out that if you host one auxiliary program per language service, you can solve both problem 1 and problem 2 above. Taking the same mobx example again, when you create the language service for project-a, you know everything that its main program already imports, so you can avoid creating the auxiliary program for it entirely if you see that all the package.json dependencies are already included in the main program. Then when you start editing project-b and create its language service, you see that mobx is not part of the main program, so you spin up the auxiliary program. When generating auto-import suggestions, you don’t have to deduplicate, because you know the contents of the two programs are mutually exclusive. At the same time, language service objects are already well-equipped to host programs with caching and efficient updating, so this solves problem 1 as well. The downside of this approach is that in a large monorepo, you could end up creating a lot of programs, and even though each auxiliary program’s contents is mutually exclusive with its main program, all the auxiliary programs’ contents could easily be redundant with each other (particularly in a monorepo where all dependencies are hoisted to the top level), which just feels like a waste of memory. (Note: my understanding is that Next steps
|
Currently you cannot trigger an import suggestion for any newly added files for a project reference. So I added all files from project reference to my tsconfig "include". Now it works. Not sure to what extent this defeats project references. BUT it's the only way I can work somewhat productively. Just wanted to add my 2c. Is there anything I am missing? |
Jumping in (if I understood correctly the topic) just to confirm this could cause a problem for projects which uses testing frameworks with conflicting global types (see Jest + Cypress combo). I think there is an ongoing effort to make jest more ESM-ish style, which could potentially avoid global-scope conflicts, but dunno about mocha, chai, jasmine, cypress, etc etc |
@arogg I think you'd be better off filing a separate issue
@IlCallo can you elaborate a little bit? This proposal expands auto-imports, so how would this cause problems? |
@IlCallo I hope I'm not over-simplifying, but wouldn't test frameworks, etc usually be |
@amcasey is correct, but it sounds like @IlCallo’s concern was (please correct me if I misunderstood) that types discovered by the auto-import provider would leak into type checking, negating any carefully defined project boundaries and Edit: I thought this comment was on my PR, not on the original issue, which might be a source of confusion. Some of the approaches discussed here would have created the problem I just described, but we have pretty much settled on an approach that avoids it, which is in progress here: #38923 |
@andrewbranch yes, that was my concern. |
@andrewbranch hey guy. I'm using latest vscode v1.54.3, typescript 4.1.5, angular v11. Still no luck to trigger auto import for Is there a clear guide for solving this problem?
|
|
Background
TypeScript's auto-import functionality has some really unpredictable behavior today where the feature doesn't work on for a package until a user manually imports from that package. This experience only affects projects that provide their own types because, by default, TypeScript always includes
@types
packages as part of every compilation; however, it doesn't include other packages innode_modules
[1]. Apart from being unfair to TypeScript library authors, this is just plain surprising for users.For a while now, we've received consistent feedback that this is a confusing experience.
Additionally, we've been seeing weird hacks in projects like Angular CLI, where
typeRoots
are set tonode_modules
andnode_modules/@types
to try to fix auto-imports.Proposal
In #31893 and #32517, we improved some rough areas of completions by making sure that we only provided completions from packages that are listed in
package.json
. That wayclearTimeout
doesn't appear in your completion list unless you specifically list@types/node
as a dependency.What I'd like to suggest is an expansion of completions sources to those in
package.json
. Specifically what I'm looking for is that for every dependency inpackage.json
, we should try to provide auto-imports. This could be done by changing the default behavior fortypeRoots
(potentially breaking) or by adding an editor-specific behavior for automatic package inclusion.There's things to keep in mind as we do this:
.d.ts
files unnecessarily.d.ts
files but a user has@types
packages installedMaybe for the general case, this won't cause a drastic increase in memory usage - most packages with explicit dependencies might get installed/eventually used anyway. But if it is, there might be some workarounds to consider:
export * from
statements), but it might just be good enough.dependencies
and notdevDependencies
I'd like to get people's thoughts on whether an approach like this would even be possible, and if so, whether we could implement it without degrading the current experience.
[1] The reason for this is the asumption that having an
@types
package in yournode_modules
directory is a pretty good sign that you intend to use it (with caveats - see #31893 and #32273). But in order to save on time/memory, we don't load up.d.ts
files from othernode_modules
packages unless we really need to.The text was updated successfully, but these errors were encountered: