2x compilation slowdown when upgrading from TS1.8 => TS2.0

[bcherny]

Is this expected? Are there ways to mitigate this?

If this is not expected, I will try to post a synthetic test case.

[bcherny]

We have a ~400 file TS project (without counting imported libs), and it will take some time to create a repro case.

My question is a high level question before I try to make that repro case: given that TS2's compiler does more than TS1.8's compiler, from your guys' tests, what kind of slowdown in cold and incremental compilation should I expect to see as a user? Is it 0? 10%? 50%?

With 1.8, our incremental build time (using gulp + browserify + tsify) was ~5-10s. Now, it is 10-20s.

[RyanCavanaugh]

2x slowdown is not something we'd expect; 10-15% is more representative from the codebases we've seen. It's possible you're using some particular patterns that we don't have similar benchmarks for.

[bcherny]

Ok, will try to put together a repro case.

[bcherny]

I wasn't able to distill our codebase down to a repro case; maybe we can leave this issue open, in case someone runs into the same thing.

At a high level, our structure is not typical - for historical reasons, instead of a well formed dependency graph, our code is a shallow graph - we have 400+ entrypoints, which we build into a single bundle with browserify + tsify.

The dependency graph looks something like this:

Have you tried benchmarking a project structure like this?

[mhegazy]

We would love to get access to the project to investigate more. we would be happy to sign any NDA's required.

We do have a set of bench marks for large projects, any where between 200 KLOC to 1 MLOC. we have not see such increase.

[bcherny]

@mhegazy Unfortunately I don't think it will be possible to share our code.

We had a few contributing factors that slowed down our build a lot already (both before and after upgrading to TS2): we were building 3 separate JS bundles from TS, with 3 gulp watchers running in parallel, often triple-watching the same file, and hogging the same thread.

Additionally:

• We have ~400 TS files (~40k LOC), compiling to 3 3MB bundles (lots of crossover and a few other libs are included in the bundles too)
• Each file is its own entry point: because it's a legacy Angular 1.x project, many files don't use imports, and register themselves with the global Angular injector instead
• We use browserify/watchify + tsify to compile, though we saw a similar slowdown when using rollup + rollup-typescript, and then with using tsc directly (browserify was the largest slowdown it turns out)

The fix in our case was to dynamically generate an index.js that requires every file. So before, we had 400 entry points, some of which imported each other; now, we have a single entry point that requires each of those 400 original entry points (the visualization in my comment above is accurate).

Our incremental build time dropped from ~4-5s to ~500ms.

Hope that's helpful.

[MarkChrisLevy]

@mhegazy I see massive slow down in incremental compilation (tsc --watch). Change in one file takes 9-10 (or even more) seconds to compile, in 1.8 was much much faster.

Our project is Angular2/Ionic2 app. We made a switch to newest release of Ionic, which uses latest Angular2.

I just made a simple test. Created new file (test.ts), with class declaration class A {prop: number} and added to my project. I run tsc --watch --traceResolution and after initial compilation I made a change in test.ts - thanks to trace resolution I've noticed, that compiler is checking all the files in my project and trying to resolve modules, that are used in those files. Obviously that takes time - I don't know whether that should work like this and if that didn't change from 1.8 (maybe is related to @types instead of typings).

[mhegazy]

There has been no changes to --watch behavior. what was the total compilation time using 1.8 vs 2.0 (tsc --diagnostics)? can you share a sample project we can look at?

[MarkChrisLevy]

Diagnostics typescript 2.0.3:

$ tsc --watch --diagnostics                 
Files:             862
Lines:          101292
Nodes:          381541
Identifiers:    127844
Symbols:        179132
Types:           38178
Memory used:   227997K
I/O read:        0.37s
I/O write:       0.06s
Parse time:      2.59s
Bind time:       1.12s
Check time:     12.17s
Emit time:       0.73s
Total time:     16.61s
18:37:47 - Compilation complete. Watching for file changes.
18:38:24 - File change detected. Starting incremental compilation...
Files:             862
Lines:          101292
Nodes:          381533
Identifiers:    127842
Symbols:        179132
Types:           38178
Memory used:   279647K
I/O read:        0.00s
I/O write:       0.04s
Parse time:      1.07s
Bind time:       0.02s
Check time:     10.73s
Emit time:       0.39s
Total time:     12.21s
18:38:36 - Compilation complete. Watching for file changes.

Diagnostics 1.8.10:

tsc --watch --diagnostics
Files:             831
Lines:          107757
Nodes:          344268
Identifiers:    115490
Symbols:      32590129
Types:           26693
Memory used:   174845K
I/O read:        0.35s
I/O write:       0.03s
Parse time:      2.42s
Bind time:       1.16s
Check time:      3.63s
Emit time:       0.63s
Total time:      7.84s
18:51:56 - Compilation complete. Watching for file changes.
18:53:55 - File change detected. Starting incremental compilation...
Files:             831
Lines:          107757
Nodes:          344275
Identifiers:    115492
Symbols:      32605151
Types:           26694
Memory used:   200667K
I/O read:        0.00s
I/O write:       0.04s
Parse time:      1.04s
Bind time:       0.03s
Check time:      3.83s
Emit time:       0.59s
Total time:      5.50s
18:54:01 - Compilation complete. Watching for file changes.

Just looking at the numbers - 2.0 compilation is at least 7 seconds slower than 1.8 and all because "check" process. Isn't that because of the switch from typings to npm ?

[vladima]

@lleevvyy is it possible to get your repro code so we can find the crux of the problem? We can sign NDA if necessary

[MarkChrisLevy]

@vladima Sure, how to send the code?

[vladima]

you can send it to vladima <at> microsoft <dot> com.

[MarkChrisLevy]

@vladima I've analyzed my code and I found, that typescript 2.0 is processing/checking much longer files, that are huge - in my app I have a few autogenerated files, and those are between 40-250KB. Each file contains few methods and inside each method there is big switch statement with hundreds of cases and each case returns new instance of some objects. Small sample below:

export class VsiItemFactoryImpl extends VsiItemFactory {

    protected  create0 (item: number) : VsiItem {
        switch(item) {
            case 3: return new VsiItemString(item, "PRODUCTION_COUNTRY", 44, [1049], null, false, false, false, null, null, undefined, VEHICLE_TYPES_CAR_CV, null);
            case 5: return new VsiItemString(item, "VERSION_NAME", 25, [1049], null, false, false, false, null, null, undefined, VEHICLE_TYPES_CAR_CV, null);
            case 6: return new VsiItemList(item, "FUEL", 119, [138], [7, 8, 9, 10, 11], false, false, true, null, [25], undefined, VEHICLE_TYPES_CAR_CV, 0, null, false, false);
            case ...
        }
    }

    protected create500 (item: number): VsiItem {
        switch(item) {
            case...
        }
    }

}

When I clear the biggest file (240KB) check time drops from 9sec to 3sec. Note, that typescript 1.8 check time is 2sec, no difference whether that big file is empty or filled.

[vladima]

@lleevvyy can you try to build compiler from this branch to check if it addresses your problem?

[MarkChrisLevy]

@vladima Yes, I confirm - much better now:

Files:           917
Lines:        105093
Nodes:        433987
Identifiers:  133966
Symbols:      183500
Types:         50668
Memory used: 265612K
I/O read:      0.08s
I/O write:     0.05s
Parse time:    2.97s
Bind time:     1.38s
Check time:    4.74s
Emit time:     1.31s
Total time:   10.39s
00:04:25 - Compilation complete. Watching for file changes.
00:05:29 - File change detected. Starting incremental compilation...
Files:           917
Lines:        105093
Nodes:        433987
Identifiers:  133966
Symbols:      183500
Types:         50668
Memory used: 331592K
I/O read:      0.00s
I/O write:     0.02s
Parse time:    0.96s
Bind time:     0.03s
Check time:    5.08s
Emit time:     0.65s
Total time:    6.73s

@vladima Thanks for your time and work!