Proposal: Limit sourcemap support to bundled/optimized/minified output in v3.0

[FredKSchott]

Note: We are looking for constructive feedback from the Snowpack community! This is not a decision that we are 100% on, and your feedback matters!

Background

Sourcemaps allow you to map any line of code in a built JS file to the original line in a source file (JS, JSX, TS, Svelte, Vue, etc). Sourcemaps can be an invaluable debugging tool in browsers, letting you step through a big JS bundle line-by-line in your debugger, but see the original source code that each line maps to.

But sourcemaps have some drawbacks:

• Non-trivial to maintain: every step/plugin in your build pipeline must create a source map to represent the individual changes that it made. This means that Snowpack and every transform plugin must use a library like magicstring to make simple annotations/transformations that could otherwise be string concatenation.
• Brittle: If a single step/plugin fails to create a source map (or creates one incorrectly), the final source map is broken.
• Buggy: Because of their brittle nature, it's common to not have 100% confidence when using source maps that your source code is correctly mapping to compiled code line-by-line. I turn off sourcemaps in my browser to debug most issues for this reason.
• Lossy: Some concepts don't map 1:1 from source to build, meaning debugging concepts like "go to next line" or "this error happened on this line" don't make sense when 1 line of source code compiles to several lines of built code.
• Slow (Computationally expensive): generating source maps for every step in the build pipeline and then composing them all together into one slows down your build. Rollup has had first-hand experience with this issue in the past. The compose step could be moved off of the main thread, but this would be non-trivial to implement.

In bundled output, it is more likely that the pros outweigh the cons, since a giant compiled 10000+ line JS file is impossible to map back to any individual source code file without some help. In Snowpack, however, we build each file 1:1 during dev. This makes debugging without a source map possible and in many ways, preferable. While compiled output is generally less easy to read than source code, in the context of a single file you can mentally map the source file to the output 1:1, and get the benefit of debugging the actual code that the browser is running (without the cons mentioned above).

For this reason, Snowpack has always had limited sourcemap support. For Snowpack v3.0, I'd like us to finally decide whether we fully support sourcemaps (and fix all outstanding bugs) or whether we explicitly do not support them during dev (and reduce maintenance cost for us and our plugin authors aka ship new features faster).

Proposal

• Continue to support sourcemaps for:
  • bundled build output (via built-in optimize, Webpack, Rollup, etc)
  • minified build output (via built-in optimize, Terser, etc).
• Remove sourcemap support for:
  • snowpack dev build output
  • unoptimized (1:1) snowpack build output
• Publish our stance in a guide/blog post on www.snowpack.dev to make it clear that this is an intentional decision (and help guide concerned users)

This proposal is built on two main ideas:

1. Snowpack is a tool that values/prioritizes developer iteration speed (see "Slow" above)
2. For 1:1 file builds, it is objectively better to debug compiled output directly (see "Brittle", "Buggy", and "Lossy" above)

Impact

Keep in mind that most files are already minimally transformed in Snowpack:

• JS: no source code transformation by default, 1-2 imports added for import.meta HMR + env support.
• TS: Type annotations stripped, but otherwise no transformations other than import.meta
• JSX: JSX compiled to React/Preact functions, but otherwise no transformations other than import.meta

Svelte and Vue are two groups that do see more aggressive transformations, since Single File Components (SFCs) aren't as close to JS as JSX & TypeScript. Although note that Svelte itself also has limited sourcemap support related to preprocessors.

Fast Refresh is another factor that can lead to more aggressive code transformations that might make mental source->build output mapping difficult. If we decide to stop supporting source maps during development then we should also work with those plugin authors to make sure that the compiled output doesn't completely break the structure of the source code.

/cc @snowpackjs/maintainers @Rich-Harris @JoviDeCroock

[ToucheSir]

I'll just echo a sentiment expressed in many of the other sourcemap threads: not being able to debug with external tooling like VS Code is a big papercut. This isn't limited to languages like TypeScript either. Babel in particular likes to generate code that is onerous to read + difficult to set breakpoints in.

Perhaps there's a better solution than sourcemaps for this, In which case I'd have no objections. Or perhaps Snowpack could present this up-front as a philosophical decision—it's fine to be opinionated! In the absence of either of those though, claiming that sourcemap-less debugging is generally preferable and should only be done in browser devtools doesn't sit particularly well.

Edit: I'm not sure how sourcemaps are necessarily in conflict with 1:1 file builds either. The first paragraph talks about a "big JS bundle", but I was using per-file sourcemaps with TypeScript back in Snowpack 1.0!

[Rich-Harris]

As someone who has been in the sourcemap trenches on many occasions, I relate to the drawbacks listed above. But sourcemaps really are essential, particularly for those of using things like Svelte and Vue, and whatever other compile-to-JS tools come to exist in future (the last thing we want is for future toolmakers to recommend against using Snowpack because it makes debugging impossible). To say that debugging without sourcemaps is somehow 'preferable' really doesn't bear scrutiny, in my view.

Developer iteration speed is hindered by slow transformations, yes. But it's hindered a lot more by anything that prevents efficient debugging. Given Snowpack's nature it would have to be an egregiously slow plugin for it to have a noticeable impact outside the cold start scenario.

I'd also push back on the notion that sourcemaps are more valuable for a production build. They're useful in production if you're using something like Sentry, but in general, you're likely to encounter errors and warnings far more frequently during development (as the code is still being written) than in production.

Although note that Svelte itself also has limited sourcemap support related to preprocessors.

This is changing! It has always been our intention to fully support sourcemaps with preprocessors, and we recently shipped a version that includes sourcemap support, with more work planned.

[FredKSchott]

Appreciate your insight on this Rich, thank you :)

[FredKSchott]

This is changing!

Fixed in OP! thanks for clarifying

[Monkatraz]

As someone who uses a Stylus -> PostCSS -> CSS chain, source map support is a good feature for me. In my use-case, I actually made a plugin to properly compile and include source maps with my CSS. However, this compile-chain means that to translate CSS back to Stylus in-browser requires source-maps, regardless of dev or build modes, and regardless of bundling and optimization.

EDIT: I'll also say, albeit slightly unrelated, that due to bundling, optimization, etc. there is/can-be significant differences between the dev "build" and the actual build build. A way of previewing a full build on the dev server might be handy, especially if source-maps and the like are build mode only.

[LarsDenBakker]

I agree that small modifications, like injecting HMR code, is generally not a huge dealbreaker.

In the context of the snowpack plugin for web test runner, one challenge for us is that snowpack changes the paths of files (a file at /src/app.js is served from the _dist_ folder). Sourcemaps help find the original associated file. This and a reference to the original code are useful for generating test coverage reports.

[alexgorbatchev]

There are about 150 engineers in my direct group at work and over 1000 at the company. I'm working on an internally facing React/TS web app. In our case there are many people making PRs to our code base who aren't front-end developers and overall don't need to care about complexities of the front-end world (my team takes care of that). Not being able to see the true source code in the browser during debugging and stepping through would be a big disconnect in our case. This especially applies to call stacks during run time errors.

I started evaluating snowpack as webpack alternative and just hit the "sourcemap issue". I would have to echo what others have said as well. For snowpack to become part of the enterprise world, source maps would have to be fully supported.

[antony]

I just want to echo that especially for things Svelte, source-maps are essential. The code which appears in your browser bears no relation to the original Svelte code you wrote, and is therefore incredibly difficult to debug. I've lost many hours already when developing relatively trivial apps which I'm using Snowpack for.

In my mind, sourcemaps are absolutely critical.

If development speed is a concern, then by all means provide a way to turn sourcemaps off in configuration, but equally, in terms of development speed, without sourcemaps, trying to debug compiled Svelte output is so difficult, obtuse and time consuming that I'm actually slower with Snowpack, as it currently stands.

[FredKSchott]

Thanks for the feedback everyone! @antony @alexgorbatchev @LarsDenBakker @Monkatraz @Rich-Harris @ToucheSir thank you for taking the time to comment and share your experience.

As someone who doesn't use sourcemaps myself (and only sees the maintenance overhead) this was a good reminder of how much value they still bring to others.

Resolution: Proposal rejected, we'll continue to support source maps in dev in v3.0 and will work to fix any known bugs in our source map support before v3.0 is released.

[LarsDenBakker]

Regarding the maintenance overhead, https://github.com/Rich-Harris/magic-string is excellent for simple string manipulation while updating the source maps.

[ToucheSir]

I believe magic-string is mentioned in the top post. Not qualified to comment on its efficacy though.

[LarsDenBakker]

Sorry I missed that :)

[FredKSchott]

yea! I believe all known bugs will be fixed by moving from direct string manipulation to magic-string manipulation

[David-Else]

Great news! Does this mean we will have source maps working for TypeScript for v3.0?

[FredKSchott]

Yes, the plan is to convert our internal transformations to use magic-string, fix any bugs, and then we should be good to go. Would love any help on this if anyone is interested in getting involved.

[David-Else]

Did it happen for v3? I dIdn't see it mentioned in the release notes. (ps v3 looks amazing!)

[FredKSchott]

Bah I unfortunately couldn't get to it in time, prioritizing this next now that things have settled a bit. Should hopefully be able to release it as a bug fix in the next few days / by next week.

[David-Else]

Thanks for being awesome!

[Xiot]

Its great that you've decided to keep sourcemap support. I generally have sourcemaps turned off as well and very rarely even look at the sources tab in chrome, let alone hook it up to my ide.
That being said, a lot of other engineers that I know live by sourcemaps and ide integration as it makes their development flow much easier. Because of this, sourcemap support for the 1-1 files during development would be much more beneficial than sourcemaps for the compiled bundle.

I've also done a bunch of source-map handling myself, and I know how brittle it can be.

[jonlepage]

Not exclusively for React and TypeScript developers, also userfull for vanilla js 😁 !
I loved the fact that when my Electron,Nwjs (with SDK) app crashes, vscode automatically leads me to the line of code for debugging from the remote port!

As someone who doesn't use sourcemaps myself (and only sees the maintenance overhead) this was a good reminder of how much value they still bring to others.

A think a picture is worth 1000 words !! allow debugging in vscode give a lot of feature and powerful tool for diagnostic.
This is a example with nwjs(sdk) + parcel + remote debug vscode.

• Centralize the workflow !
• Allow see evaluate value beside code !
• MouseOver a symbole to get the reel value. (no more need console.log() everywhere) !
• Track and navigate step by step !
• Spy globals contexts variables !
• Better visual of CallStack ,Scopes and Closure.
• Allow invoke debugger statement debugger statement
• ... may i bypass ... live editing and fix issue, see who commit ....ext
  

I agree with the comments above ! i think this is the last crucial thing that SnowPack lacks. (allow pro debugging with IDE)
thank you very much for all your great work, SnowPack is awesome !
I can not wait to allow doing this soon :)

[jonlepage]

Maybe those can be a solution for implementation ?
https://github.com/mozilla/source-map
knowing mozilla source-map generator have some performance issue !
Parcel changed to a custom sourcemap-generator, maybe studying how they do can help to get some inspiration ?
https://github.com/parcel-bundler/source-map

[matthewp]

Hey @djmisterjon, we're actually going to start working on source maps integration very soon.

[jonlepage]

This will be nice matt, thanks for all.
For now i found a alternative , i falling on Vite !
#1213 (comment)

[benatshippabo]

@matthewp any progress on the sourcemap integration?

[matthewp]

@benatshippabo are you in the Discord? @lukejacksonn is actually working on sourcemap support and has been asking for feedback on his progress.

[benatshippabo]

yeah I saw that, but it looked svelte specific 🤔