import conflict with 3.0.0 update

[freeman-lab]

Hi! I happened to pull in v3.0.0 of d3-format today and the recent type: "module" change broke an existing use case. I realize it's a major version bump, and I hate to be a bother, but wanted to report in case it's not an expected behavior with this change.

I have a very simple set up, entirely reproduced in this repo here.
https://github.com/freeman-lab/d3-import-test

It's a tiny next.js app that depends only on d3-import in one file using the syntax

import { format } from 'd3-format'

Previously with ^v2.0.0 this worked fine. With v3.0.0 I get this error

Error: Must use import to load ES Module: /Users/freeman/github/freeman-lab/d3-import-test/node_modules/d3-format/src/index.js
require() of ES modules is not supported.
require() of /Users/freeman/github/freeman-lab/d3-import-test/node_modules/d3-format/src/index.js from /Users/freeman/github/freeman-lab/d3-import-test/.next/server/pages/index.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename /Users/freeman/github/freeman-lab/d3-import-test/node_modules/d3-format/src/index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/freeman/github/freeman-lab/d3-import-test/node_modules/d3-format/package.json.

I'm not an expert on some of the recent module ecosystem preferences, so I'm not totally sure how to think about the problem.

I did confirm that I could get it to work by putting "type": "module" in the top-level package.json of my app. But that then made it impossible to use many other modules, including all the d3-* modules that have not yet been updated. And of course everything also worked if I rolled back to ^v2.0.0, but then I'm stuck with older versions.

So, is my understanding correct that using any of the new versions of the d3-* modules will force the "type": "module" requirement on the importing module as a whole? Or am I importing incorrectly?

Again, apologies if I'm misunderstanding, this might all be intended behavior and I just need to change the usage pattern on my side.

Thanks for all the hard work you do on these libraries!

[aulonm]

I'm not an expert on this, but the things I've read today is that it doesn't look like next.js supports ESM builds quite yet.

They have opened up a RFC to support it (here) and the issue asking for support is here.

Looks like you actually need to keep using the old version if you want to keep using d3. @mbostock and @Fil could support two different builds, but that would mean extra work and I'm not sure that is the best solution.

For the maintainers, I found a good gist about adding support for ESM from Sindre Sørhus that may be nice to read through. link

[mbostock]

This is an issue with Next.js, not with D3, so closing. You can either stick with the previous major version, workaround it as described below, ask the Next.js team to fix it, or contribute a fix to Next.js.

I believe the workaround for Next.js involves putting something like this in your next.config.js:

function generateIncludes(modules) {
  return [
    new RegExp(`(${modules.join("|")})$`),
    new RegExp(`(${modules.join("|")})/(?!.*node_modules)`)
  ];
}

const includes = generateIncludes([
  "d3-format" // list any other ES module packages here
]);

module.exports = {
  webpack: (config, options) => {
    config.externals = config.externals.map((external) => {
      if (typeof external !== "function") return external;
      return (context, request, callback) => {
        return includes.find((i) =>
          i.test(
            request.startsWith(".") ? path.resolve(context, request) : request
          )
        )
          ? callback() // i.e., not an external
          : external(context, request, callback);
      };
    });
    return config;
  }
}; 

[freeman-lab]

Thanks both for the help and clarification!

[vasco3]

I resolved the problem by using next-transpile-modules in next.config.js and assigning d3-format to be transpiled.

That works (using webpack5 and latest nextjs) but it throws some warnings about hot-refresh de-optimizations

All are similar but involved different d3-format methods:

 ./node_modules/d3-format/src/precisionRound.js
 Anonymous function declarations cause Fast Refresh to not preserve local component state.
 Please add a name to your function, for example:
 
 Before
 export default function () { /* ... */ }
 
 After
 export default function Named() { /* ... */ }
 
 A codemod is available to fix the most common cases: https://nextjs.link/codemod-ndc

[ashlink11]

Got lucky today. Updated Node version (by clicking the installer on their website) and Next.js version to latest (by yarn add next@latest). No longer have the error. Seems that Next.js team supports ESM modules now needed for import * as d3 from 'd3'. Good luck out there!

[vasco3]

@cruikshankss works for me too. Thanks for letting us know!

[chalupagrande]

For those that landed here looking for possible solutions: You can use ssr:false in a Nextjs Dynamic import explained here to import a module that uses D3. In the example below "TradingNetworkGraph" is a component with a D3 visualization

const TradingNetworkGraph = dynamic(
  () => import('~/components/Visualizations/TradingNetworkGraph'),
  { ssr: false }
)

Trading network graph looks something like this:

import React, { useEffect, useRef } from 'react'
import * as d3 from 'd3'

export default TradingNetworkGraph(props){
  const svgRef = useRef()
  useEffect(()=> {

     // ... all the code to run the simulation

  }, [false])

  return (<svg
      id="svg"
      width={opts.width}
      height={opts.height}
      ref={svgRef}
      viewBox={....... etc etc........}
    ></svg>)

}