Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(build): make Node.js scripts work with Node.js 20 #4558

Merged
merged 14 commits into from
Feb 21, 2024

Conversation

maxpatiiuk
Copy link
Member

@maxpatiiuk maxpatiiuk commented Feb 19, 2024

  • Don't hardcode Node.js version number in the GitHub Action Workflows, but get it dynamically from the engines field in the package.json
  • Update GitHub Action to newest versions
  • Run prettier on the .yml files

Checklist

  • Self-review the PR after opening it to make sure the changes look good
    and self-explanatory (or properly documented)
  • Add automated tests

Testing instructions

Most changes in this PR are build-related, so don't affect the user-facing features, as long as everything builds correctly. Still, to be safe, please verify that map in query builder works correctly:

  • Leaflet map shows up correctly in the following places:
    • When entering Lat/long in query builder, can open the map and specify the coordinates using the ap
    • If query includes locality table, can plot results on the map
  • All GitHub Actions should pass, and use the correct version:
    Screenshot 2024-02-19 at 10 29 24

@maxpatiiuk maxpatiiuk requested review from acwhite211 and a team February 19, 2024 18:26
@maxpatiiuk maxpatiiuk self-assigned this Feb 19, 2024
@maxpatiiuk
Copy link
Member Author

I also made an analogous change on the weblate-localization branch in 9b25801 to fix the GitHub Action failing https://github.com/specify/specify7/actions/runs/7962942939/job/21737510948
Screenshot 2024-02-19 at 10 31 30

cc @acwhite211

Workaround for privatenumber/tsx#38
But also, it's time for those to be TypeScript
Note, I added `// @ts-nocheck` to the top of the files to suppress
TypeScript errors for now, but this should be removed once we move way
from Backbone
I wasn't able to get swc-node to work.
Tsx however works fine when using "type": "module" in package.json.
And it's about time to convert to ESM anyway - while there would still
be some friction expected, going forward, there would be more friction
from not using ESM than using ESM
The update to node.js 20.11 seemed to cause the following issues to
appear. This commit resolves them:

- .css imports in node.js begun fail
  - partially resolved by using esm-loader-css, but it is not
    ready for production yet given
    [the kind of bugs it has](brev/esm-loaders#10).
    Also, it doesn't work correctly with swc-node and tsx (each tool
    breaks in it's own way)
  - thus, I instead wrote a small custom loader
- it doesn't work with non-default exports from JS files
  - Converted to TypeScript in a3b51f7
- it seems to change the module resolution order(?) because a lot of
  new errors regarding "window" being undefined started appearing - and
  some of those are tricky to resolve (because Leflet crashes if it is
  imported, even indirectly in the Node.js, and they
  [refuse to fix it](Leaflet/Leaflet#6332)
  saying it's not their responsibility)
  - to be fair, running a small CLI command should not require importing
    any React components or Leaflet, but our code is deeply
    interconnected, and decoupling imports is tricky. And because of the
    nature of an application like Specify 7, strict bundle separation
    wasn't a priority, and changing that just for the sake of a few CLI
    commands seems like a poor way to spend time
  - Even after making all imports of Leaflet async, there is a problem
    of `@shopify/draggable`, which references window in global scope
    too, and who knows what other libraries there are that do that too
  - Thus, I added JSDOM when running in node.js and excluded the
    Leaflet plugin imports
    - Pros: it works, and doesn't require refactor. It also executes a
      lot of the codebase on any command invocation, making more errors
      be caught before production
    - Cons: it makes command invocation closer, but not by much, and
      not a big deal as we don't have that many CLI commands
We don't need an `:is()` polyfill, so the warning is just an
annoyance.

Also, remove explicit `autoprefixer` as it's included in
`postcss-preset-env`.
@maxpatiiuk
Copy link
Member Author

After spending several hours trying to resolve the build issues from https://github.com/specify/specify7/actions/runs/7963889013/job/21740345853, I got these results:

  • As a workaround for Cannot import named export from a typescript file in a workspace package if entry is using es modules and other package is not. privatenumber/tsx#38, converted .js files to .ts
    But also, it's time for those to be TypeScript
    Note, I added // @ts-nocheck to the top of the files to suppress
    TypeScript errors for now, but this should be removed once we move way from Backbone
  • Refactored our configs to use ESM over Common JS because I wasn't able to get swc-node to work.
    tsx however works fine when using "type": "module" in package.json.
    And it's about time to convert to ESM anyway - while there would still be some friction expected, going forward, there would be more friction from not using ESM than using ESM
  • Fix issues with Leaflet in node.js. See commit comment in 342a727

Outcome:

  • We are now using ESM, with some dependencies updated to newest version, and no more .js files
  • We are on Node.js 20.11, and everything works

@maxpatiiuk maxpatiiuk changed the title refactor(ci): don't hardcode node.js version fix(build): make Node.js scripts work with Node.js 20 Feb 20, 2024
@maxpatiiuk maxpatiiuk requested review from CarolineDenis and a team February 20, 2024 04:09
Copy link
Contributor

@pashiav pashiav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected!

Copy link
Contributor

@melton-jason melton-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad to see this issue finally resolved!

To better ensure that the suggested changes (properly convert the old js files to ts and the REFACTOR suggestions with tailwind) are not lost and forever stuck as comments, the context can be put into an Issue (or two), if there is not another Issue which can capture the required context.

The Javascript files here are near their end! They've definitely overstayed their welcome 😄

@CarolineDenis CarolineDenis merged commit 9e66b01 into production Feb 21, 2024
9 checks passed
@CarolineDenis CarolineDenis deleted the ci-refactor branch February 21, 2024 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants