-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Migrate to ES Module structure/runtime - Node 20, Vite, Vitest, Storybook 8 #272
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- update typescript and ts-node config - file path and extension changes - migrate away from styled-icons/boxicons due to typing issue with esm - update flatgeobuf, remove old workarounds - add skipLibCheck to resolve flatgeobuf internal import error - update turf to v7 prerelease, migrate to exporting latest geojson types, tighten up geometry generics - ignore rbush type import errors - switch from ts-node to tsx for prepare script - migrating use of __dirname to new method
…port. Migrate use of require in some places, enough to get init working.
* start switch to vitest * drop jest stuff, other dirname and import dep fixes, reset lerna.json * use default-import to use and extend rbush * add vitest dep formally, add vitest-setup loading jest-dom * Migrate more tests, make them runnable standalone in vscode debug with direct import * update geoblaze, update overlap output changes related to turf area. skip many remaining tests * fix prepare * comment out and skip tests with jest mocks * initial migrate of scripts to vitest, skipping failing precalc datasource tests * stub vitest project runner * clean up vitest config type error, and punt on browser test mock error and bump vi* versions * skip remaining failing tests, add back test commit gate * try making vitest regular dependency * update runner to config project vitest * merge cli options * fix vitest cli command * switch to TEST_ROOT env var * shift jsdom dep for project use * fix TEST_ROOT * simplify test commands, tune test config * update vitest comments * add data server back to test command * vitest comments * drop ts-node for tsx * add testing of base and template packages, skipping most. Migrate remaining commands to vitest. Migrate to turf 7 in projects
…rce not fetching properly?
…ded datasource fetch also resolved.
* convert function builder to esbuild * fix other function output dir, cleanup negative zero in cleanCoords * add buildClient using esbuild with html plugin, refactor App to receive lazy loaded report clients, fix loose client-ui exports * add esbuild polyfill, address esbuild strict module import - pare down projectClient import paths, switch client away from using top-level gp lib import, shift top-level indexes to prevent pollution of node code in ui code, guard use of process env var * Switch to generating ReportApp.tsx in .build-web with lazy load of report clients * on build copy language assets to .build-web
* upgrade aws-cdk-lib and aws-sdk, add package.json to build to get esm enabled in lambda, move esbuild dep to top of monorepo, add node 20 runtime * update data scripts to work in monorepo * fix precalc script env * build - create isolated mini package per function with own entry point. Add type assertion to language json import
twelch
changed the title
Migrate to ES Module structure
Migrate to ES Module structure, Node 20, Vitest, esbuild
Apr 17, 2024
…larify build script
…extract, add client polyfill globals process true
…es in directories. Fix dynamic import of translations in browser client
Upgrade WSL Ubunto to 22.04 Jammy
|
…outside of current directory
…s to older assert syntax but should change at some point
- polygon-clipping upgrade resulted in additional vertice in some clip results slightly changing value (<.04 sq meters). - Update base translations. Fix translations extract with babel import assertion plugin.
…tributes to pass to components
* switch to transient prop for duration ProgressBar * start conversion to Vite for client build, still need to fix import attributes and add dev server * Migrate start:client to Vite, language dynamic import not working or keeping with:json import assertion * Migrate gp lib to storybook 8 using Vite, missing project storybook still * drop babel from gp vite config, causing require error * add back storybook to build docs * migrate from start-storybook to just storybook * refactor report stories to use code generator with story-cache * add story-cache to base project gitignore * change report-stories to example-stories. Migrate templates to use them. reports required to use named export * migrate create:report to node 20 patterns. Fix classColumn type error * glob delete story caches on startup, add vite types using d.ts file for asset import * switch to Vite glob loader for language modules * no need to copy language modules into build-web anymore * add vite to base-project * export pathUtils, simplify tsconfig * Drop unused TranslatorSync component * shift storybook to dependencies for project user * Change story folder structure * Add turf/helpers to base-project since common project use * add all to dependencies to try and solve storybook start error * export datasource helpers, remove unused vitest workspace, update min node and npm in package.json * Make project storybook+vite independent of gp version * make styled-components props transient, drop unused main from base-project package.json * vite configure web worker * remove vite worker config since web worker not being used * fix story sort order * fix blank-project functions * attempt fix to favicon install command * add storybook install script for migraters. Add initial Migrating docs * add back default export to report clients require for React.lazy load
twelch
changed the title
Migrate to ES Module structure, Node 20, Vitest, esbuild
Migrate to ES Module structure/runtime - Node 20, Vite, Vitest, Storybook 8
May 14, 2024
twelch
added a commit
that referenced
this pull request
Jun 25, 2024
…book 8 (#272) * - update lerna and node, add eslint esm preset - update typescript and ts-node config - file path and extension changes - migrate away from styled-icons/boxicons due to typing issue with esm - update flatgeobuf, remove old workarounds - add skipLibCheck to resolve flatgeobuf internal import error - update turf to v7 prerelease, migrate to exporting latest geojson types, tighten up geometry generics - ignore rbush type import errors - switch from ts-node to tsx for prepare script - migrating use of __dirname to new method * switch CI to use node 20 * Migrate base-project and template packages to esm * disable run of test suite on commit until resolved * missed test * fix fetchPolyfill import * switch to nodenext module and moduleResolution. Add assert to json imports * clean up import of CJS dependencies that don't have a true default import. Migrate use of require in some places, enough to get init working. * switch to tsx for translation extract and migrate babel config to cjs extension * fix report name for template-blank-project * Switch from Jest to Vitest (#273) * start switch to vitest * drop jest stuff, other dirname and import dep fixes, reset lerna.json * use default-import to use and extend rbush * add vitest dep formally, add vitest-setup loading jest-dom * Migrate more tests, make them runnable standalone in vscode debug with direct import * update geoblaze, update overlap output changes related to turf area. skip many remaining tests * fix prepare * comment out and skip tests with jest mocks * initial migrate of scripts to vitest, skipping failing precalc datasource tests * stub vitest project runner * clean up vitest config type error, and punt on browser test mock error and bump vi* versions * skip remaining failing tests, add back test commit gate * try making vitest regular dependency * update runner to config project vitest * merge cli options * fix vitest cli command * switch to TEST_ROOT env var * shift jsdom dep for project use * fix TEST_ROOT * simplify test commands, tune test config * update vitest comments * add data server back to test command * vitest comments * drop ts-node for tsx * add testing of base and template packages, skipping most. Migrate remaining commands to vitest. Migrate to turf 7 in projects * fix type error, upgrade node-fetch, fix launcher paths, VectorDatasource not fetching properly? * add new world fgb and confirm test passing with latest unstable gp docker workspace * downgrade flatgeobuf to resolve timeout error, add back skipped tests * migrate to new import.meta.dirname available with node 20.12 * update fetch polyfill and upgrade again to latest flatgeobuf. subdivided datasource fetch also resolved. * unskip remaining tests utilizing flatgeobuf, update sideEffects for fetch polyfill * Switch to esbuild (#278) * convert function builder to esbuild * fix other function output dir, cleanup negative zero in cleanCoords * add buildClient using esbuild with html plugin, refactor App to receive lazy loaded report clients, fix loose client-ui exports * add esbuild polyfill, address esbuild strict module import - pare down projectClient import paths, switch client away from using top-level gp lib import, shift top-level indexes to prevent pollution of node code in ui code, guard use of process env var * Switch to generating ReportApp.tsx in .build-web with lazy load of report clients * on build copy language assets to .build-web * bring back template smoke tests * AWS - deploy pure ESM code for Lambda and client, on node 20 (#279) * upgrade aws-cdk-lib and aws-sdk, add package.json to build to get esm enabled in lambda, move esbuild dep to top of monorepo, add node 20 runtime * update data scripts to work in monorepo * fix precalc script env * build - create isolated mini package per function with own entry point. Add type assertion to language json import * upgrade zx, refactor translation install and extract to use scripts clarify build script * add missing esbuild plugins add missing babel plugin for translation extract, add client polyfill globals process true * cleanup isHostedOnLambda * process empty * bump * directly set client lambda env vars * add esbuild metafile output * add ANALYZE env vars * fix metafile output path * Switch to gp core imports in client to not pull backend modules * Allow minify to be overridden, defaults to true * fix minify arg passing * add cloufront access to client bucket to be able to access client files in directories. Fix dynamic import of translations in browser client * update translationInstall to be more comprehensive * update dirname in importTerms and publishTerms, loosen gitignore on i18n dir * Make sure .build dir created, remove old createManifest * bump * simplify generateHandler, add guard to .build dir copy * Cut out intermediate build in gp .build dir, go straight to project path .build * simplify build client gp path discovery * make build client use gp dependencies. removes need for esbuild deps in project * replace use of esbuild-plugin-html which doesn't support entry point outside of current directory * upgrade to typescript 5.4.5 * swtich from assert json to with json * fully switch from assert to with import syntax, esbuild still converts to older assert syntax but should change at some point * rebuild package lockfile, major non-breaking npm upgrades * additional upgrades: - polygon-clipping upgrade resulted in additional vertice in some clip results slightly changing value (<.04 sq meters). - Update base translations. Fix translations extract with babel import assertion plugin. * upgrade babel proposal plugins to official * update husky prepare command, husky install is deprecated * migrate to new styled-components .attrs() to define caller-defined attributes to pass to components * switch to transient prop for duration ProgressBar * Vite for client dev, build, storybook (#282) * switch to transient prop for duration ProgressBar * start conversion to Vite for client build, still need to fix import attributes and add dev server * Migrate start:client to Vite, language dynamic import not working or keeping with:json import assertion * Migrate gp lib to storybook 8 using Vite, missing project storybook still * drop babel from gp vite config, causing require error * add back storybook to build docs * migrate from start-storybook to just storybook * refactor report stories to use code generator with story-cache * add story-cache to base project gitignore * change report-stories to example-stories. Migrate templates to use them. reports required to use named export * migrate create:report to node 20 patterns. Fix classColumn type error * glob delete story caches on startup, add vite types using d.ts file for asset import * switch to Vite glob loader for language modules * no need to copy language modules into build-web anymore * add vite to base-project * export pathUtils, simplify tsconfig * Drop unused TranslatorSync component * shift storybook to dependencies for project user * Change story folder structure * Add turf/helpers to base-project since common project use * add all to dependencies to try and solve storybook start error * export datasource helpers, remove unused vitest workspace, update min node and npm in package.json * Make project storybook+vite independent of gp version * make styled-components props transient, drop unused main from base-project package.json * vite configure web worker * remove vite worker config since web worker not being used * fix story sort order * fix blank-project functions * attempt fix to favicon install command * add storybook install script for migraters. Add initial Migrating docs * add back default export to report clients require for React.lazy load * nit Migrating
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Dev
Key Requirements/Decisions
docker-gp-workspace
to the latest GDALToDo
geoprocessing init
start:client
Testing
Release
"type": "module"
to package.jsonfile] and add back any that you added just for the project, if any.
build
commandnvm alias default v20.12.1
andnvm use default
in vscode terminal if your vscode terminal keeps not using the right version of nodedocker pull seasketch/geoprocessing-workspace:unstable
Follow-on tickets:
Basic smoke test fails for ecoregions, FAIL src/functions/simpleFunctionSmoke.test.ts > Basic smoke tests > simpleFunction
fix snapshot update via
test:scripts:update
, need to migrate away from jest-based solutionDo sideEffects need to be added back for fetchPolyfill?
Vitest - leftovers from Switch from Jest to Vitest #273
Once migrate to aws-sdk v3, bring back the dynamodb tests - https://daaru.medium.com/testing-lambda-functions-with-vitest-fc713caa975d
Once migrate to aws-sdk v3, filter it and aws-cdk out of the function builds using
externals
upgrade Typescript and switch from
assert
type JSON towith
Update base translations
Merge client-ui and client-core
Reduce project dependencies by bringing CLI commands into GP space (via cd GP_PATH) so that GP dependencies are used.
translation:extract
is one example where babel and plugin should be able to be moved.Try to use top-level turf library
Consolidate
src
andscripts
into one tsc build step (assuming pure esm)Enable eslint - with warnings
Add monorepolint like turf uses for template packages
Convert "G extends Geometry" to "G extends Geometry | null" to support null geometry case?
Update React
Update i18next
Migrate from polygon-clipping back to turf clip if possible
v7 release:
Publish new base EN translations and translate them to all languages
Old - will not do
NODE_OPTIONS=--openssl-legacy-provider
Questions
nodenext
, to get json assert support, which new project init was throwing an error on for import of all json files, even though node16. 🤷esModuleInterop
andallowSyntheticDefaultImports
enabled?Notes on steps
"test:node": "glob -c \"node --import tsx --test\" \"./src/**/*.node.test.ts\"",
NODE_OPTIONS=--openssl-legacy-provider
. Webpack 4 said to not fix this.last 2 version
or similarcode
property - https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda-readme.html.mjs
file extension andimport.meta
not supported aws/aws-cdk#13274require()
calls of Node built-ins to import statements when emitting ESM for Node evanw/esbuild#2067 (comment)require()
calls of Node built-ins to import statements when emitting ESM for Node evanw/esbuild#2067