Fixes a number of issues preventing Kepler from building on fresh checkout

[alex-bain]

When I perform a fresh clone, a number of build issues prevent Kepler from starting successfully.

This collection of changes ensures that the application can be cloned / installed / built without error.

Testing

1. Clone this PR into a fresh installation
2. Run yarn bootstrap
3. Run yarn start

Everything should build successfully and the demo app should load without error.

May be related to #2594, #2593, and earlier reports.

[netlify]

✅ Deploy Preview for keplergl2 ready!

Name	Link
🔨 Latest commit	e60abef
🔍 Latest deploy log	https://app.netlify.com/sites/keplergl2/deploys/66b473794688980008c2ed5b
😎 Deploy Preview	https://deploy-preview-2596--keplergl2.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[echo292]

Ran the following in git-bash on Windows 10.
It breaks right when it tries to creates the subfolders a second time @kepler.gl/constants/dist

Run yarn
Run yarn start

Error results:

[5/5] Building fresh packages...
success Saved lockfile.
$ yarn fix-dependencies && yarn workspaces run stab && yarn workspaces run prepublish && yarn add-licence &&  yarn build:umd && yarn build:types
yarn run v1.22.19
$ ./scripts/fix-dependencies.sh
Done in 3.71s.
yarn workspaces v1.22.19

> @kepler.gl/types
yarn run v1.22.19
$
Done in 0.23s.

> @kepler.gl/constants
yarn run v1.22.19
$ mkdir -p dist && touch dist/index.js
A subdirectory or file -p already exists.
Error occurred while processing: -p.
A subdirectory or file dist already exists.
Error occurred while processing: dist.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.
Exit code: 1
Command: H:\nodejs\node.exe
Arguments: C:\Users\foo\AppData\Roaming\npm\node_modules\yarn\lib\cli.js run stab
Directory: C:\dev\kepler.gl-build-fixes\src\constants
Output:

info Visit https://yarnpkg.com/en/docs/cli/workspaces for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[echo292]

Ran in /examples/demo-app
yarn
yarn stat

yarn_output.txt is attached.
yarn_output.txt

[alex-bain]

The stab command isn't something I touched, but I can modify all instances to redirect errors to /dev/null to mask errors if that is preferable by the maintainers.

Example:

    "stab": "mkdir -p dist && touch dist/index.js >/dev/null 2>&1"

The errors you're seeing are the ones that should be resolved by this PR, I'm not able to reproduce those locally from this branch.

Can you tell me more about your environment?

[echo292]

@alex-bain Hi Alex, thank you for the response. My first comment was based on running the demo from kepler.gl root. My bad!
I reviewed your commit and realized it applies to the examples/demo-app.

My second comment is from the /examples/demo-app.
I cloned your branch and ran yarn, then yarn start.

My environment is on Windows 10 with running the commands in git-bash..
I also have a linux workstation I can try as well. I will report back once I have the results.

[alex-bain]

@echo292 I've pushed up a fix that will silently consume the "directory already exists" error, which should get multiple invocations of npm run stab to complete without error.

[echo292]

@alex-bain
I re-cloned, ran the build on both Windows 10 Pro 22H2 and Linux Ubuntu 20.04.2 LTS and received the same errors. See attached errors.
yarn_output.txt

I tried it with node versions (v18.18.2 and v22.3.0).

kepler.gl/examples/demo-app.
yarn
yarn start

Let me know if there's anything you want me to try different on my end.

[henryreed]

@alex-bain You're fantastic. With the latest commit, I am able to successfully compile Kepler.gl. My Dockerfile is as follows:

FROM node:18

COPY kepler.gl /kepler.gl

WORKDIR /kepler.gl

RUN apt update && apt full-upgrade -y
RUN yarn

CMD ["yarn", "start"]

Only thing I had to change was to add --host 0.0.0.0 to kepler.gl/examples/demo-app/package.json on line 5, like so:
"start-local": "webpack-dev-server --host 0.0.0.0 --mode development --env.es6 --progress --hot --open",

For whatever reason, webpack-dev-server listens on IPv6 only. Maybe it's a Docker quirk? Anyway, thanks again!

[pablos1rvent]

I'm beginning to believe that the developers of this started on mac/unix. Gonna try clean without branches on Debian and try again. In all branches and tags cross env is not applied, some env vars are just setted hard way like Unix, so I think it's not compatible to dev on Windows.

Maybe with some changes, it's possible, but I'm not gonna fix it due to my personal situation.

[alex-bain]

@echo292 I tried adding all of the Babel transform plugins to each babel.config.js and .babelrc file in the repository. That may help with your build issues, but since I cannot reproduce those failures locally I'm not certain if this will help. Please let me know if this resolves the issues for you?

[pablos1rvent]

@echo292 I tried adding all of the Babel transform plugins to each babel.config.js and .babelrc file in the repository. That may help with your build issues, but since I cannot reproduce those failures locally I'm not certain if this will help. Please let me know if this resolves the issues for you?

I can try.

Clone fresh, yarn and yarn start, any extra steps maybe on your branch? Thanks

[echo292]

@alex-bain Thank you! I will give this a try now.

@psirdev - As far as I know, this branch commit is specific to the kepler.gl/examples/demo-app.

[pablos1rvent]

@alex-bain Thank you! I will give this a try now.

@psirdev - As far as I know, this branch commit is specific to the kepler.gl/examples/demo-app.

Yes, I'm on the same point. Clean fresh, yarn, yarn start and everything crashes trying the demo app.

Tried Windows 10, Debian, branches that I tried: master, alex-bain:build-fixes, igr/fix-yarn-start...

Your commit also doesn't work on Windows. Gonna try again on Debian...

[pablos1rvent]

Yikes, failing... What can we do? Maybe switch to another commit that works?

[pablos1rvent]

Hello @alex-bain , I'm on the Slack, if you wanna join and try to resolve with me (pair programming), I'm here all day D:

I tried some old branches and it happens, maybe is some NPM dependencies (react, redux or babel) that broke all the code

P.D: It happened with other repositories too that are not affiliated with kepler

[alex-bain]

@psirdev - Thank you. I believe the latest commit should resolve the issues that @echo292 reported, however the scope of this PR has increased as I was tracking down and trying to resolve those errors.

[pablos1rvent]

@psirdev - Thank you. I believe the latest commit should resolve the issues that @echo292 reported, however the scope of this PR has increased as I was tracking down and trying to resolve those errors.

Thanks for everything. It looks like the pipeline is working. I tried to install on Windows and it shows:

But I think it's something on my own. Tried to delete those folders, change the stab scripts...

Gonna try on Debian and I will tell you in a couple minutes. Thanks.

[pablos1rvent]

Looks like mkdir -p (stab script on every folder in src) on unix works like a charm, but in Git Bash of Windows it fails due to yarn executing like CMD style.

Maybe It's because I have WSL subsystem. Gonna try Cygwin or another bash and try again.

EDIT: it fails too with Cygwin. Gonna change workaround and see if I can use another command.

[echo292]

@psirdev
I also successfully got it working on Ubuntu 20.04.2 LTS.
Window returned the same @loader.gl errors in Git Bash on my end.
Are you running from the kepler.gl root or examples/demo-app?

[pablos1rvent]

@psirdev I also successfully got it working on Ubuntu 20.04.2 LTS. Window returned the same @loader.gl errors in Git Bash on my end. Are you running from the kepler.gl root or examples/demo-app?

root, if you look the yarn start it redirects to the example, and it loads from the node_modules from the root folder

I'm slow because I had problems with Virtualbox and switched everything to Hyper-V, so I'm gonna confirm in 10 mins at least

[pablos1rvent]

@echo292 @alex-bain I have a fix for the stab script (for cross compatibility on Windows & Unix) on every folder on src:

 "scripts": {
    // bla bla bla... another scripts...
    "stab": "bash -c 'mkdir -p dist && touch dist/index.js'"
  },

But.... It fails later resolving the ts (it's empty) dist folder:

> @kepler.gl/utils
yarn run v1.22.22
$ babel-node ../../scripts/license-header/bin --license ../../FILE-HEADER && yarn build && yarn build:types
$ rm -fr dist && babel src --out-dir dist --source-maps inline --extensions '.ts,.tsx,.js,.jsx' --ignore '**/*.d.ts'
Successfully compiled 0 files with Babel (8ms).
$ tsc --project ./tsconfig.production.json
Done in 4.85s.

> @kepler.gl/styles
yarn run v1.22.22
$ babel-node ../../scripts/license-header/bin --license ../../FILE-HEADER && yarn build && yarn build:umd && yarn build:types
$ rm -fr dist && babel src --out-dir dist --source-maps inline --extensions '.ts,.tsx,.js,.jsx' --ignore '**/*.d.ts'
Successfully compiled 0 files with Babel (4ms).
$ cross-env NODE_OPTIONS=--openssl-legacy-provider webpack --config ./webpack/umd.js --progress --env.prod
Hash: 102f7846386cdaca8fc9
Version: webpack 4.47.0
Time: 1224ms
Built at: 26/07/2024 23:31:13
 1 asset
Entrypoint KeplerGl = keplergl.min.js
[3] external {"commonjs":"styled-components","commonjs2":"styled-components","amd":"styled-components","root":"styled"} 42 bytes {0} [built]   
[4] ./src/index.ts 784 bytes {0} [built]
[5] ./src/base.ts 78.9 KiB {0} [built]
[9] ./src/media-breakpoints.ts 2.54 KiB {0} [built]
    + 6 hidden modules

ERROR in ./src/base.ts
Module not found: Error: Can't resolve '@kepler.gl/constants' in 'C:\Users\TERMINUS\Desktop\fix2\kepler.gl\src\styles\src'
 @ ./src/base.ts 14:17-48
 @ ./src/index.ts
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 2.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed.

So I give up, throw the keyboard and tried on Debian. And it works!!!

Just thinking Windows is not supported or maybe nobody programs on windows anymore lol

[echo292]

@alex-bain @psirdev
I had Kepler.gl 3.0 demo working in windows in December and I can't remember how I got it to work. Since then I corrupted the demo folder.
I do have a working version of the /custom-reducer in windows. If I recall it was specific to how I ran the build maybe.

I attached the yarn.lock.txt hoping this can provide some clues.

yarn.lock.txt

[echo292]

@alex-bain @psirdev

Got the demo to work in WSL/VS Code in Windows 10.

1. wsl --install in the powershell
2. sudo apt update && sudo apt upgrade
3. sudo apt-get install curl
4. curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/master/install.sh | bash
5. nvm install --lts
6. I switched to node 18.18.2. --> npm use 18.18.2
7. In VS Code install the WSL extension
8. Switch the terminal to WSL Ubuntu
9. cd to the /demo-app folder
10. yarn
11. yarn start

[alex-bain]

I've reverted the >/dev/null 2>&1 change since that did not fix the problem with the stab script.

I'll also call out I had to add a number of @ts-ignore statements to deal with TS related build issues, but it appears the repo is now cleanly building.

[echo292]

@alex-bain Thank you! Your latest commit worked on the first try.
This branch works on WSL 2 / windows 10, however, it's incredibly slow to build.

[heshan0131]

I don't think you need to include @kepler.gl here, the babel loaders of kepler.gl src is handled by makeBabelRule