Can't install superset-frontend dependencies by using node v20.11.1 and yarn v1.22.22

[corocoto]

Bug description

Introduction

I use node v20.11.1 and yarn v1.22.22 for my project. And represented the superset-frontend directory as a workspace module, by using a custom bash script (which called on the preinstall step):

• clone-subdir.sh

#!/bin/bash
REPO_NAME="$(echo $2 | grep -oE '[^/]+$')";
git clone --filter=blob:none --no-checkout $2 $REPO_NAME
cd $REPO_NAME;
git sparse-checkout set --no-cone "$1/*"
if [ -n "$3" ]; then
    git pull origin $3;
    git checkout $3;
else
    git fetch origin;
    git checkout main
    [[ $? != 0 ]] && git checkout master;
fi

• package.json

{
  "name": "superset_wrapper",
  "workspaces": ["superset/superset-frontend"],
  "private": true,
  "scripts": {
    "preinstall": "./scripts/clone-subdir.sh superset-frontend git@github.com:apache/superset master"
  },
  "dependencies": {
    "axios": "^1.6.8"
  }
}

Problem

It works fine. But the problem happens on the installation step. I got the next error log at the installation of the superset-frontend dependencies:

error superset@0.0.0-dev: The engine "node" is incompatible with this module. Expected version "^18.19.1". Got "20.11.1"
error Found incompatible module.
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Solution

The reason why it happens is the concrete node's version which is set at the engine's field. It will be better to replace the current value with a logical expression like: "node": ">= 18". It will save backward compatibility with the v18 and allow to work with the frontend part by using newer versions.

How to reproduce the bug

1. Use node v20 and yarn v1 for this case
2. Create new project
3. Initialize package.json (with "private": true field)
4. Set preinstall script
5. Set workspaces field
6. Run yarn install

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

Not applicable

Node version

18 or greater

Browser

Not applicable

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[mistercrunch]

Recommended node is 18 at this time. nvm use 18

Are you trying to to wrap superset into something else? There are subpackage where you can do some of that here https://github.com/apache/superset/tree/master/superset-frontend/packages, but it's not common or recommended to install the whole app as a dependency

[corocoto]

@mistercrunch, yeah, I researched how to integate and use new versions of the @superset-ui/core, @superset-ui/chart-controls and @superset-ui/plugin-chart-echarts. And thats it. Because the old ones from the apache-superset/superset-ui repo are deprecated and don't have a support on the npm. And it doesn't work for me.

I tried to install them pointwise via a modified bash-script. But that way was ended unsuccessfully. And as I remember, the errors linked to tsc and files that don't exist in the concrete module (I should try to repeat it, and do research again, I think).

In the end, I decided to try to add superset-frontend as a workspace module. And check if will it work, and will have the same errors. The last ones (about existing files) were resolved, which was important for me. Because the first one can resolved by reconfiguring the root tsconfig.

But also I met with installation problem. But, as I checked, node v20 works correctly as v18. And it installs only with warnings if I install the module separately (without using yarn workspaces). And I thought: "why not to add a range of node' versions?"

P.S. And, sure, I wouldn't set the whole superset-frontend module as a dependency at the end. It contains a huge codebase and a lot of deps.

[corocoto]

@mistercrunch, thanks a lot for your comment! While I described my problem above, I got a new alternative idea about how to try to work with the @superset-ui/core, @superset-ui/chart-controls and @superset-ui/plugin-chart-echarts

[mistercrunch]

It'd be really great if these packages can be self-standing, that's the vision for sure, in practice may be tricky with all the webpack/babel/frontend-infra stuff. I know in the past it was super difficult to work on visualization or components without having it as a mono-repo. Let us know how it goes, and anything you can do to make it easier for the next person is well appreciated.