Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Proposal] Update webpack to v5 #2875

Open
joshuarrrr opened this issue Nov 15, 2022 · 14 comments
Open

[Proposal] Update webpack to v5 #2875

joshuarrrr opened this issue Nov 15, 2022 · 14 comments
Labels
dependencies Pull requests that update a dependency file discuss technical debt If not paid, jeapardizes long-term success and maintainability of the repository.

Comments

@joshuarrrr
Copy link
Member

Much of the js bundling tasks are currently handled by the osd-optimizer package, which uses webpack and babel (via babel-loader) to build plugin-specific bundles. This is also the process by which plugin-specific dependencies and project-wide dependencies are resolved and bundled together.

osd-optimizer has a dependency on "webpack": "^4.41.5" (as do other OpenSearch Dashboards packages that depend directly on webpack, because packages are not allowed to have diverging versions of the same dependency). That version is nearly 3 years old, with the most recent webpack version being 5.6.0. If we plan to continue to rely on webpack for bundling it's likely worthwhile to upgrade to version 5, because we've already run into instances where version 4 struggles to handle common ES6 syntax in javascript files.

But it's also a bit unclear what our future bundling strategy will be. The README of osd-pm implies that the previous developers planned to eventually drop webpack altogether:

Long-term we want to get rid of Webpack from production (basically, it's causing
a lot of problems, using a lot of memory and adding a lot of complexity).
Ideally we want each plugin to build its own separate production bundles for
both server and UI. To get there all OpenSearch Dashboards plugins need to

This question may also have implications for plugin decoupling and the extensions initiative. I'm also not sure to what extent our solution here will be related to questions over the future of the osd-pm package.

@joshuarrrr joshuarrrr added dependencies Pull requests that update a dependency file technical debt If not paid, jeapardizes long-term success and maintainability of the repository. untriaged labels Nov 15, 2022
@joshuarrrr
Copy link
Member Author

@AMoo-Miki @ashwin-pc @kavilla I'd appreciate any feedback or additional context you can provide on this topic.

@mihirsoni
Copy link
Contributor

Ideally IMO, we should step back and look for long term, this may become major upgrade for OSD. I would evaluate the other option and see if that makes sense for us and can be more community friendly. lIke Rollup.

@BSFishy
Copy link
Contributor

BSFishy commented Nov 15, 2022

The README of osd-pm implies that the previous developers planned to eventually drop webpack altogether:

IMO, we shouldn't deviate from standard tools. By using tools that the community is already familiar with, like Webpack, the project is a lot less intimidating to newcomers because there are already parts that they know and understand.

I think a short-er term goal could be to upgrade to Webpack 5, as we already have a lot of configuration and things for Webpack, and like @mihirsoni mentioned, we could take this as an opportunity to see if Webpack still fits our long-term goals for OSD. There are a lot of bundlers and compilers out there, and some might fit this project better than Webpack.

@joshuarrrr
Copy link
Member Author

@mihirsoni Can you explain a bit more about why you think this would be a major upgrade? It seems like bundling today is largely an implementation detail that doesn't directly affect all developers in a direct way.

and can be more community friendly. lIke Rollup

Can you clarify what are the important criteria that make a given bundling approach more or less community friendly? Is it the approachability/familiarity of the actual bundling tools themselves or the way they're integrated into the projcet (osd-optimizer, osd-pm), or both?

@kavilla
Copy link
Member

kavilla commented Nov 15, 2022

I think one of the reasons historically is that we have this monolith repo and it has to be optimized to build and share dependencies. But then it also has externalized code. We can also look if we are actually changing some aspects of how we source control these items. If break apart this repo do we also need any optimization for a monolith repo?

It osd-pm also inspired by Lerna working with yarn workspaces so should we also figure out how much we have deviated from Lerna?

@joshuarrrr
Copy link
Member Author

Yeah, @kavilla I think our research and decisions around the future of osd-pm can continue in #294. Here, I'm mostly trying to figure out if there are any particularly strong reasons for us to not go ahead and upgrade webpack within our current bundling framework. (Or if we need to wait for more research and knowledge.)

@ashwin-pc
Copy link
Member

A webpack upgrade was attempted in the past for 2.0 #1118. The refactor involved was apparently significant. Identifying if thats still the case will determine the path forward here. If it is a big lift, we should evaluate our plan long term instead of upgrading since it will be a lot of throwaway work. If we can relatively easily upgrade it, a short term plan that involves an upgrade is possible.

@joshuarrrr
Copy link
Member Author

😆 @ashwin-pc Thanks for linking #1118 - I don't know why I didn't search for it! For what it's worth, I think I'm fairly close to a working PoC for the upgrade that won't be too bad, but I'll definitely timebox it to estimate feasibility.

@tmarkley
Copy link
Contributor

tmarkley commented Dec 8, 2022

@joshuarrrr did you finish a working PoC? I'm curious to hear about how it went!

@joshuarrrr
Copy link
Member Author

No, not yet - I got stuck enough I needed to set it aside for a bit to just unblock the vega-lite upgrade.

@neodescis
Copy link
Contributor

I've been trying to incorporate an Angular 15 standalone component into a custom plugin visualization. Theoretically this should be possible. It looks like someone has created a simple React wrapper around Angular 14.2+ standalone components.

I tried to achieve this in an OpenDashboards plugin, but ultimately webpack refused to resolve many of the @angular ES modules. I was able to get some of them working by adding .mjs support (including type: 'javascript/auto') to the osd-optimizer webpack config, but then I was seeing imports from @angular modules to other @angular modules couldn't be resolved. I believe this might be a webpack 4 issue? At any rate, I attempted to update things to webpack 5. I got a fair way down the rabbit hole, but I've had to throw in the towel when I got to BundleRefsPlugin and I was seeing lots of TypeError: Invalid value used as weak map key messages.

@plin13
Copy link

plin13 commented May 30, 2024

hi, I'm curious if there's been any progress or further discussion on this webpack issue.

@ashwin-pc
Copy link
Member

There were multiple to do this migration a few times in the past but each time we ran into issues given the custom build system for OSD. @plin13 is this blocking you in some way right now?

@plin13
Copy link

plin13 commented Jun 13, 2024

In a way yeah. We're using opensearch and dashboard on an enterprise level and webpack 4 is quarantined due to security risk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file discuss technical debt If not paid, jeapardizes long-term success and maintainability of the repository.
Projects
None yet
Development

No branches or pull requests

8 participants