Browser-Only: Run Theia without a backend #12852

sdirix · 2023-08-22T13:05:42Z

Current state

The foundation for this feature was merged with Support 'browser-only' Theia #12853
We are still missing a lot of features, contributions are welcome!

File management

Finish the BrowserFS filesystem provider
- Implement file watching
- Implement directory deletion
- Implement file renaming
Missing browser-only feature: Import Files into workspace (context menu > Upload files…)
Missing browser-only feature: Download files from workspace (context menu > Download)
Missing browser-only feature: Search in files

Features

Testing

Implement UI smoke tests which run regularly

Documentation

Document on Theia website

tl;dr;

We would like to enable a new use case for Theia applications: Support for "browser-only" Theia applications which can be executed without a backend.

Our vision is to support Theia "browser-only" applications as a first class citizen, next to the "browser" and "electron" targets. For this we adapted the Theia CLI tooling to support the new browser-only target and frontendOnly Theia extensions. Using this tooling the feature can be supported in a minimally invasive way, requiring none (or little to none) changes to existing Theia based applications.

Check it out yourself

Because it's so easy to host static sites, we deployed three example applications for you to check out:

Minimal browser-only application
Including all Theia packages
POC for VS Code web extension support
- You can execute a hello world command implemented by a Theia plugin
- By opening a Typescript file you can see the Typescript VS Code web extension at work
- You can clone Github repositories via the File > Import repository action. Note that this uses the open proxy cors.isomorphic-git.org so it's pretty slow. This proxy is freely available so please be considerate with its usage.
- Note that this POC is not optimized, so it takes a while to start.

Details

Click here for the original issue text including high level overview of the feature

Motivation

Support of browser-only use cases is a value adding feature for the whole Theia ecosystem. While the existing Theia framework offers large flexibility and therefore supports a wide array of use cases, it requires the deployment and execution of a Node based backend. This often comes with a lot of operational effort and additional costs.

There are many use cases in which not the full feature set of Theia based applications are required. For example, demo and quick start applications often come with hard coded examples anyway, do not require persistent storage and can work without terminal support.

This feature is requested from time to time in the community too:

Implementation

Theia's tooling already generates the frontend separately from the backend in the lib/frontend directory. So technically the frontend can already be served with any web server. However, the frontend will try to establish a web socket connection to its backend and many services will not work when their proxy counterparts are not available.

Tooling changes

We added a new target for Theia applications: browser-only, e.g

  "theia": {
    "target": "browser-only"
  }

We added a new entry for Theia extension modules: frontendOnly, e.g.

  "theiaExtensions": [
    {
      "frontend": "lib/browser/i18n/i18n-frontend-module",
      "frontendOnly": "lib/browser-only/i18n/i18n-frontend-only-module",
      "backend": "lib/node/i18n/i18n-backend-module"
    },
  ]

For this target, the Theia CLI (theia build) will:

skip generating the backend
for src-gen the frontend/frontendOnly pairs are handled in the same way as backend/backendElectron are, i.e. if both are present, frontend will be skipped.

Using these new tools we were able to replace, mock and/or remove all services which need to connect to the Theia backend.

theia start is supported for browser-only targets too: It will launch the frontend with the webpack-dev-server.

Implementation changes

Using the tooling described above we were able to support the new browser-only target almost without any changes to the existing code base. The current PR only contains two breaking changes: A rename of a class, and the move of some classes from the node directory to the common directory. All other changes are only contained within frontendOnly modules.

The following subsections describe the most important features:

Filesystem

Since there is no native file system available in the browser, we settled on using BrowserFS, as it offers a Node compatible API, is feature rich and is well supported.

We foresee that every browser-only Theia application might want to handle the filesystem in a different way, for example to exclusively bundle some hard coded files, handling persistence via webworker cache, local storage, IndexedDB or only in memory. Therefore, we added a new service, the BrowserFSInitialization, that can be customized to support any behavior.

In the PR, we used this service in the new "browser-only" example application to initialize the file system with some basic example JSON files.

WebSocketConnectionProvider

The WebSocketConnectionProvider of Theia is used to create proxy services for Theia's RPC mechanism.

Via our proposed PR, we already support a minimal Theia without using any proxy services consisting of @theia/core, @theia/filesystem, @theia/editor, @theia/getting-started, @theia/keymaps, @theia/markers, @theia/monaco, @theia/navigator, @theia/outline-view, @theia/preferences, @theia/property-view, @theia/userstorage, @theia/variable-resolver and @theia/workspace.

To enable using all Theia packages, even if they still depend on WebSocketConnectionProvider, we rebind WebSocketConnectionProvider to a mock implementation whose connections will always timeout. This allows using all Theia packages without further changes, even though the functionality will be broken wherever it relies on the backend, e.g. the file search will never complete.

Eventually we would like to see all Theia packages supporting their full feature set (as far as possible) for the browser-only case as well. Once that is the case, we technically no longer need a mocked WebSocketConnectionProvider in the browser-only frontend, however it might make sense to still keep it for 3rd-party dependencies which are not adapted yet.

Outlook

For performance reasons it could make sense to move some services to web workers to lessen the load on the main Javascript thread of the frontend. For this use case an alternative ConnectionProvider could be implemented which bridges the gap to the web worker instead of to the backend.

Open Tasks

We think it already makes sense to merge this feature into the Theia code base via this PR. It's already possible to create specific feature-complete browser-only applications for certain use cases. The situation will only become better with more features being iteratively enabled over time. In communication with the community it must be made clear that a "browser-only" Theia can't support all use cases of a regular "browser" or "electron" Theia and that for now this feature shall be considered experimental.

We collected a non-exhaustive list of tasks which can be tackled as a follow up of the current PR.

File management

Finish the BrowserFS filesystem provider
- Implement file watching
- Implement directory deletion
- Implement file renaming
Missing browser-only feature: Import Files into workspace (context menu > Upload files…)
Missing browser-only feature: Download files from workspace (context menu > Download)
Missing browser-only feature: Search in files

Features

Support Theia plugins and VS Code web extensions
- for more information see the following section
Support user preferences
- Workspace preferences are already supported via BrowserFS filesystem
Adapt remaining pieces of preloader
Support terminal / console
Support localization
Support source control
Support debugging
Support OpenVSX registry
Support tasks
- Or remove feature if not applicable
Show list of extensions in about dialog
Support log levels
Support theia test for browser-only targets

Documentation

Add documentation for browser/browser-only modules in https://theia-ide.org/docs/architecture#separation-by-platform

Theia plugins and VS Code web extensions support

The largest missing piece is Theia plugins and VS Code web extensions support.

The good news is that we already have a POC showcasing that the support of Theia plugins and VS Code web extensions is conceptually feasible:

You can find the hosted POC here
- You can execute a hello world command implemented by a Theia plugin
- By opening a Typescript file you can see the Typescript VS Code web extension at work
- You can clone Github repositories via the File > Import repository action. Note that this uses the open proxy cors.isomorphic-git.org so it's pretty slow. This proxy is freely available so please be considerate with its usage.
- Note that this POC is not optimized, so it takes a while to start.
You can find the code on a branch here

Conceptually, Theia is well prepared to support Theia plugins and VS Code web extensions in the browser-only case, as these are already executed in the browser in a webworker.

However, it will require some substantial effort until plugin support will be ready for production in the browser-only case:

All plugins are organizationally exclusively handled on the backend side, especially metadata collecting and deploying
Any call to the Theia API / VS Code API which is currently routed through the backend will not work in the browser-only case

Even though there is no obstacle for eventually fully supporting Theia plugins and VS Code web extensions in the browser-only case, it will require some work.

Summary

With the support of browser-only applications we hope to open an exciting new chapter for the Theia framework. Please have a look at our PR and the hosted example applications and share your opinion below. Thank you!

The text was updated successfully, but these errors were encountered:

goyalyashpal · 2023-12-21T15:53:49Z

so, this "backend-less/browser only" theia is to theia-desktop what github.dev or vscode.dev are to github.codespaces right?

sdirix · 2023-12-21T16:10:24Z

Yes that's a fair comparison when looking at the "github.dev web based editor". The clarification is needed as GitHub codespaces is also served via the github.dev domain.

goyalyashpal · 2024-01-03T19:10:57Z

this blog about vscode-dev lists many more (practical) scenarios where the serverless application becomes indispensible

my ~~summary~~ distillate of it:
any scenario where full fledged desktop application can't be installed but a browser is available. so like:

restricted machines: offices, schools, labs, libraries, etc
restricted os-es: chromebook or window lite mode etc
devices with unsupported os: e.g. mobile devices like android / ipad tablets etc

msujew · 2024-01-16T11:09:44Z

@goyalyashpal For these use cases, it's probably still best to have a full Theia instance running (for example in the cloud). The backend-less Theia version has other use-cases. For example, we're currently working on #2842 which in conjunction with the backend-less Theia would allow users to join any workspace in the web without needing to setup a whole Theia server instance.

erlmachinedev · 2024-01-21T00:11:02Z

Extremely valuable contribution 🚀 I do Theia based project from the beginning and must admit that security considerations around backend-less option give me a way better confidence.

You have an option to control everything you want via dedicated service (the great way for virtualization of resources).

P.S. Taking into account extensible design of the tool I would wish (as a user) to see browser-only version first (where the backend comes as an option)

jjkavalam · 2024-09-05T14:53:01Z

@sdirix Hi, found this while looking for a viable front end option for a desktop app I am planning to build.

https://minimal--browseronly-theia.netlify.app/# is almost exactly what I want as a starting point.

I will be really grateful if you could point me to the repository where the sources for this app can be found.

(I am not looking to use electron for the desktop app itself; i just need the front end that theia provides)

msujew · 2024-09-05T15:11:14Z

@jjkavalam It's the browser-only example app found in this repo.

jjkavalam · 2024-09-06T00:42:32Z

@msujew Sorry for not being clear enough.

I want to extract that example out of the repo into an independent directory (basically integrate it into another project that I have).

So I am doing:

mv eclipse-theia/examples/browser-only myapp
cd myapp
yarn

This gives me:

% yarn
yarn install v1.22.22
info No lockfile found.
[1/4] 🔍  Resolving packages...
error Error: https://registry.yarnpkg.com/@theia%2fapi-samples: Not found
    at params.callback [as _callback] (/Users/jobin.kavalam/.nvm/versions/node/v20.17.0/lib/node_modules/yarn/lib/cli.js:66680:18)

Also, I understand that the browser-only project is relying on generated sources.

How to generate them ? After generation can I just move the browser-only directory out as mentioned above and use it independently.

msujew · 2024-09-06T09:10:07Z

@jjkavalam Please take a look at our documentation on how to build your own app. In addition to the steps outlined there, you need to set the theia.target" setting to browser-only` like its done in the example linked above.

jjkavalam · 2024-09-06T09:38:01Z

@msujew Thank you. I was able to make progress. Following the guide, i have a package.json like below:

{
  "private": true,
  "dependencies": {
    "@theia/callhierarchy": "latest",
    "@theia/file-search": "latest",
    "@theia/git": "latest",
    "@theia/markers": "latest",
    "@theia/messages": "latest",
    "@theia/mini-browser": "latest",
    "@theia/navigator": "latest",
    "@theia/outline-view": "latest",
    "@theia/plugin-ext-vscode": "latest",
    "@theia/preferences": "latest",
    "@theia/preview": "latest",
    "@theia/search-in-workspace": "latest",
    "@theia/terminal": "latest"
  },
  "devDependencies": {
    "@theia/cli": "latest"
  }
}

However, am interested in a very minimal setup similar to https://minimal--browseronly-theia.netlify.app/.

I wonder which packages i could safely remove without breaking anything else ?

msujew · 2024-09-06T09:41:32Z

@jjkavalam You can likely get away just with @theia/navigator and @theia/monaco.

sdirix mentioned this issue Aug 22, 2023

Support 'browser-only' Theia #12853

Merged

1 task

vince-fugnitto added the proposal feature proposals (potential future features) label Aug 23, 2023

sdirix changed the title ~~Run Theia without a backend~~ Browser-Only: Run Theia without a backend Apr 22, 2024

JonasHelming added the browser-only issues specific to the browser-only use-case label Jun 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Browser-Only: Run Theia without a backend #12852

Browser-Only: Run Theia without a backend #12852

sdirix commented Aug 22, 2023 •

edited

Loading

Motivation

Implementation

Tooling changes

Implementation changes

Filesystem

WebSocketConnectionProvider

Outlook

Open Tasks

Theia plugins and VS Code web extensions support

Summary

goyalyashpal commented Dec 21, 2023

sdirix commented Dec 21, 2023

goyalyashpal commented Jan 3, 2024

msujew commented Jan 16, 2024

erlmachinedev commented Jan 21, 2024 •

edited

Loading

jjkavalam commented Sep 5, 2024 •

edited

Loading

msujew commented Sep 5, 2024

jjkavalam commented Sep 6, 2024

msujew commented Sep 6, 2024

jjkavalam commented Sep 6, 2024

msujew commented Sep 6, 2024

Browser-Only: Run Theia without a backend #12852

Browser-Only: Run Theia without a backend #12852

Comments

sdirix commented Aug 22, 2023 • edited Loading

Current state

tl;dr;

Check it out yourself

Details

Motivation

Implementation

Tooling changes

Implementation changes

Filesystem

WebSocketConnectionProvider

Outlook

Open Tasks

Theia plugins and VS Code web extensions support

Summary

goyalyashpal commented Dec 21, 2023

sdirix commented Dec 21, 2023

goyalyashpal commented Jan 3, 2024

msujew commented Jan 16, 2024

erlmachinedev commented Jan 21, 2024 • edited Loading

jjkavalam commented Sep 5, 2024 • edited Loading

msujew commented Sep 5, 2024

jjkavalam commented Sep 6, 2024

msujew commented Sep 6, 2024

jjkavalam commented Sep 6, 2024

msujew commented Sep 6, 2024

sdirix commented Aug 22, 2023 •

edited

Loading

erlmachinedev commented Jan 21, 2024 •

edited

Loading

jjkavalam commented Sep 5, 2024 •

edited

Loading