"Fenced" Apps and Plugins

[lee-orr]

Overview / TLDR

Separate the App itself from many of the system-tied concerns, such as windowing, to make the ECS app and world more flexible in use, portable, easy to develop, and testable.

This is a proposal that builds on ideas from #6552 and #9122, as well as some of the challenges I've been handling in Dexterous Developer, and foresee existing in the Editor.

Note that by default I would call this a "Frame", but I'm using the term "Fence" instead to avoid confusion with graphics frames. Happy to find a different set of terms.

Status Quo

As of today, a Bevy app and world can have a variety of different types of data it stores and interactions it handles.

Some of these are long-living, not trivially re-created, or tied to a specific thread:

• Interaction with the OS for windowing, which is often tied to the main thread
• Interaction with the file system or networking stack, which can be triggered from any thread by might involve some degree of specific thread ownership (for example with a live socket)
• Storing references to surfaces and devices for GPU interaction
• Storing assets in memory

While others are much more flexible:

• Most components and resources are just plain data, and can be moved between threads and copied
• Some items my not be as flexible, but might be very cheap to re-create - for example thread-local optimizations, small assets that can be reloaded easily, and small serializable items

Existing PR's and discussions have touched on separating out non-send elements from the bevy "World", and relying on messaging for communication between them and the main app. However, I wanted to raise the possibility of having some additional elements for a clear, explicit separation.

The Problems that Remain

Even if we manage to get the main non-send elements out of the bevy world, these approaches still maintain a tight coupling between the portions that require these non-send elements and the rest of the app:

• The app will still remain fully bound to the OS window's event loop
• Elements will still be tightly bound to the same processes
• Elements aren't easily mocked, replaced or ignored - you still need to explicitly set up a "winit" plugin, a "wgpu" renderer, etc., and they still must be present for the app itself to run.

This creates a few challanges for making development easier:

• An editor wouldn't be able to replace the built in "winit" plugin with a different windowing approach without significant risks or potential issues
• A hot reload system wouldn't be able to work in certain environments, like the browser or embedded systems, where the full binary needs to be reloaded/replaced
• Automated non-graphical tests would require fully disabling certain systems and plugins, creating a dedicated build, and siginificant additional tooling to handle inputs and avoid the need for a graphics driver
• Automated graphical tests would also require dedicated builds with a separate input & windowing system, or significant additional tooling to screen grab the correct sections and mock inputs
• Creating capacities to pause a game, serialize it's memory state, or duplicate it's memory state somewhere else are all significant tasks that would need to be set up on a game-by-game basis or with dedicated plugins

"Fences" as a Potential Solution

The idea is that you would have a Fence, which interacts with things like winit, inputs, file/network io, etc, and then the actual App which is limited to things act on plain data, are easily recreated, or are serializable. Communication between the two would occur with one of two approaches:

• Messaging for any non-send elements or interaction triggers
• Optionally (if we can figure it out) - Requesting a reference to certain resources, for things that are larger data you don't want to re-set (like, say, textures) or that aren't serializable (like a RawWindowHandle)

This would come with a new FencePlugin trait, that could be used to enhance/build out elements of a fence.

For a normal game execution, we could use a default fence that hides this from the user. But if we want to run the game in the editor, use hot reload, or embed the game in another application, we could construct a fence directly ourselves. This can then allow for things like pausing the App within a larger environment, replacing a running app with a different one, or swapping out elements of the app's environment. It could also make it easier to run apps in a simulated environment, since you could potentially create a "Test Fence" for running pre-recorded scenarios or the like.

Some notes:

• Because of the way WGPU works, you would actually still be able to have most of the render plugin stay within the app - it's just the initial surface creation that would need to occur in the fence
• For some things, like large assets, we'd still need to figure out whether we want the asset to live in the app or in the fence. It might be worth having that be something that is user-definable with sensible defaults, and an API to move things from one to the other
• It's uncertain if we'd be able to make this work across processes - but having some capacity for cross-process handling would allow for better handling in scenarios like the editor or in-browser

Obviously, this is very early on as an idea, but I was wondering what people think about it as a concept? What are the big concerns? Do you think it's worth exploring further?

---

EDIT: Better sections, added explanation of what this would be trying to solve.

Original Text

This is a proposal that builds on ideas from https://github.com//discussions/6552 and https://github.com//pull/9122, as well as some of the challenges I've been handling in Dexterous Developer, and foresee existing in the Editor.

Note that by default I would call this a "Frame", but I'm using the term "Fence" instead to avoid confusion with graphics frames. Happy to find a different set of terms.

As of today, a Bevy app and world can have a variety of different types of data it stores and interactions it handles.

Some of these are long-living, not trivially re-created, or tied to a specific thread:

• Interaction with the OS for windowing, which is often tied to the main thread
• Interaction with the file system or networking stack, which can be triggered from any thread by might involve some degree of specific thread ownership (for example with a live socket)
• Storing references to surfaces and devices for GPU interaction
• Storing assets in memory

While others are much more flexible:

• Most components and resources are just plain data, and can be moved between threads and copied
• Some items my not be as flexible, but might be very cheap to re-create - for example thread-local optimizations, small assets that can be reloaded easily, and small serializable items

Existing PR's and discussions have touched on separating out non-send elements from the bevy "World", and relying on messaging for communication between them and the main app. However, I wanted to raise the possibility of having some additional elements for a clear, explicit separation.

The idea is that you would have a Fence, which interacts with things like winit, inputs, file/network io, etc, and then the actual App which is limited to things act on plain data, are easily recreated, or are serializable. Communication between the two would occur with one of two approaches:

• Messaging for any non-send elements or interaction triggers
• Optionally (if we can figure it out) - Requesting a reference to certain resources, for things that are larger data you don't want to re-set (like, say, textures) or that aren't serializable (like a RawWindowHandle)

This would come with a new FencePlugin trait, that could be used to enhance/build out elements of a fence.

For a normal game execution, we could use a default fence that hides this from the user. But if we want to run the game in the editor, use hot reload, or embed the game in another application, we could construct a fence directly ourselves. This can then allow for things like pausing the App within a larger environment, replacing a running app with a different one, or swapping out elements of the app's environment. It could also make it easier to run apps in a simulated environment, since you could potentially create a "Test Fence" for running pre-recorded scenarios or the like.

Obviously, this is very early on as an idea, but I was wondering what people think about it as a concept? What are the big concerns? Do you think it's worth exploring further?