Scripting RFC #1
Conversation
|
I really like all of this. My only suggestion is a name change, rather than augmented FFI we should call it Automatically Idiomatic Foreign Function Interface. AI-FFI. |
What about languages that use green threads? |
|
Feel free to use GitHub's review features so we can have threads. |
I mean more like threads that are managed by a language runtime, goroutines or Erlang processes for example, but the OS views as just one OS thread. |
|
It would depend on specific cases then. I haven't thought about it yet. If you want to make an addition, please write a paragraph and PR it to my fork. |
Something about this bugs me, but I'm not sure what. Will have to think about it more. At a minimum, I think this will need a strict schema. I just have a suspicion that the driver code responsible for the the translation will end up messy. But we won't know until we try it. |
|
It looks pretty solid to me. I'd vote move forward with a minimal PoC using LuaJIT to get an idea of the difficulties associated with this level of genercism. |
|
|
||
| Anyway, keep in mind that what approach is chosen can be different for each language, as different languages give us more or less flexibility (Rust as a scripting language for example already enforces all of this out of the box). | ||
|
|
||
| ## Hotswapping |
There was a problem hiding this comment.
Does this mean that script system(s) are run in a separate World(?). I don't remember where I read it but since there is no way to add or remove systems after constructing a World it was recommended to keep dynamic systems in a separate world to avoid needing to reconstruct everything everytime something is reloaded.
There was a problem hiding this comment.
The architecture Specs will have in the future will differ from the current quite a bit. It's already in design ;)
There was a problem hiding this comment.
Regarding your comment, I think you're mixing up Dispatcher and World. Both are very likely going to change (the latter one also renamed to SystemGraph which will not need any of those workarounds).
| We will quote here what I said in slide-rs/specs#462 regarding this issue: | ||
|
|
||
| * Creating a new component type at runtime could be handled by having a special non-generic storage type that would store `Vec<u8>`. Those `Vec<u8>` would contain the raw data of the new components. Then, those components would be exposed to the scripting runtime through newly generated C headers corresponding to the component schema the user provided. That however will mean that TypeId can no longer be the only thing determining ResourceId, as those storages would share the same type but a different ID. But it does not seem unreasonable to have ResourceId become a `(TypeId, InstanceId)`, at least from my uneducated point of view. Also, at the end of the execution, the dynamic component should be collapsed into a statically generated component to no longer need to store the data in a `Vec<u8>`. The typical wokflow would be: create your components on the fly, do your testing with it, and if it's too slow then reboot the game. It seems reasonable to me, and besides I don't really expect it to be that slow, especially if you're only using it for logic creation. | ||
| * Modifying existing component types on the fly would only work on component types that were created from the scripting environment, but I think it's a reasonable constraint. Modifying a component in dynamic state would simply be a matter of iterating over the `Vec<u8>` storage and moving the data inside around. Modifying a component in statically-collapsed state however is a bit trickier, but not that hard either. You need to replace all static instances of that component into a dynamic instance of the new component. |
There was a problem hiding this comment.
Modifying existing component types on the fly would only work on component types that were created from the scripting environment, but I think it's a reasonable constraint.
Does this mean that scripts would not be able to modify core components like Transform? That seems like it would reduce the power of scripts quite a bit. Is there any reason that component methods couldn't be exposed? Or is there another way that scripts might access static components that I'm missing?
There was a problem hiding this comment.
What I meant is that scripts can't change the type signature of components defined outside of the scripting environment, as in they can't modify them. In other words, if your project creates a Rust struct as a component, you won't be able to hot-reload its structure definition.
There was a problem hiding this comment.
Ah, you meant the component types, not component instances. So scripted systems will be able to read arbitrary storages from the world and modify the components in those storages, correct?
There was a problem hiding this comment.
Not the systems. Scripted systems can do exactly as much as Rust systems.
But from the scripting ecosystem, you can't edit static Rust types.
|
My 2c as someone who is not (yet) a contributor but a prospective user: A bird in the hand is worth 2 in the bush. Don't design an API for all languages up front. Just the fact that there exists an API separating the engine and the language is sufficient abstraction for a first pass. Make the first language a mature one. Don't adopt someone else's technical debt. LuaJIT seems like a fine choice. Once people can actually write and run some kind of script at all, then start thinking about what kinds of accommodations would have to be made to support MUMPS or Befunge or whatever. Don't be afraid to say no if the language concepts just don't map well to the engine data structures. |
|
Could you give more details on the reasoning behind that conclusion? |
http://wiki.c2.com/?PrematureAbstraction |
|
I published a post on the Amethyst forums containing more details on the LuaJIT implementation. |
|
@jmqualls I wouldn't call premature a solution to a problem all major game engines have failed at solving because of their legacy code and userbase. By only thinking about a single language, we get ourselves trapped in that language and might eventually be stuck with it. Unity is so stuck with C# that they ended up creating a custom compiler for it when they realized the language requirements had changed. Besides, so far, this general purpose solution has not been in the way at all of the LuaJIT integration. If you want to consider this + LuaJIT as focusing on a single language first, then it's all good! It isn't very complicated either, we are just describing the essence of an ECS and the requirements for it to fit to a language. |
|
One possibility is to have something that other languages can compile to easily. Even Lua can fit this purpose, with several languages that can compile to it, including C#. Typescript to Lua compilers are particularly interesting to me, basically free types and other nice things, and much better IDE support than plain Lua. |
|
@DianaNites My issue with this is that it would introduce one more potentially costly abstraction layer. For example, say we want to integrate Go with this method, we cannot take advantage of its pretty performant static builds. Also, this does limit what languages we want to integrate, as we wouldn’t have the opportunity to maintain different compilers to Lua. But this approach is not incompatible with the one suggested in this RFC, so if it makes sense in some casss it can definitely be used. |
|
Merging this as @khionu is satisfied. |
|
When does the builder phase happen in this design? The build is taken care of automatically by Amethyst or other Amethyst tooling without you having to re-compile Amethyst itself, right? I want to understand whether or not I can build Amethyst itself one time, distribute that to users, and then allow them to write games with just the scripting languages and the Amethyst binary ( even if that includes more tooling like an Amethyst CLI ). |
|
You are correct. The build is, at least during development, a dynamic process. If the language is a dynamic one (like Lua for example), then it would work very simarly to how it is commonly done. If the language is a more static one (like Rust for example), it’s a module of the engine that builds your code and dynamically links it without even needing to shut down the engine. |
|
For those following along, the implementation for this RFC is being discussed here:
The implementation repo lives here: https://github.com/katharostech/amethyst-scripting-lab |
|
A proof of concept for this RFC can be found here: https://github.com/redcodestudios/rust-scripting-example |
This RFC introduces our scripting proposal.
Rendered
@kabergstrom