-
-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add dependency handling and enable users to organize and publish addons and addon-collections as packages #142
Comments
@Calinou While you are working on the Godot Asset Library Laravel implementation and work on adding asset version tracking, I request that you leave room for non-assets to also be version tracked (like these Packages) in the future. You might already be planning to do that, but just in case. |
As for the "addon collection" thing, can this be achieved by publishing a dummy package that lists a bunch of dependencies? I think some npm packages do this. In any case, I'd prefer going for a simple solution 🙂 |
I would agree, except that I don't know if we want typical addons supporting "optional" dependencies. |
@Calinou Maybe we could make the underlying data in the database support both required and optional dependencies, and make addons and packages use the same configuration file to define themselves, and then have a flag set within that config file to say whether it is a package or not. And then, if it is a package, it will additionally search for and use an optional dependencies field which, if defined for an addon, just won't do anything. So the expectation would be...
Does that sound good? Any suggested changes? Do we like this idea at all? |
Additional thought: I wonder if assets should be able to declare scenes/scripts to be automatically added as singletons when installing them. The name and path to the scene/script would have to be written down somewhere in a dictionary property. Removing the asset should also remove the singleton declaration. This would make assets centered around singletons significantly faster to set up (which is good for prototypes or game jams). |
@Calinou That sounds good to me. It looks like there's someone else interested in contributing to improving the assetlib's frontend. Figured I'd let you know. Also, do we know anything about if/when/how dependency management will be implemented? I remember there was an issue with a lot of discussion about it, but I don't think a consensus was every quite reached and idk if reduz has discussed those ideas on IRC at any point. I figured, if there is a data-driven way to extract the dependencies of an asset, then we'd want to be able to display those dependent asset links within an asset's webpage. However, we don't want Godot dependent directly on the Asset Library, so there'd have to be some way of referencing an asset that works both locally in Godot and in the Asset Library (fully qualified namespaces maybe? godotengine/godot#21215). |
No, it seems reduz is way too busy to discuss this right now. |
@nonunknown Out of curiosity, how does your proposal differ from this one? |
it does not differ, it complements this one, specifying how plugins dependency should work :D |
I need my plugin Y to not even start installing without plugin X installed and active. Could this proposal solve this? |
Yes, but nothing has been implemented yet. |
@me2beats @Calinou There's also the fact that the mere decision of whether to implement dependency management at all hasn't actually been decided from what I've seen (unless there's a consensus in a proposal I missed somewhere). There's also this alternative proposal that reduz made a while back about creating a sub-project system and replacing addons with projects wholesale, so if you had dependencies, you'd literally just include them a copy of those dependencies directly in your project. So, there'd be a project tree just like there'd be a scene tree (except, you know, without the magic of scene instancing for projects). Basically, I haven't seen any actual decisions made on how to move forward with resolving any of this mess, or any action taken to implement any of the proposed solutions, and it makes community-driven addon/plugin development stick in the mud and move like molasses. |
I would like to see some kind of dependency management in Godot. However, we need to make every add-on a proper plugin before we can consider adding a way for plugins to declare dependencies. Having dependencies definable only for editor plugins isn't very useful. It's also likely we need to implement #554 first. Another issue is that dependency resolution can get quite complex if you allow multiple dependency levels – and it can encourage npm-style micropackages to pop up, with all their issues. It would be possible to limit dependencies to a single level to prevent that, but it would also make the system less useful. |
Say there's kenney particle pack. In my opinion the plugin / asset should just have
This system should only be supported for those assets/plugins, all files of which are in a separate asset folder:
I don't see a problem with dependency levels - in my view it just has to be incompatible with the dependency system: each addon should be still installed in This does not mean that all plugins must follow this rule. |
That's effectively what he's saying. With the current system, if a given addon has any global script classes, the editor can only toggle on/off those scripts if the addon in question has an EditorPlugin defined for it, thereby making it show up in the ProjectSettings' Plugins tab. And addons themselves aren't required to have any config file, as you mentioned, while plugins do require it. Considering we are coming up on a compatibility-breaking release, I agree, it is important that we make critical changes that can establish new conventions, even if it doesn't create a full-blown dependency management paradigm. So, requiring a config file for every addon sounds like a good idea to me, and then we can have things in the Asset Library tab that allow you to "manage" the addons present in the (Edit: actually, I guess, technically speaking, how it is arranged now with the ProjectSettings configuring the management of addon directories (currently as plugins), and then having a shortcut to open that management tab from the Asset Library is probably the best way to do that.)
This actually sounds like a really good short-term solution imo. Until the sub-project stuff is implemented (which I would guess would be coming with some sort of 5.0 release), we really need some sort of barebones way of officially stating the set of dependent addons that a given addon requires, just so that it can ensure that those things are also installed. |
We could adopt the solution the composer project uses. In composer the package library (or addon in this case) the This is detailed here https://getcomposer.org/doc/05-repositories.md#package-2 and adopting this concept could make it so this change doesn't break BC. |
Hi All! Recently I implemented my own plugin management system for my own modules. It mostly designed for process binary precompiled modules for third-party SDKs for iOS and Android. There are 37 production ready packages: analytics, ads, social network integration and so on. The system has dependencies, autoloading gd wrappers, export hooks, uniform module settings for app keys and so on. It can be used as GUI addon and as CLI utility (for example when you want build your project with CI and don't want to store all binary frameworks in github repo. They can be automatically downloaded before your project was build.) Full info and package list: https://drmoriarty.github.io/nativelib/ What do you think of it? |
So a year has gone by and I'd like to ask about the state of this. I've heard rumours about "a new asset lib" and would've guessed that this would be part of that. We see a surge of addons and more are about to come it with more widespread adoption of Godot. Can somebody please update the core team's current stance on this proposal? Thanks! |
npm-style dependency handling is not planned for 4.0, as there are too many other things to work on currently. I think the plugin system in general will not change much compared to what we have right now (other than replacing GDNative with GDExtension). The issues about allowing nested dependencies are still present and haven't been debated much by the core team yet.
The new asset library is only a web backend – it does not impact how the editor uses assets. In fact, the REST API it exposes is fully compatible with |
@Calinou thanks a lot! For dependencies to work, a list of dependent addons would be required as a field in the backend. So this proposal would be about
Is there already some sort of epic/label with proposal for a new frontend? I'm asking because I was tinkering with creating a new addon for a new asset lib frontend that could at some point be used as a base for a core implementation. |
Not that I know of. As I said, the feature proposed here still hasn't reached consensus among core developers, so it may not be worth doing yet. |
Ah okay, makes sense. |
It just occurred to me. Godot 4.0 is gonna be a compatibility-breaking release, potentially for quite some time. Is there any plan to update/improve the addon/plugin ecosystem for 4.0, or is there gonna be some way to not introduce compatibility breakage for a new system so that it can be part of subsequent 4.x releases? |
Feature freeze is imminent, so I'm afraid we can't make any significant changes now. The only change I'd consider making for 4.0 is #554, but I won't be working on that. We expect some compatibility breakage to happen in 4.1 regardless, so we still have some leeway (though not as much as 3.x -> 4.0). |
Sounds about right. My guess then is that we'd introduce some entirely new means of handling things like this, but set it up so that the old system is still "working", but deprecated (more than likely). |
As I mentioned in #6704, this shouldn't be too difficult to implement if done through simply compressing the addon's folder into a .zip. Since Godot already has support for reading from zips, this should just be a matter of using this on the plugins, correct? (do let me know if I am wrong, as I have not looked through the source, and am relying solely on how I think it is done) |
In keeping with Godot's philosophy of project source being text files,
|
I've recently been wondering about dependencies between GDExtensions specifically. If the path is known, it could be linkable using That would require officially endorsing one path for addons, such as Anyway, would everyone consider the issue of linking GDExtensions amongst themselves to be a part of this feature proposal, or should I create a new one for it? |
I want to put in some things I found while looking into the topic. Unity uses OpenUPM https://openupm.com/, which is basically an NPM extension to manage Unity internal packages. The Unity Package manager got really good lately as well. Someone started a similar project here for Godot: https://github.com/godot-package-manager It is not fully up to date, but it is pretty much something that could be fixed. I like the approach. I don't like NodeJS or the NPM repo, but we host a verdaccio in our company and are managing packages in Unity exclusively with that and we don't have to think about npm even once. we basically would only need to have a bit better dependency management within the project.godot file and a few more additions to the plugin.cfg mainly... a unique identifier (for example those reverse dns identifiers. org.godotengine.some_package) and a dependency management on that basis, which mentioned above already. although I would get more inspiration from toml: project.godot:
my question currently would be. can we inject and read content from the plugin.cfg and godot.project without godot it trying to parse and fail and remove it? if it does could we get the engine to not delete those areas? Edit: for godot.project I think I solved it because I can make a plugin that stores user properties so I can work on it myself. we still would need a way for unique identifiers across the landscape though and I'm still not sure if I can add stuff to a plugin.cfg without the godot editor removing it. (haven't really tried admittedly) |
@Sythelux Note that, while this is a third-party utility and isn't technically affiliated with Godot, the GodotEnv repository gives you a CLI that can manage Godot installations and Godot addons. Might try peeking at that as well. While I haven't tried the addon management myself yet, it looks somewhat robust and well documented: it'll track dependency management between your addons which are just configured as git URL or local symlink targets, although I can't quite tell from the README whether versions are involved/compared. If you want to avoid NPM, then this might be another viable alternative, perhaps with some tweaks/contributions.
As far as I know (my knowledge is ~2 years out of date, only recently returned to Godot stuff), |
Describe the project you are working on:
The Godot Next plugin: a collection of scripts that are useful in a variety of projects (utilities, editor tools, singletons, nodes, resources, etc.).
Describe the problem or limitation you are having in your project:
Godot Next is large enough that it has a small network of internal dependencies (ranging from 0 and 1 all the way up to 6 in the rarest case). No big deal if you grab the whole thing, but if you just want to solve a problem with one script, then you might have to deal with dependency management, even inside a single addon.
This is a big headache, both for me as the maintainer and for users who get interested and then very quickly frustrated/turned off due to needing to deal with resolving out-of-nowhere dependency chains.
Describe how this feature / enhancement will help you overcome this problem or limitation:
The best solution would be to increase the granularity of addons in Godot Next and introduce dependency handling to Godot. After all, some people want it all; some want just the most necessary bits. The choice of whether to make users manually pick out the parts they want from the whole or piece together the whole from a bunch of parts shouldn't be a mutually exclusive one. Both options should be available.
Right now we have 1 addon with a wide variety of classes. Some are related to others; many aren't. But, they are all a package deal. They are part of Godot Next's brand of, "a generic extension to Godot."
Rather than that, I would rather have N addons, one for each class or minor collection of classes, and have each tiny addon declare the dependencies that it has on other addons. I would then want to be able to declare all of these addons to be part of a thematic/topical "package" that users can use to mass select a collection of other addons. These could be defined as part of an AssetPackage resource type for arbitrary creation and storage. The Godot Asset Library and the EditorSettings would also be able to store these AssetPackage files for remotely distributed and locally shared "packages" of addons.
Furthermore, I would want to be able to browse and configure these "packages" of addons during the project creation and addon browsing processes to provide more concentrated value and discoverability to users.
If a new-ish Godot user opens up the Project Manager to create a new project, they should be able to view a list of popular Packages that they can install alongside their project. Rather than needing to create their project, go to the AssetLib tab, and then browse through hundreds of addons and install interesting ones one-by-one, they could instead see a GUI of addons to select which are batch installed at the end of the process and which can be selected/de-selected in groups by selecting/de-selecting themed Packages.
Users should even be able to create their own AssetPackage resources, either for local use (to auto-install a collection of preferred addons) or to upload and manage for the community (like what Godot Next could do).
A "Package" could then be...
Note that one difference between an addon and a package is that addons simply declare a set of dependencies while packages can have both required and optional addons associated with them, as demonstrated in the final mockup below.
Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
The final UI I imagine for this would be like a blend of the VS Code extensions UI and the Visual Studio Installer.
The former contains a list of elements where selecting any one element displays an editor-rendered Markdown README + any other custom data (repo web URL, rating, whatever data we can safely store and use, assuming it doesn't violate GDPR).
For reference, here is the VS Code extensions image in #139:
.
The latter maintains a list of selected editor components and will mass add/remove components from the batch installation dialog as users add/remove packages of functionality from the installation. Individual components can also be browsed and added/removed as desired.
Here is the original Visual Studio Installer:
I'm imagining that we could setup our own packages-over-addons setup with a VS Code-like README renderer. Something along these lines:
And this would be something that can appear both in the AssetLib tab and as a popup when users are creating a new project. Packages would be pulled from the Godot Asset Library as well as from local user files.
Describe implementation detail for your proposal (in code), if possible:
If this enhancement will not be used often, can it be worked around with a few lines of script?:
This enhancement is largely based on modifications to the Asset Library, the Editor's AssetLib tab, and introducing a new resource: AssetPackage. It also changes the Project Manager and is dependent on standardization changes in the addon ecosystem. It would revolutionize the accessibility and growth of addons, so yes, it would be used often.
Is there a reason why this should be core and not an add-on in the asset library?:
Tons of changes to NOT script-accessible stuff. See above.
The text was updated successfully, but these errors were encountered: