Add a method to list the contents of PCK files or resource associated with it

[Xrayez]

Describe the project you are working on:
Goost - Godot Engine Extension.
Working with unit testing frameworks such as Gut or WAT.
Working on a project with modding support.

Describe the problem or limitation you are having in your project:
I'm using Gut to test scripts in my project. It allows you to specify a single directory from which all scripts are detected recursively. It uses Directory class to traverse those. While it works in debug/editor builds, there's no way to collect the scripts once the project is exported, because all the assets are packed into the .pck file, hence there's no way to test the project in release builds, which is crucial given the sheer number of bug reports pertaining to debug/release inconsistencies (wrong debug checks which are originally meant to be present in release, you can also unknowingly rely on debug-only features which are not documented etc).

Gut does provide a way to import/export the tests and you can see how complex it can get.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:
It would be nice if Godot could provide a really simple method to inspect the contents of a .pck file by returning an array of resource paths within a PCK, so things like the scripts could be collected in release builds and tested accordingly.

It is not limited to testing of course. Any other resource type could be listed.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:
Similarly to how we have ProjectSettings.load_resource_pack(), it would be nice to have a method such as list_resource_pack(). The full support for traversing the .pck file just like any other filesystem would be a much better and ultimate solution of course, but given godotengine/godot#7845, I understand that this might not what core developers are interested to work on.

In fact, this kind of feature would only enhance the modding community/support in Godot. Yeah, you could load_resource_pack() from external source, but how do you actually know what this PCK file represents? You might want to validate if that's really a PCK that you want to load (is it even compatible?), or you could list all assets and display that in a game gallery dynamically (once a PCK is loaded). I mean, in some cases you might not know what individual resources you could actually load() without knowing the actual paths, and this proposal is the answer to this.

If this enhancement will not be used often, can it be worked around with a few lines of script?:
There are currently some existing plugins which provide the support for walking the .pck files:

• https://github.com/MrJustreborn/godot_PCKManager (not updated in 3 years, crashes with some PCKs in 3.2)

Is there a reason why this should be core and not an add-on in the asset library?:
I believe the engine should provide first party support for this, because feature-wise it's currently incomplete. If Godot provides it's own virtual file system, there should be a way to interact with it, at least on some level.

[yvie-k]

Just dropping my 2 cents here.

Godot already has a PCKPacker class. This proposal could be a class of its own. Maybe something like PCKReader.

[EMBYRDEV]

This! It seems super strange that the developers wouldn't add this to the core, it seems like missing functionality of a system that exists. We can write and load them but not see anything about them?

This comment is just puzzling.

[Calinou]

See also godotengine/godot#34444.

As a workaround, you can include a paths.txt file at the root of the PCK with all resource paths listed in the file (this file can be generated with a script). Make sure to always include this file at the same path so you know where it's located in advance.

[EMBYRDEV]

As a workaround, you can include a paths.txt file at the root of the PCK with all resource paths listed in the file (this file can be generated with a script). Make sure to always include this file at the same path so you know where it's located in advance.

Unfortunately that's not ideal for anyone wanting to do nice mod support as layering modes on top of each other wont work.

I've been using this in a personal engine experiment and this kind of mounting and exploration is perfect for that application.

See also godotengine/godot#34444.

That's also not ideal as it doesn't do the patch on top of the existing files that PCK files do.

Honestly just some way of exploring the tree of a PCK is all that should be needed and I don't imagine that hard...

@Calinou if I were to look into creating a PR for this what are the chances it will be accepted?

[EMBYRDEV]

For reference this is the use case in which I see this as necessary. Ideally, the PCK system would be entirely transparent to game code.

• The developer releases a game and a separate mod kit with the project files (any proprietary code being located in a GDNative plugin or custom build of the engine).

• The player downloads the mod kit and creates a new level that interfaces with all existing code and assets. This player we don't really want to edit the level select screen just to include their level as the player may have many custom maps installed. This also rules out having a maps.txt file or something as modders would all just overwrite each other.

• Instead we'd have the modder place their map in a specific directory where we could use Directory or something to list all files in res://mods/ and then the game is coded to load them a certain way automatically. The crux here is that we don't know the name of the mods or maps and the modder cant maintain a list as they would overwrite each other like above.

If this issue is correct then the Directory class just ignores any newly loaded PCKs which seems more like a bug in the Directory class than a new feature...

[Calinou]

@Calinou if I were to look into creating a PR for this what are the chances it will be accepted?

I can't give any guarantees about it – it depends on the quality of the implementation 🙂

I recommend joining the Godot Contributors Chat to discuss the implementation of any sizeable feature.

[Kersoph]

This would be very helpful as well for asset organisation by file structure (next to modding support, which I’m trying to provide as well).
For example, if you have a folder with all fruit trees and various specific fruit trees (like apple tree, cherry tree, etc.) you could automatically load and handle all variations by their file location. Same for Items or skins or mesh variations etc. This currently works in the editor ("Load all files in folder X as fruit trees and add it to the game"). But currently not at runtime because the location is lost. As Calinou pointed out, you can create big lists that could be updated or manually handled. But this breaks if you want to support mods as those lists will be overridden or need to be known as STUDIOEMBYR pointed out.

I was running in this problem when I wanted to organise all my visual variations of plants and buildings by such a convention. As a workaround, I'm using lists to track all assets which is prone to errors and quite some work to implement an automatic tracker. Also modding support is harder to achieve. I'm thinking about loading the .pak myself again to build the file structure for me but it would be great if there is a class for it in Godot directly as Godot most likely does it already while loading the pak.

PackedDir looks suspiciously like something I'm looking for.
https://github.com/godotengine/godot/blob/master/core/io/file_access_pack.cpp