Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cyclic loading abstraction to ResourceLoader #71478

Conversation

adamscott
Copy link
Member

@adamscott adamscott commented Jan 15, 2023

This creates an abstraction for ResourceLoader and ResourceFormatLoader to load cyclic resources, ie. ressources currently loading that normally returns a null Resource when queried.

Supersedes #71004
Fixes #70985
Fixes #71610

@adamscott adamscott requested review from a team as code owners January 15, 2023 20:39
@Calinou Calinou added this to the 4.0 milestone Jan 15, 2023
@adamscott adamscott force-pushed the add-cyclic-loading-abstraction-to-resource-loader branch 5 times, most recently from 4121a76 to b288a60 Compare January 16, 2023 15:13
Comment on lines +2664 to +2667
if (r_error) {
// As we could open the file, we can assume that any error opening the file is linked to a cyclic reference
*r_error = ERR_CYCLIC_LINK;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems wrong, it will be overridden below by

	if (r_error) {
		*r_error = OK;
	}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semantically, it's also weird to set an error in an external variable before actually attempting to load the script which may not at all be an error. It's less descriptive for the reader, and could lead to bugs if there are race conditions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see pre-existing code does the same though with

	if (r_error) {
		*r_error = ERR_FILE_CANT_OPEN;
	}

It's a bit weird.
Normally we'd use a local Error err to keep track of the error, and only set r_error before returning (on failure, or at the end of the function on success).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All loaders use a pointer for errors instead of editing the reference itself. It's because it's editing the value of the ResourceLoader::ThreadLoadTask struct error value.

See this example:

Ref<Resource> ResourceFormatLoaderImage::load(const String &p_path, const String &p_original_path, Error *r_error, bool p_use_sub_threads, float *r_progress, CacheMode p_cache_mode) {
Ref<FileAccess> f = FileAccess::open(p_path, FileAccess::READ);
if (f.is_null()) {
if (r_error) {
*r_error = ERR_CANT_OPEN;
}
return Ref<Resource>();
}

*r_error = ERR_CYCLIC_LINK;
}

err = OK;
Ref<GDScript> scr = GDScriptCache::get_full_script(p_path, err, "", p_cache_mode == CACHE_MODE_IGNORE);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't get_full_script() tell you that it can't get a full script due to cyclic dependency? Instead of having to make an educated guess.

Copy link
Member Author

@adamscott adamscott Jan 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Loaders must make educated guesses because when loading, they don't have control anymore.

@adamscott
Copy link
Member Author

@akien-mga @vnen The code now tests if the file can be loaded. If so, the error is changed to be a cyclic load error.

It will not affect other loaders and there's no presumption to be done about ERR_FILE_CANT_OPEN.

https://github.com/godotengine/godot/blob/b288a60e51b95dfc433cc705ba9ba519d14d2d9f/modules/gdscript/gdscript.cpp#L2651-L2670

@adamscott adamscott force-pushed the add-cyclic-loading-abstraction-to-resource-loader branch from b288a60 to 84e84b7 Compare January 16, 2023 15:25
@adamscott adamscott force-pushed the add-cyclic-loading-abstraction-to-resource-loader branch from 84e84b7 to ce2a4fe Compare January 18, 2023 17:54
@adamscott
Copy link
Member Author

My PR fixes #71610 too.

@reduz
Copy link
Member

reduz commented Jan 19, 2023

I am still not convinced in the approach or whether this a problem, if you cyclic preloads() with scenes, then you will not be able to easily free them eventually. I think its correct that it fails and should not be fixed.

The error you receive should let you know that this is a problem and that you should use load() instead and take care of properly freeing this scene manually to break the cyclic link.

@adamscott
Copy link
Member Author

@reduz If I would happen to create a global scope function to free cyclic references, like you wanted to implement, would this PR legitimacy be impacted positively?

The usage of the issue #70985 seems to be legitimate. a.gd is loaded, refers to b.gd that preloads a.tscn scene that a.gd happens to be the script. It causes that the a.tscn scene will lose it's script.

I'll suggest the creation of the Squirrel-like function tomorrow at our GDScript meeting.

@reduz
Copy link
Member

reduz commented Jan 20, 2023

@adamscott Circular reference for scripts should be valid to some extent, because you can have a reference to the parser code (at least how it was meant to be implemented when we discussed about this with @vnen) however cyclic reference for resources is not because its way too difficult to fix and extremely error prone. As such the use case described in the linked issue should always be considered a bug.

@reduz
Copy link
Member

reduz commented Jan 20, 2023

Creating code that checks scripts and see if they contain cyclical references and eventually break them is not very difficult, because all you have to do is eliminate the scripts from the objects that contain it and were marked as unused (hence the cyclical link breaks and the objects are freed).

Extending this to the whole resource system is extremely inefficient and would force users to have to do this check for cyclical inclusion errors constantly when instead you can just prevent them on load.

@akien-mga akien-mga modified the milestones: 4.0, 4.1 Feb 17, 2023
@adamscott
Copy link
Member Author

Closing this PR. The GDScript team will have to discuss how to fix this issue otherwise, taking @reduz comments into account.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants