New submod framework

[Booplicate]

Ideally should get updated after #9736 is merged.

Major refactoring of the submod framework due to Python 3 update.

Changes:

• Loading procedure has been changed:
• • MAS now prevents arbitrary rpy loading unless debug mode is on
• • Submods now must be explicitly defined akin to spritepacks - via a header file, as well as being placed in the appropriate folder
• • • This will require a wiki update
• • • The header format is very close to the old-style submods, but is using json
• • • If any changes are made in the future, we will need to update:
• • • • _SubmodSchema (a dataclass-like structure for json validation)
• • • • HEADER_VERSION (version number of the json schema)
• • • • the _Submod class itself (if new fields were added, for example)
• • • Submod init, parsing, and loading happens at -999
• • • • All submod's AST nodes will be merged with the existing nodes, preserving order
• • • • All early blocks will be executed at -999, but before submod init nodes
• • • Submods must be places within game/Submods (sub-folders are supported)
• • • Header file must be in the root folder of the submod, it should be called header.json
• • Most of submod API is now private and should be used only by MAS code. Submods should not access submods objects or interfere their loading in any way
• • • mas_submod_utils.isSubmodInstalled and mas_submod_utils.getSubmodDirectory are public and safe to use by submods
• • • We can expose more of the API if submod-makers require more tools
• moved mas_can_import into mas.can_import py package
• • gutted the renpy version (0imports.rpy) and pointed everything to the py pkg
• moved mas_threading into mas.threading py package
• • pointed everything in the rpy (zz_threading.py) to the mas.threading py package

Additions:

• new package - pydantic, a great tool for validation of arbitrary data
• mas_submod_utils.require - akin to NodeJS require, can import a python module from a filepath, handy for submods
• _mas_loader namespace (private, if some functionality is desired, it should be imported as a public function/class):
• • include_module - allows "including" a renpy module to load down the init pipeline with respect to the nodes init order
• • do_modules_exist - checks if a renpy module exists
• • import_from_path - import py modules from file system
• • • this is mostly for submods (see mas_submod_utils.require) to allow importing local python packages w/o interfering the global scope. For example when 2 submods need to import different versions of the same library
• _mas_root namespace:
• • is_dm_enabled - checks if debug mode is on
• added a mas root folder for mas-specific py packages

Fixes:

• fixed a crash when a submod used a undefined screen for its setting pane

[dreamscached]

Finally, an entry level increaser upgrade with great functionality! (Not sarcastic, this is awesome)

[BoofOof32]

Are there any blockers currently for this getting merged? It seems like it's mostly there aside from the 1 requested change (and even there, I can't seem to find which file needs changing, all of them appear resolved)

[Booplicate]

#9493

[ThePotatoGuy]

summary of latest changes:

• I pretty much killed most of the certifi/ssl MASImport object code. I've left the base MASImport in case that has uses.
• Added a new mas folder in python-packages. this should be the root folder for all of our python code, and we should move stuff to there as time allows.
  • (If you're using pycharm, set the source root to python-packages and everything will work fine.)
  • moved mas_threading and mas_can_import into the mas py package (mas.threading and mas.can_import). see those files for examples on how to do a transition
  • added a mas.renpyspec package that should be a dummy class/set of classes so typing works for py packages that need access to renpy stuff (like the main renpy import or a specific store). I've only added barebones so add as necessary.
• changed the dev env var key - github will probably kill us if we keep that last part so removed it

[Booplicate]

• added a mas.renpyspec package that should be a dummy class/set of classes so typing works for py packages that need access to renpy stuff

Did you try to import renpy directly? We should be able to use that. If we can't, then we should use typeshed vs dummy classes.

[ThePotatoGuy]

• added a mas.renpyspec package that should be a dummy class/set of classes so typing works for py packages that need access to renpy stuff

Did you try to import renpy directly? We should be able to use that. If we can't, then we should use typeshed vs dummy classes.

removed renpyspec and just imported renpy. requires having the sdk as a library to have type hints.