From 0e4587533f656e19789bcd9bc5c28a4c92f83279 Mon Sep 17 00:00:00 2001 From: Leonardo Gama Date: Tue, 31 May 2022 14:25:03 +0000 Subject: [PATCH] Fix `dill._dill` submodule being saved as `GLOBAL "dill._shims" "_dill"` I'm tracing some problems with my "portable" mode prototype and hit a bug with references the `_dill` submodule saved as global: ```python >>> import dill >>> dill.dumps(dill._dill, 0) b'cdill._shims\n_dill\np0\n.' ``` The `_dill` submodule is saved as an attribute of the `dill._shims` submodule, which is valid, but it should be just `dill._dill`. The `_dill` submodule is special-cased to be saved as global: https://github.com/uqfoundation/dill/blob/8e5e450b9ed8dff164fc259c468365e2235c6581/dill/_dill.py#L1807-L1810 But `pickle.whichmodule` misidentifies it as pertaining to `dill._shims`, because this entry is set earlier in `sys.modules` and it picks the first match. --- The change fixes things for this especial case, but there is potential for new bugs related to other submodules: ```python >>> import dill, glob, importlib, os, pickle, pprint >>> os.chdir(dill.__path__[0]) >>> modules = [mod.rpartition('.')[0] for mod in glob.glob('*py') if not mod.startswith('__')] >>> modules = {mod: importlib.import_module('dill.' + mod) for mod in modules} >>> pprint.pprint({name: pickle.whichmodule(mod, name) for name, mod in modules.items()}) {'_dill': 'dill._shims', '_objects': 'dill', '_shims': 'dill._dill', 'detect': 'dill', 'objtypes': 'dill', 'pointers': 'dill', 'settings': '__main__', 'source': 'dill', 'temp': 'dill'} ``` Note how `_shims` is attributed to `dill._dill` and `settings` is attributed to `__main__`(???). But currently they are not saved as globals. --- dill/_dill.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/dill/_dill.py b/dill/_dill.py index 5b599b30..40fd4fcc 100644 --- a/dill/_dill.py +++ b/dill/_dill.py @@ -21,6 +21,8 @@ 'UnpicklingError','HANDLE_FMODE','CONTENTS_FMODE','FILE_FMODE', 'PickleError','PickleWarning','PicklingWarning','UnpicklingWarning'] +__module__ = 'dill' + import logging log = logging.getLogger("dill") log.addHandler(logging.StreamHandler())