Failed to loads a dumped MetaClass object

[yitso]

Hello, I met a problem when i call cloudpickle.loads.

This is my code:

import cloudpickle
from scrapy.item import Item, Field

MyItem = type('MyItem', (Item,), dict({'id': Field()}))
item = MyItem()
print(cloudpickle.loads(cloudpickle.dumps(item))) # {}
print(isinstance(dict().values(), MyItem)) # False
print(cloudpickle.loads(cloudpickle.dumps(item))) # raise 

I guess the cache of MetaClass cause to this problem.

isinstance call __instancecheck__ of ABCMeta and finally call __subclasscheck__ of ABCMeta.
code: https://github.com/python/cpython/blob/3.5/Lib/abc.py#L210

And cloudpickle.dumps will dumps the _abc_cache and _abc_negative_cache which has some invalid object, so when I call cloudpickle.loads in second , it raise exception.

[pierreglaser]

Hi, thanks for reporting. I can reproduce on python3.6, your example raises:

Traceback (most recent call last):                                                                                                                                                                                 
  File "test_scrapy.py", line 8, in <module>                                                                                                                                                                       
    print(cloudpickle.loads(cloudpickle.dumps(item)))  # raise                                                                                                                                                     
  File "/home/pierreglaser/repos/cloudpickle/cloudpickle/cloudpickle.py", line 1308, in _rehydrate_skelet                                                                                                          
on_class                                                                                                                                                                                                           
    setattr(skeleton_class, attrname, attr)                                                                                                                                                                        
TypeError: can't set attributes of built-in/extension type 'dict_values'   

I'll look into it.

[pierreglaser]

Indeed, the error comes from the fact that the negative cache of MyItem got populated with an unpickleable class (dict_values) during

isinstance(dict().values(), MyItem)

In general, I would lean towards not pickling the abc caches of a specifc class: we don't need the objects in the cache to pickle the said class, and this actually may make the final pickle unnecessarily large.

[yitso]

Thanks for your reply.
not pickling the abc caches is great idea~