Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

a problem about training #15

Open
Minamoto575 opened this issue Dec 6, 2023 · 1 comment
Open

a problem about training #15

Minamoto575 opened this issue Dec 6, 2023 · 1 comment

Comments

@Minamoto575
Copy link

Thanks for your work, I need your help.
I just follow readme, but there is a problem when running pymic_train config/unet.cfg
I didn't modify any configuration or code, there is stack:

Traceback (most recent call last): File "\\?\C:\Users\91119\anaconda3\envs\krl\Scripts\pymic_train-script.py", line 33, in <module> sys.exit(load_entry_point('PYMIC==0.4.0', 'console_scripts', 'pymic_train')()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\91119\anaconda3\envs\krl\Lib\site-packages\pymic-0.4.0-py3.11.egg\pymic\net_run\train.py", line 103, in main File "C:\Users\91119\anaconda3\envs\krl\Lib\site-packages\pymic-0.4.0-py3.11.egg\pymic\net_run\agent_abstract.py", line 316, in run File "C:\Users\91119\anaconda3\envs\krl\Lib\site-packages\pymic-0.4.0-py3.11.egg\pymic\net_run\agent_seg.py", line 333, in train_valid File "C:\Users\91119\anaconda3\envs\krl\Lib\site-packages\torch\utils\data\dataloader.py", line 438, in __iter__ return self._get_iterator() ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\91119\anaconda3\envs\krl\Lib\site-packages\torch\utils\data\dataloader.py", line 386, in _get_iterator return _MultiProcessingDataLoaderIter(self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\91119\anaconda3\envs\krl\Lib\site-packages\torch\utils\data\dataloader.py", line 1039, in __init__ w.start() ^^^^^^^^^^^^^^^^^^ File "C:\Users\91119\anaconda3\envs\krl\Lib\multiprocessing\popen_spawn_win32.py", line 94, in __init__ reduction.dump(process_obj, to_child) File "C:\Users\91119\anaconda3\envs\krl\Lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'NetRunAgent.create_dataset.<locals>.worker_init_fn' Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\91119\anaconda3\envs\krl\Lib\multiprocessing\spawn.py", line 122, in spawn_main exitcode = _main(fd, parent_sentinel) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\91119\anaconda3\envs\krl\Lib\multiprocessing\spawn.py", line 132, in _main self = reduction.pickle.load(from_parent) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ EOFError: Ran out of input

@Minamoto575
Copy link
Author

the key is log AttributeError: Can't pickle local object 'NetRunAgent.create_dataset.<locals>.worker_init_fn'
maybe this function in agent_abstract.py in line 252 lead to this problem, I'm not familiar with python

           if(self.deterministic):
                def worker_init_fn(worker_id):
                    # workder_seed = self.random_seed+worker_id 
                    workder_seed = torch.initial_seed() % 2 ** 32
                    np.random.seed(workder_seed)
                    random.seed(workder_seed)                    
                worker_init = worker_init_fn
            else:
                worker_init = None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant