multiprocessing

[kellermac]

Hi again! I was hoping you might be able to trace the root issue of my problem. I am trying to plot a heatmap using a gff file for signal (this file is large 9.1Mb). I tried running it on my local machine but kept getting errors like this:
NotifierThreadProc: could not create trigger pipe
NotifierThreadProc: could not create trigger pipe
NotifierThreadProc: could not create trigger pipe
NotifierThreadProc: could not create trigger pipe
NotifierThreadProc: could not create trigger pipe
Traceback (most recent call last):
File "heatmo.py", line 18, in
Bkornabf_signal = Bkornabf_signal.array(sites_1kb, bins=100, processes=3)
File "/usr/local/lib/python2.7/dist-packages/metaseq/_genomic_signal.py", line 122, in array
chunksize=chunksize, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/metaseq/array_helpers.py", line 383, in _array_parallel
itertools.repeat(kwargs)))
File "/usr/lib/python2.7/multiprocessing/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/usr/lib/python2.7/multiprocessing/pool.py", line 558, in get
raise self._value
KeyError: "unknown preset 'empty', valid presets are 'vcf,sam,bed,psltbl,pileup,gff'"

After inspecting the genomic_signal script and converting my gff to bed I kept getting similar errors. So I moved my data to a bigger machine with 16 processors. This seemed to accommodate the "triggered pipe" but I get the following message:

/home/keller/anaconda2/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.
warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')
[E::bgzf_flush] hwrite error (wrong size)
Traceback (most recent call last):
File "heatmo.py", line 20, in
Bkornabf_signal = Bkornabf_signal.array(sites_1kb, bins=100, processes=processes)
File "/home/keller/anaconda2/lib/python2.7/site-packages/metaseq/_genomic_signal.py", line 122, in array
chunksize=chunksize, **kwargs)
File "/home/keller/anaconda2/lib/python2.7/site-packages/metaseq/array_helpers.py", line 383, in _array_parallel
itertools.repeat(kwargs)))
File "/home/keller/anaconda2/lib/python2.7/multiprocessing/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/home/keller/anaconda2/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
OSError: writing failed

[daler]

No idea, I haven't seen this before. Can you provide an example with data that reproduces the problem?

[kopardev]

I am getting the same error:
"""
Traceback (most recent call last):
File "../scripts/bam2tssscore.v2.py", line 70, in
bam_array=bam.array(features = tss_ext,bins = bins,processes = args.ncpu)
File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/site-packages/metaseq/_genomic_signal.py", line 122, in array
chunksize=chunksize, **kwargs)
File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/site-packages/metaseq/array_helpers.py", line 383, in _array_parallel
itertools.repeat(kwargs)))
File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/multiprocessing/pool.py", line 253, in map
return self.map_async(func, iterable, chunksize).get()
File "/usr/local/Anaconda/envs/py2.7/lib/python2.7/multiprocessing/pool.py", line 572, in get
raise self._value
KeyError: "unknown preset 'empty', valid presets are 'psltbl,gff,sam,bed,vcf'"
"""
Any idea? Any solutions?

[kopardev]

I am still getting the same error... Any solution guys?