RuntimeError: BlasrService returned a non-zero exit status -6:

[starsyi]

Hi, when I used smrtlink 6.0.0 run resequecing to polish my reference, when running to pbalign step, I get the following errors . but some pbalign task can continue run successfully to the end.

[INFO] 2018-12-09 10:54:51,018Z [pbsmrtpipe.opts_graph _default_nchunks 186] Default nchunks 128
[INFO] 2018-12-09 10:54:51,020Z [pbsmrtpipe.graph.bgraph resolve_io_files 1587] ('mapped', 'alignmentset.xml')
[ERROR] 2018-12-09 10:54:51,150Z [pbsmrtpipe.engine backticks 81] Return code 2  of cmd bash /home/smrtanalysis/smrtlink/userdata/jobs_root/000/000167/tasks/pbalign.tasks.pbalign-92/cluster.sh
[INFO] 2018-12-09 10:54:51,151Z [pbsmrtpipe.tools.runner run_task_on_cluster 461] Cluster command return code 2 in 2233.35 sec
[INFO] 2018-12-09 10:54:51,152Z [pbsmrtpipe.tools.runner run_task_on_cluster 465] 2018-12-09 18:54:51.152634 Completed running cluster command in 2233.35 sec. Exit code 2 (task-type pbalign.tasks.pbalign)
[INFO] 2018-12-09 10:54:51,184Z [pbsmrtpipe.tools.runner run_task_on_cluster 500] Writing task id pbalign.tasks.pbalign task final report to /home/smrtanalysis/smrtlink/userdata/jobs_root/000/000167/tasks/pbalign.tasks.pbalign-92/task-report.json
[INFO] 2018-12-09 10:54:51,189Z [pbsmrtpipe.engine run 649] exiting Worker worker-task-pbalign.tasks.pbalign-92 (pid 48311) task-id:pbalign.tasks.pbalign-92 task-uuid:dd3b310b-a310-4fbb-8261-9bf083bbab18 TaskManifestWorker.run
[ERROR] 2018-12-09 10:54:51,652Z [status.pbsmrtpipe.driver _log_task_failure_and_call_services 492] Task pbalign.tasks.pbalign-92 task pbalign.tasks.pbalign failed (exit-code 2) after 2233.35 sec
Failed task pbalign.tasks.pbalign exit code 2 in 2230.90 sec (See file '/home/smrtanalysis/smrtlink/userdata/jobs_root/000/000167/tasks/pbalign.tasks.pbalign-92/stderr'.)
Extracted from stderr
BlasrService returned a non-zero exit status -6: 
[]
BlasrService returned a non-zero exit status -6: 
[]
Traceback (most recent call last):
  File "/home/smrtanalysis/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrttools/install/smrttools-release_6.0.0.47835/private/thirdparty/python/python_2.7.9/site-packages/pbcommand/cli/core.py", line 138, in _pacbio_main_runner
    return_code = exe_main_func(*args, **kwargs)
  File "/home/smrtanalysis/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrttools/install/smrttools-release_6.0.0.47835/private/thirdparty/python/python_2.7.9/site-packages/pbalign/pbalignrunner.py", line 293, in _resolved_tool_contract_runner
    return args_runner(args, output_dataset_type=output_dataset_type)
  File "/home/smrtanalysis/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrttools/install/smrttools-release_6.0.0.47835/private/thirdparty/python/python_2.7.9/site-packages/pbalign/pbalignrunner.py", line 284, in args_runner
    return PBAlignRunner(args, output_dataset_type=output_dataset_type).start()
  File "/home/smrtanalysis/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrttools/install/smrttools-release_6.0.0.47835/private/thirdparty/python/python_2.7.9/site-packages/pbcore/util/ToolRunner.py", line 85, in start
    return self.run()
  File "/home/smrtanalysis/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrttools/install/smrttools-release_6.0.0.47835/private/thirdparty/python/python_2.7.9/site-packages/pbalign/pbalignrunner.py", line 240, in run
    self._alnService.run()
  File "/home/smrtanalysis/smrtlink/install/smrtlink-release_6.0.0.47841/bundles/smrttools/install/smrttools-release_6.0.0.47835/private/thirdparty/python/python_2.7.9/site-packages/pbalign/alignservice/align.py", line 198, in run
    raise RuntimeError(str(e))
RuntimeError: BlasrService returned a non-zero exit status -6: 
[]




[ERROR] 2018-12-09 10:54:51,653Z [pbsmrtpipe.driver _log_task_failure_and_call_services 493] Task pbalign.tasks.pbalign-92 task pbalign.tasks.pbalign failed (exit-code 2) after 2233.35 sec
Failed task pbalign.tasks.pbalign exit code 2 in 2230.90 sec (See file '/home/smrtanalysis/smrtlink/userdata/jobs_root/000/000167/tasks/pbalign.tasks.pbalign-92/stderr'.)
Extracted from stderr

my version, Workflow and Task Options:as follow:

pbsmrtpipe 0.66.0
Workflow Options:
[INFO] 2018-12-11 02:27:39,663Z [status.pbsmrtpipe.driver_utils job_resource_create_and_setup_logs 388] 
{   'pbsmrtpipe.options.chunk_mode': True,
    'pbsmrtpipe.options.cluster_manager': '/home/smrtanalysis/smrtlink/userdata/generated/config/jms_templates',
    'pbsmrtpipe.options.debug_mode': False,
    'pbsmrtpipe.options.distributed_mode': True,
    'pbsmrtpipe.options.exit_on_failure': False,
    'pbsmrtpipe.options.max_nchunks': 40,
    'pbsmrtpipe.options.max_nproc': 20,
    'pbsmrtpipe.options.max_nworkers': 100,
    'pbsmrtpipe.options.max_total_nproc': 800,
    'pbsmrtpipe.options.progress_status_url': None,
    'pbsmrtpipe.options.tmp_dir': '/home/smrtanalysis/smrtlink/userdata/tmp_dir'}
[INFO] 2018-12-11 02:27:39,683Z [status.pbsmrtpipe.driver_utils job_resource_create_and_setup_logs 390] Task Options:
[INFO] 2018-12-11 02:27:39,698Z [status.pbsmrtpipe.driver_utils job_resource_create_and_setup_logs 391] 
{   u'genomic_consensus.task_options.algorithm': 'arrow',
    u'genomic_consensus.task_options.gff2bed_purpose': 'variants',
    u'genomic_consensus.task_options.masking': True,
    u'genomic_consensus.task_options.min_confidence': 40,
    u'genomic_consensus.task_options.min_coverage': 5,
    u'genomic_consensus.task_options.track_description': 'PacBio: snps, insertions, and deletions derived from consensus calls against reference',
    u'genomic_consensus.task_options.track_name': 'variants',
    u'genomic_consensus.task_options.use_score': 0,
    u'pbalign.task_options.algorithm_options': '--minMatch 12 --bestn 10 --minPctSimilarity 70.0 --refineConcordantAlignments',
    u'pbalign.task_options.concordant': True,
    u'pbalign.task_options.consolidate_aligned_bam': False,
    u'pbalign.task_options.consolidate_n_files': 1,
    u'pbalign.task_options.hit_policy': 'randombest',
    u'pbalign.task_options.min_accuracy': 70.0,
    u'pbalign.task_options.min_length': 50,
    u'pbalign.task_options.no_split_subreads': False,
    u'pbcoretools.task_options.downsample_factor': 0,
    u'pbcoretools.task_options.other_filters': '',
    u'pbcoretools.task_options.read_length': 0,
    u'pbreports.task_options.batch_sort_size': 10000,
    u'pbreports.task_options.force_num_regions': False,
    u'pbreports.task_options.how_many': 100,
    u'pbreports.task_options.max_contigs': 25,
    u'pbreports.task_options.max_region_size': 100000,
    u'pbreports.task_options.num_regions': 1000,
    u'pbreports.task_options.region_size': 0}
[INFO] 2018-12-11 02:27:40,061Z [pbsmrtpipe.driver_utils job_resource_create_and_setup_logs 399] wrote current env to /home/smrtanalysis/smrtlink/userdata/jobs_root/000/000167/workflow/.env.json
[INFO] 2018-12-11 0

Urgently need your help,
and Thanks.

[ylipacbio]

Would you please try latest bioconda release of pbalign? It provides better error logging.

A better alternative way is to use pbmm2 to map pacbio reads to reference (Yes we are deprecating pbalign).
pbmm2 is also released on bioconda, and its document is https://github.com/PacificBiosciences/pbmm2

[mhsieh]

(this is not an official pacbio support)
The SMRT Link 6.0.0 issue you encountered have been resolved in the next version, but if you insist to use SMRT Link 6.0.0 in you current environment, you can however do this

sed -i -e 's/-m 4G/-m 768M/g' /home/smrtanalysis/smrtlink/current/bundles/smrttools/current/private/pacbio/pythonpkgs/pbalign/lib/python2.7/site-packages/pbalign/bampostservice.py

(this is not an official pacbio support)

[starsyi]

(this is not an official pacbio support)
The SMRT Link 6.0.0 issue you encountered have been resolved in the next version, but if you insist to use SMRT Link 6.0.0 in you current environment, you can however do this

sed -i -e 's/-m 4G/-m 768M/g' /home/smrtanalysis/smrtlink/current/bundles/smrttools/current/private/pacbio/pythonpkgs/pbalign/lib/python2.7/site-packages/pbalign/bampostservice.py

(this is not an official pacbio support)

thanks for your help, but I can't run successfully through you method. It doesn't seem to be the cause, because there are 256GB of memory per compute node in our server.

[armintoepfer]

In that case, please contact PacBio's tech support, since this is not related to the bioconda distribution.