Doesn't run with demo data

[chelidphile]

I tried pb-falcon with demo data (https://github.com/pb-cdunn/FALCON-examples)
But it stops soon:(
Does anybody help?

Here is the stderr:

-bash-4.1$ cd run-P7091943e0f9a21.bash.stderr 
-bash: cd: run-P7091943e0f9a21.bash.stderr: •«•£•Ï•Ø•»•Í§«§œ§¢§Í§fi§ª§Û
-bash-4.1$ less run-P7091943e0f9a21.bash.stderr 

executable=${PYPEFLOW_JOB_START_SCRIPT}
+ executable=/yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build/run-P7091943e0f9a21.bash
timeout=${PYPEFLOW_JOB_START_TIMEOUT:-60} # wait 60s by default
+ timeout=60

# Wait up to timeout seconds for the executable to become "executable",
# then exec.
#timeleft = int(timeout)
while [[ ! -x "${executable}" ]]; do
    if [[ "${timeout}" == "0" ]]; then
        echo "timed out waiting for (${executable})"
        exit 77
    fi
    echo "not executable: '${executable}', waiting ${timeout}s"
    sleep 1
    timeout=$((timeout-1))
done
+ [[ ! -x /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build/run-P7091943e0f9a21.bash ]]

/bin/bash ${executable}
+ /bin/bash /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build/run-P7091943e0f9a21.bash
+ '[' '!' -d /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build ']'
+ cd /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build
+ eval '/bin/bash run.sh'
++ /bin/bash run.sh
export PATH=$PATH:/bin
+ export PATH=/home/geadmin/N1GE/bin/lx-amd64:/opt/intel/Compiler/18.0.3/20180410/compilers_and_libraries_2018.3.222/linux/bin/intel64:/opt/intel/Compiler/18.0.3/20180410/compilers_and_libraries_2018.3.222/linux/mpi/intel64/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/bin
+ PATH=/home/geadmin/N1GE/bin/lx-amd64:/opt/intel/Compiler/18.0.3/20180410/compilers_and_libraries_2018.3.222/linux/bin/intel64:/opt/intel/Compiler/18.0.3/20180410/compilers_and_libraries_2018.3.222/linux/mpi/intel64/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/bin
cd /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build
+ cd /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build
/bin/bash task.sh
+ /bin/bash task.sh
task.sh: line 18: python3: •≥•fi•Û•…§¨∏´§ƒ§´§Í§fi§ª§Û
+++ pwd
++ echo 'FAILURE. Running top in /yshare2/home/hshibata/FALCON-examples/run/greg200k-sv2/0-rawreads/build (If you see -terminal database is inaccessible- you are using the python bin-wrapper, so you will not get diagnostic info. No big deal. This process is crashing anyway.)'
++ rm -f top.txt
++ which python
++ which top
++ env -u LD_LIBRARY_PATH top -b -n 1
++ env -u LD_LIBRARY_PATH top -b -n 1
++ pstree -apl

real    0m0.534s
user    0m0.007s
sys     0m0.018s
+ finish
+ echo 'finish code: 127'

[pb-cdunn]

task.sh: line 18: python3: •≥•fi•Û•…§¨∏´§ƒ§´§Í§fi§ª§Û

???

Try

cd 0-rawreads/build
bash -vex user_script.sh

If that doesn't work, try running bits of that scripts. There must be a simple explanation.

[chelidphile]

Hi, I tried it and here are what I get:

-bash-4.2$ bash -vex ../run_Falcon_habu_111319.sh
module () { eval /usr/bin/modulecmd bash $*
}
#! /bin/bash
#$ -S /bin/bash
#$ -cwd
#$ -l os7,s_vmem=48G,mem_req=48G

run it!

~/miniconda3/envs/falcon_env/bin/fc_run /home/hshibata/FALCON-examples/run/greg200k-sv2/fc_run.cfg

• /home/hshibata/miniconda3/envs/falcon_env/bin/fc_run /home/hshibata/FALCON-examples/run/greg200k-sv2/fc_run.cfg
  falcon-kit 1.8.0 (pip thinks "falcon-kit 1.8.0")
  pypeflow 2.3.0
  [INFO]Setup logging from file "None".
  [INFO]$ lfs setstripe -c 12 /yshare2/home/hshibata/Falcon_demo_run_111819/0-rawreads >
  [WARNING]'lfs setstripe -c 12 /yshare2/home/hshibata/Falcon_demo_run_111819/0-rawreads' failed to produce any output.
  [INFO]Lustre filesystem detected. This lfs stripe (12) should propagate to subdirs of '/yshare2/home/hshibata/Falcon_demo_run_111819/0-rawreads'.
  [INFO]fc_run started with configuration /home/hshibata/FALCON-examples/run/greg200k-sv2/fc_run.cfg
  [WARNING]Please supply a default for "njobs" (aka concurrency) in section [job.defaults]. For now, we will use 8
  [INFO]cfg=
  {
  "General": {
  "LA4Falcon_preload": false,
  "avoid_text_file_busy": true,
  "bestn": 12,
  "dazcon": false,
  "falcon_sense_greedy": false,
  "falcon_sense_option": "--output-multi --min-cov-aln 4 --min-idt 0.70 --min-cov 4 --max-n-read 200 --n-core 8",
  "falcon_sense_skip_contained": false,
  "fc_ovlp_to_graph_option": " --min-len 1",
  "genome_size": "200000",
  "input_fofn": "input.fofn",
  "input_type": "raw",
  "length_cutoff": "1000",
  "length_cutoff_pr": "1",
  "overlap_filtering_setting": "--max-diff 120 --max-cov 120 --min-cov 2 --n-core 8",
  "ovlp_DBdust_option": "",
  "ovlp_DBsplit_option": "-s5",
  "ovlp_HPCdaligner_option": "-v -B128 -M24",
  "ovlp_daligner_option": "-e.9 -l2500 -k24 -h1024 -w6 -s100",
  "pa_DBdust_option": "",
  "pa_DBsplit_option": "-a -x500 -s4",
  "pa_HPCTANmask_option": "-l500",
  "pa_HPCdaligner_option": "-v -B128 -M24",
  "pa_REPmask_code": "0,300/0,300/0,300",
  "pa_daligner_option": "-e.8 -l3200 -k18 -h480 -w8 -s100",
  "pa_dazcon_option": "-j 4 -x -l 500",
  "pa_fasta_filter_option": "streamed-internal-median",
  "pa_subsample_coverage": 0,
  "pa_subsample_random_seed": 12345,
  "pa_subsample_strategy": "random",
  "seed_coverage": 20,
  "skip_checks": false,
  "target": "assembly"
  },
  "job.defaults": {
  "NPROC": "4",
  "job_type": "string",
  "njobs": 8,
  "pwatcher_type": "blocking",
  "submit": "bash -C ${CMD} >| ${STDOUT_FILE} 2>| ${STDERR_FILE}",
  "use_tmpdir": false
  },
  "job.step.asm": {
  "NPROC": "32"
  },
  "job.step.cns": {
  "NPROC": "8"
  },
  "job.step.da": {},
  "job.step.dust": {},
  "job.step.la": {},
  "job.step.pda": {},
  "job.step.pla": {}
  }
  [INFO]In simple_pwatcher_bridge, pwatcher_impl=<module 'pwatcher.blocking' from '/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/pwatcher/blocking.py'>
  [INFO]job_type='string', (default)job_defaults={'job_type': 'string', 'pwatcher_type': 'blocking', 'submit': 'bash -C ${CMD} >| ${STDOUT_FILE} 2>| ${STDERR_FILE}', 'NPROC': '4', 'njobs': 8, 'use_tmpdir': False}, use_tmpdir=False, squash=True, job_name_style=0
  [INFO]Setting max_jobs to 8; was None
  [INFO]Num unsatisfied: 2, graph: 2
  [INFO]About to submit: Node(0-rawreads/build)
  [INFO]Popen: 'bash -C /home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/pwatcher/mains/job_start.sh >| /yshare2/home/hshibata/Falcon_demo_run_111819/0-rawreads/0-rawreads/build/run-P92df9074051f1c.bash.stdout 2>| /yshare2/home/hshibata/Falcon_demo_run_111819/0-rawreads/0-rawreads/build/run-P92df9074051f1c.bash.stderr'
  [INFO](slept for another 0.0s -- another 1 loop iterations)
  [INFO](slept for another 0.30000000000000004s -- another 2 loop iterations)
  [ERROR]Task Node(0-rawreads/build) failed with exit-code=1
  [ERROR]Some tasks are recently_done but not satisfied: {Node(0-rawreads/build)}
  [ERROR]ready: set()
  submitted: set()
  [ERROR]Noop. We cannot kill blocked threads. Hopefully, everything will die on SIGTERM.
  Traceback (most recent call last):
  File "/home/hshibata/miniconda3/envs/falcon_env/bin/fc_run", line 11, in
  load_entry_point('falcon-kit==1.8.0', 'console_scripts', 'fc_run')()
  File "/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/falcon_kit/mains/run1.py", line 706, in main
  main1(argv[0], args.config, args.logger)
  File "/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/falcon_kit/mains/run1.py", line 73, in main1
  input_fofn_fn=input_fofn_fn,
  File "/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/falcon_kit/mains/run1.py", line 235, in run
  dist=Dist(NPROC=4, MB=4000, job_dict=config['job.step.da']),
  File "/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/falcon_kit/pype.py", line 109, in gen_parallel_tasks
  wf.refreshTargets()
  File "/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/pypeflow/simple_pwatcher_bridge.py", line 278, in refreshTargets
  self._refreshTargets(updateFreq, exitOnFailure)
  File "/home/hshibata/miniconda3/envs/falcon_env/lib/python3.7/site-packages/pypeflow/simple_pwatcher_bridge.py", line 362, in _refreshTargets
  raise Exception(msg)
  Exception: Some tasks are recently_done but not satisfied: {Node(0-rawreads/build)}
  -bash-4.2$

[armintoepfer]

Closing due to lack of activity.