Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DSL2 nextflow stopped when put into background in shell #1221

Closed
rithy8 opened this issue Jul 10, 2019 · 4 comments
Closed

DSL2 nextflow stopped when put into background in shell #1221

rithy8 opened this issue Jul 10, 2019 · 4 comments

Comments

@rithy8
Copy link

rithy8 commented Jul 10, 2019

Bug report

bash shell running nextflow stopped when running in background mode. When running in foreground everything worked.

Expected behavior and actual behavior

nextflow should continue to run when launched in background mode.

Steps to reproduce the problem

Below is my script. I ran it with the following command line:

nextflow run dotestsub.nf 2>&1 > foo.txt &
And, when I tailed foo.txt, nextflow stuck at this line:

N E X T F L O W  ~  version 19.05.0-edge
Launching `dotestsub.nf` [disturbed_liskov] - revision: cfa14e665c
WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE
[warm up] executor > sge

When I type jobs -l, my system showed the following:

[1]+ 29350 Stopped (tty output)    nextflow run dotestsub.nf 2>&1 > foo.txt
nextflow.preview.dsl=2

process sayHello {
  echo true
  cpus 1
  memory '1 GB'
  container interopImage
  clusterOptions = '-l h_vmem=1G ' + '-S /bin/bash -l docker_version=new -P onco'
  executor = 'sge'
  queue = 'all.q'
  penv = 'smp'

  input:
      val(aaa)
  script:
  """
  echo "hello" > hello.txt
  """
}

sayHello("bbb")

Program output

N E X T F L O W  ~  version 19.05.0-edge
Launching `dotestsub.nf` [disturbed_liskov] - revision: cfa14e665c
WARN: DSL 2 IS AN EXPERIMENTAL FEATURE UNDER DEVELOPMENT -- SYNTAX MAY CHANGE IN FUTURE RELEASE
[warm up] executor > sge

When I type jobs -l, my system showed the following:

[1]+ 29350 Stopped (tty output)    nextflow run dotestsub.nf 2>&1 > foo.txt

Environment

  • Nextflow version: [19.05.0-edge]
  • Java version: [1.8.0_92]
  • Operating system: [CentOS Linux]

Additional context

Please help me. Thanks

@pditommaso
Copy link
Member

When running in the background you should specify the -bg command-line option.

@rithy8
Copy link
Author

rithy8 commented Jul 11, 2019

Oh, did not know nextflow has background option. Thank you.

@rithy8 rithy8 closed this as completed Jul 11, 2019
@grayshiwb512
Copy link

Thank you, @pditommaso !
Does the "-bg" command-line option documented? I search the documentation and cannot find it as shown below,
image

thanks,
Paul

@pditommaso
Copy link
Member

It should, so far the only doc is available through nextflow run -h

TimothyOlsson added a commit to statisticalbiotechnology/quandenser-pipeline that referenced this issue Aug 14, 2019
### Changes in the shell script:

**Major changes:**

* Due to a [new update on singularity 
hub](https://singularityhub.github.io/singularityhub-docs/2019/security-release/), 
 the API get requests are limited, meaning the it will not be possible 
to determine version number by using the Singularity hub API, thus 
preventing updates from the shellscript. Therefore, the new .sh file 
will use github's api to fetch the version number instead. This means 
that both the tag name of  each release in github and the version number 
in the image needs to be  updated on each release while before, only the 
version number in the image was needed to update. **THIS MEANS THAT 
USERS BEFORE 0.08 NEEDS TO UPDATE TO THE NEWEST VERSION OF THE SHELL 
SCRIPT, SINCE THE OLD VERSION WILL NOT AUTOMATICALLY FIND UPDATES**. 
Please download the newest version of the shell script to fix this 
issue.

### Changes in building the image:

**Minor changes:**

* [New version of nextflow requires the user to run it in the 
background](nextflow-io/nextflow#1221):
This breaks several things which will require some rework. With the new 
nextflow version, *run_quandenser.sh* will suspend all processes if the 
tag "-bf" is not added. Adding the "background" tag will break the 
ability to kill processes in the GUI, since the .sh script will no 
longer have nextflow as its child-process. Some functions used in the 
pipeline will be depreciated in future nextflow versions, which might 
cause further problems. Thus, the update for nextflow 19.01 --> 19.07 
causes a lot of issues, which might not be worth fixing at the moment. 
Nextflow version 19.01 will be the default, static version for now.

* Added some permission for the new quandenser version, so users can run 
it without any problems.

### Changes in the image:

**Major changes:**

* A new version of quandenser has been installed 
([v0.02](https://github.com/statisticalbiotechnology/quandenser/releases/tag/rel-0-02)) 
which includes updated versions of submodules and huge improvements to 
speed of parallelization and the stability of the processing.

* A new triqler version has been added 
([v0.03](https://github.com/statisticalbiotechnology/triqler/releases/tag/rel-0-03-0), 
which has several improvements, such as optimizations for very large 
datasets.

**Minor changes:**

* Removed the ability to choose max missing value directly from the GUI. 
The reason is that few users know what the the option does and that it 
should be changed depending on the amount of files available. The 
default value in quandenser takes this into account, but is overwritten 
by the GUI value. The value can still be changed by adding 
"--max-missing" <value> into quandenser's optional parameters. This 
should prevent confusion of new users, but still allow users to change 
the values to their liking.

* Removed parallel step 5, by merging 4 and 5. The last step is not 
necessary and only adds to the total calculation time.

* Fixed some bugs in the nextflow pipeline when running quandenser v0.02

* Fixed some bugs where elements of the GUI are hidden if the window 
size is too small.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants