resource allocation

[xiaoyezao]

Hi, I have questions about the job configuration options:

[job.defaults]
job_type=slurm
pwatcher_type=blocking
JOB_QUEUE = test
JOB_NAME = 200kb_test
MB = 32768
NPROC = 12
njobs = 32
submit = srun --wait=0 -p ${JOB_QUEUE} -J ${JOB_NAME} -o ${JOB_STDOUT} -e ${JOB_STDERR} --mem-per-cpu=${MB}M --cpus-per-task=${NPROC} ${JOB_SCRIPT}

I work on a cluster that has 24 cores and 65G RAM per node. Should I set NPROC=24 MB=2700 (65/24)?
How to set njobs? How many jobs are needed to run for a 1.3G plant genome with about 100X pacbio reads?

Thanks

[pb-cdunn]

You want 3GB per CPU, so you will need to configure DALIGNER that way. By default it uses 4 CPUs per task, so you need -M12000 in you daligner_opt configuration. You would need to ensure that your block-size allows processing to fit within 12GB, Gene Myers suggests approximately DB split -s150`.

For job submissions,

submit = srun --wait=0 -p ${JOB_QUEUE} -J ${JOB_NAME} -o ${JOB_STDOUT} -e ${JOB_STDERR} --mem-per-cpu=3000 --cpus-per-task=4 ${JOB_SCRIPT}

That will give you 4 CPUs per task, which means you will end up with . However, some tasks benefit from more, and some need only 1. You might want to override these per-task-group.

Here is a ton of documentation:

• https://github.com/PacificBiosciences/pb-assembly

[xiaoyezao]

Thanks very much for your explanation.
I am still a little bit confused.

You want 3GB per CPU, so you will need to configure DALIGNER that way. By default it uses 4 CPUs per task, so you need -M12000 in you daligner_opt configuration. You would need to ensure that your block-size allows processing to fit within 12GB, Gene Myers suggests approximately DB split -s150`.

1. In my case (24core-65G per node), if --mem-per-cpu=3000 --cpus-per-task=4, does that mean 6 tasks can run simultanueously on one node?

2. If using 4 CPUs by default, can I set --mem-per-cpu=16250, so to fully employ the memory (65G per node) ? I understand that this setting can help to increase the DB split -s, thus speed up the processing. Am I right?

However, some tasks benefit from more, and some need only 1. You might want to override these per-task-group.

3. Is there any empirical knowledge about which task need more, which need only 1?

4. If I want to use multiple nodes, do I need to specified the -N option in submit = srun ... ? or falcon knows how to submit the tasks to multiple nodes?

So sorry for these annoying questions. I am a beginner in this area and your explanation would be appreciated very much!

[pb-cdunn]

In my case (24core-65G per node), if --mem-per-cpu=3000 --cpus-per-task=4, does that mean 6 tasks can run simultanueously on one node?

Yes.

If using 4 CPUs by default, can I set --mem-per-cpu=16250, so to fully employ the memory (65G per node) ? I understand that this setting can help to increase the DB split -s, thus speed up the processing. Am I right?

Basically. DBsplit chooses the block size. Given that, you can assume how much memory daligner needs (and assume 4 cores). You need to pass that to daligner in its -M flag so it limits the kmer table size too. And given all that, you can adjust how you submit jobs. (Yes, it's a bit complicated.)

Is there any empirical knowledge about which task need more, which need only 1?

Unfortunately, we've been a bit lax in that. We're working with other systems internally.

srun ...

I'd rather not starting helping with job-submission systems. You're much better off talking to your sysadmin and some other resource.

Cheers!