Multiple node computation

[jk262626]

Hi Nakib,
Would it be allowed to run the Elphbolt using multiple nodes?
Say for example, use 2 compute nodes with 50 cores per node.

[nakib]

Dear jk262626,

Of course. I think the largest computation I have ever done with this code was with 10 nodes x 36 cores/node. Note that since the code does a large amount of disk i/o, beyond a certain number of cores, whatever benefit you would get from parallelization might get diminished by the i/o latency. It is all system dependent and the best way to proceed is to first to do a benchmark on the cluster you are using. I'd appreciate it if you reported back your experience.

Best regards,
Nakib

[jk262626]

I'm a little bit confused about whether I should use 360 or 36 for the number of coarray images when I use 10 nodes x 36 cores/node.
(in the input script, "cafrun -n 360 elphbolt" this line)

And another thing I wonder, what was the memory per node for your computation?
I am running (50, 150) mesh file with 1 node x 25 cores/node under the computation node with 256GB memory and fails to run.

[nakib]

The number of images should be 360. Every core is an image.

The amount of memory required depends on the sizes of the various wannierized quantities. I can't comment without knowing some of the details of the parameters of your calculation. For all of my calculations, the number of Wannier bands was quite small, so memory was never an issue.

Best,
Nakib

[jk262626]

I see, it is good to know. Can I use just a few Wannier band around transport bands? Best regards, Jiyoung Kim *----------------------------------------------------------------------------------------------* *Jiyoung Kim* Ph.D. Student Sibley School of Mechanical and Aerospace Engineering| Cornell University Ithaca, NY 14850 | ***@***.*** ***@***.***> *Gender Pronouns: she, her, hers*

…

On Wed, Jul 3, 2024 at 9:05 AM Nakib H. Protik ***@***.***> wrote: The number of images should be 360. Every core is an image. The amount of memory required depends on the sizes of the various wannierized quantities. I can't comment without knowing some of the details of the parameters of your calculation. For all of my calculations, the number of Wannier bands was quite small, so memory was never an issue. Best, Nakib — Reply to this email directly, view it on GitHub <#143 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BA5LQ5I26IROT4WJYKZQPLLZKPZH7AVCNFSM6AAAAABKIB5346VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TSNBYGA3DC> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[nakib]

Yes. You already choose the number of transport active bands in elphbolt. But the number of Wannier bands included in all the wannierized files is what dictates how much space will be needed. You can check in this paper how the wannerized quantities scale in space: https://journals.aps.org/prb/pdf/10.1103/PhysRevB.76.165108

Best,
Nakib

[jk262626]

Awesome, thanks for the info!

…

On Wed, Jul 3, 2024 at 9:16 AM Nakib H. Protik ***@***.***> wrote: Yes. You already choose the number of transport active bands in elphbolt. But the number of Wannier bands included in all the wannierized files is what dictates how much space will be needed. You can check in this paper how the wannerized quantities scale in space: https://journals.aps.org/prb/pdf/10.1103/PhysRevB.76.165108 Best, Nakib — Reply to this email directly, view it on GitHub <#143 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BA5LQ5PO6FLOCVJ6AMLJEU3ZKP2S7AVCNFSM6AAAAABKIB5346VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TSNBYGE4DQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>