Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve MPI scaling of set_maxwell_dielectric #1255

Open
stevengj opened this issue Jun 19, 2020 · 0 comments
Open

improve MPI scaling of set_maxwell_dielectric #1255

stevengj opened this issue Jun 19, 2020 · 0 comments

Comments

@stevengj
Copy link
Collaborator

stevengj commented Jun 19, 2020

Right now, MPB calls back to the meep_mpb_eps Meep for every pixel of the mode profile to find the ε profile for the eigenmode solver, and this performs a collective communication. This is a problem for scaling, because, you then have huge numbers of small collective calls, one per pixel in the eigenmode region.

Should be straightforward to fix. Probably something like:

  • Call set_maxwell_dielectric with a dummy function (just returns 1 everywhere), which caches the inv(ε) value for owned chunks in an array and zero for nonowned chunks. (i.e. pass parallel=false to get_chi1inv).

  • Call sum_to_all once on this cache array to synchronize the values.

  • Call set_maxwell_dielectric again, this time returned the cached values from each process.

Hence, only a single collective call is required in the second step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant