Skip to content

Commit

Permalink
Add global domain id offset to conduit wrapper (#3524)
Browse files Browse the repository at this point in the history
## Summary
Make sure all domains have unique domain ids by adding a global domain
offset

## Additional background
Currently meshes and particles will have non unique domain ids. 

## Checklist

The proposed changes:
- [x] fix a bug or incorrect behavior in AMReX
- [ ] add new capabilities to AMReX
- [ ] changes answers in the test suite to more than roundoff level
- [ ] are likely to significantly affect the results of downstream AMReX
users
- [ ] include documentation in the code and/or rst files, if appropriate
  • Loading branch information
nicolemarsaglia authored Sep 5, 2023
1 parent 9a85e50 commit 36dac3a
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
6 changes: 5 additions & 1 deletion Src/Extern/Conduit/AMReX_Conduit_Blueprint.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,10 @@ MultiLevelToBlueprint (int n_levels,
int rank = ParallelDescriptor::MyProc();
int ntasks = ParallelDescriptor::NProcs();

// get global domains already present in node
long domain_offset = (long)res.number_of_children();
ParallelDescriptor::ReduceLongSum(domain_offset);

Vector<const BoxArray*> box_arrays;
Vector<int> box_offsets;

Expand Down Expand Up @@ -398,7 +402,7 @@ MultiLevelToBlueprint (int n_levels,
for(MFIter mfi(mf); mfi.isValid(); ++mfi)
{
// domain_id is mfi.index + all patches on lower levels
int domain_id = mfi.index() + num_domains;
int domain_id = mfi.index() + num_domains + domain_offset;
const std::string& patch_name = amrex::Concatenate("domain_",
domain_id,
6);
Expand Down
6 changes: 5 additions & 1 deletion Src/Extern/Conduit/AMReX_Conduit_Blueprint_ParticlesI.H
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,10 @@ ParticleContainerToBlueprint(const ParticleContainer<NStructReal,
int num_levels = pc.maxLevel() + 1;
int num_domains = 0;

// get global domains already present in node
long domain_offset = (long)res.number_of_children();
ParallelDescriptor::ReduceLongSum(domain_offset);

// loop over levels

int rank = ParallelDescriptor::MyProc();
Expand Down Expand Up @@ -288,7 +292,7 @@ ParticleContainerToBlueprint(const ParticleContainer<NStructReal,
for (MyParConstIter pti(pc, lev); pti.isValid(); ++pti)
{
// this will give us a unique id across levels for each tile
int domain_id = lvl_offset + num_lvl_local_tiles;
int domain_id = lvl_offset + num_lvl_local_tiles + domain_offset;

const auto& ptile = pti.GetParticleTile();
const std::string& patch_name = amrex::Concatenate("domain_",
Expand Down

0 comments on commit 36dac3a

Please sign in to comment.