Fix re-redistribution

[marchdf]

No description provided.

[baperry2]

Can you update the Docs to reflect the new re-re capability? https://amrex-combustion.github.io/PeleC/geometry/EB.html#re-redistribution

Also, I tried running the sod-4 case on Macbook and it fails with the following error:

[Level 0 step 1] Advanced 32768 cells
[Level 1 step 1] ADVANCE at time 0 with dt = 0.0003961660569
... Computing MOL source term at t^{n}
SIGILL Invalid, privileged, or ill-formed instruction
See Backtrace.0 file for details

(base) bperry-38649s:Sod bperry$ cat Backtrace.0
0: amrex::BLBackTrace::print_backtrace_info(__sFILE*) (in PeleC3d.gnu.ex) (AMReX_BLBackTrace.cpp:189)
1: amrex::BLBackTrace::handler(int) (in PeleC3d.gnu.ex) (AMReX_BLBackTrace.cpp:99)
2: _sigtramp (in libsystem_platform.dylib) + 56
4: amrex::EBFluxRegister::FineAdd(amrex::MFIter const&, std::__1::array<amrex::FArrayBox const*, 3ul> const&, double const*, double, amrex::FArrayBox const&, std::__1::array<amrex::FArrayBox const*, 3ul> const&, amrex::FArrayBox const&, amrex::RunOn) (in PeleC3d.gnu.ex) (AMReX_EBFluxRegister.cpp:149)
5: PeleC::update_flux_registers(double, amrex::MFIter const&, amrex::FabType const&, std::__1::array<amrex::FArrayBox const*, 3ul> const&, amrex::FArrayBox const&) (in PeleC3d.gnu.ex) (PeleC.cpp:0)
6: PeleC::getMOLSrcTerm(amrex::MultiFab const&, amrex::MultiFab&, double, double, double) (in PeleC3d.gnu.ex) (Diffusion.cpp:0)

[marchdf]

Updated the docs, good catch.

I will look into sod-4... I could swear I was using debug just fine...

[baperry2]

Hmm it seems to run fine for me in Debug but fails in regular mode. The most fun kind of issue to debug haha

[jrood-nrel]

I'm worried that error is buffer overflowing into instruction pointer code and why it's saying bad instruction.

[jrood-nrel]

What does ASAN say?

[marchdf]

Documenting here for posterity:

What Bruce saw is a typical thing we see on OSX and clang every so often. We guard against this in cmake with maytrap https://github.com/AMReX-Combustion/PeleC/blob/development/CMake/SetAmrexCompileFlags.cmake#L8. This doesn't happen on llvm+linux. So it's hard to replicate... I commented out the trapping so we only do it when we actually want to (which should just be in debug mode).

[marchdf]

We are going to wait on AMReX-Codes/amrex#3506 before we merge this

[baperry2]

Thanks for updating the documentation.

I can successfully run the test case with fpe trapping turned off and the results look reasonable.

[marchdf]

Pics of what we can do now:

EB-C8:

Sod + sphere:

[drummerdoc]

Excellent! Getting better all the time!