fix 'pure virtual method called' error

[chongchonghe]

Description

Move DeviceVector inside setInitialConditionsOnGrid to avoid the pure virtual method called error, first seen in #761 .

This branch is based on dependabot/submodules/extern/amrex-6d9c25b .

Related issues

Fix issues in PR #761

Checklist

Before this pull request can be reviewed, all of these tasks should be completed. Denote completed tasks with an x inside the square brackets [ ] in the Markdown source below:

• I have added a description (see above).
• I have added a link to any related issues see (see above).
• I have read the Contributing Guide.
• I have added tests for any new physics that this PR adds to the code.
• I have tested this PR on my local computer and all tests pass.
• I have manually triggered the GPU tests with the magic comment /azp run.
• I have requested a reviewer for this PR.

[chongchonghe]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[chongchonghe]

@BenWibking This solves the problem, although I don't understand exactly why. It seems like the cause of the error is because amrex::Gpu::DeviceVector is declared globally and accessed inside the CPU function preCalculateInitialCondition. I moved everything in preCalculateInitialCondition to setInitialConditionsOnGrid and the issue is gone.

Missed important issue

[BenWibking]

@BenWibking This solves the problem, although I don't understand exactly why. It seems like the cause of the error is because amrex::Gpu::DeviceVector is declared globally and accessed inside the CPU function preCalculateInitialCondition. I moved everything in preCalculateInitialCondition to setInitialConditionsOnGrid and the issue is gone.

Ok, that makes sense. The issue is that global objects are destroyed after main() ends, so it tries to deallocate the device memory (or maybe perform other operations) after amrex::Finalize() has been called. However, the GPU is not accessible after calling amrex::Finalize(). I'm not actually sure how it worked before.

We don't want to re-read the file for each grid, though, so we still need to do this inside preCalculateInitialConditions. You can move the DeviceVectors to SimulationData<TubeProblem> instead of global variables.

[BenWibking]

You can move the variables into SimulationData in the same way you did so here:

quokka/src/problems/RadDustMG/test_rad_dust_MG.cpp

Line 35 in 5f84397

template <> struct SimulationData<DustProblem> {

The only difference is that you need a DeviceVector instead of a std::vector.

[chongchonghe]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 2 pipeline(s).

[sonarcloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[chongchonghe]

@BenWibking @markkrumholz Should be ready to merge.