Unable to build REST with Geant4

[zykure]

I'm trying to setup REST on our institute cluster (at KIT). The system is Ubuntu 20.04.3 with GCC 9.4.0 installed. I'm using ROOT version 6.24 and Geant4 version 11.0.1.

I followed the instructions in https://rest-for-physics.github.io/installation and used the pull-submodules.py script on the tagged version 2.3.12 of the REST sources. List of git submodules:
GitStatus.txt

CMake runs through without errors but the build fails with many compiler errors. A common error is that string is undefined (should be std::string or TString) but there are others as well that are less clear. For example "assignment of read-only location" in TRestGeant4Event and various issues with templated classes. The full error log is here:
ErrorLog_Geant11.txt

Using an older version of Geant4 (10.7.3) and ROOT (6.18.04) does not improve the situation.

The build succeeds with just the Detector/Raw/Track modules enabled, so it must have something to do with the Geant4 libraries.

---

I don't really understand why there are so many compiler errors. They seem to point at some underlying issue but I can't tell what could be wrong. What is the recommended system to use? Maybe I can try my luck with a Docker container.

[lobis]

I'm trying to setup REST on our institute cluster (at KIT). The system is Ubuntu 20.04.3 with GCC 9.4.0 installed. I'm using ROOT version 6.24 and Geant4 version 11.0.1.

I followed the instructions in https://rest-for-physics.github.io/installation and used the pull-submodules.py script on the tagged version 2.3.12 of the REST sources. List of git submodules: GitStatus.txt

CMake runs through without errors but the build fails with many compiler errors. A common error is that string is undefined (should be std::string or TString) but there are others as well that are less clear. For example "assignment of read-only location" in TRestGeant4Event and various issues with templated classes. The full error log is here: ErrorLog_Geant11.txt

Using an older version of Geant4 (10.7.3) and ROOT (6.18.04) does not improve the situation.

The build succeeds with just the Detector/Raw/Track modules enabled, so it must have something to do with the Geant4 libraries.

I don't really understand why there are so many compiler errors. They seem to point at some underlying issue but I can't tell what could be wrong. What is the recommended system to use? Maybe I can try my luck with a Docker container.

I will take a look. It should work with Geant4 11.0.1 and older (up to 10.4.3) so this is not an issue.

Could you test the build with Detector/Raw/Track/Geant4 libraries enabled? The Geant4 REST module does not require Geant4 to work, only the restG4 package does.

[jgalan]

Hi, perhaps we need to improve the documentation.

I believe the problems come from an incompatibility between submodules and core libraries.

Make sure you download an official image of REST, described here:
https://rest-for-physics.github.io/downloading.html#downloading-an-official-rest-release-using-git

The trick is to call pull-submodules.py with --clean argument

python3 pull-submodules.py --clean

Double-check that the main framework repository is at commit 0cf8f3c using git log. I.e. release v2.3.12. There is an issue related to curl libraries on that release. You might be interested to just place yourself at the latest master and pull latest submodules.

python3 pull-submodules.py --latest

Likely we should not allow to execute pull-submodules.py without arguments? Which I believe was the origin of the problem at this issue. @lobis ? See the quick patch I introduced to pull-submodules.py at #212

[jgalan]

BTW, in case you go for --latest and if you are using -DREST_ALL_LIBS=ON make sure you disable -DRESTLIB_AXION=OFF because there is now a compilation issue. See #212

[zykure]

Thanks for your help!

First, I ran the pull-submodules.py script again, this time with --clean:

$ python3 pull-submodules.py --clean                                                                                                                                                                                          This will override local changes on the files. And will bring your local repository to a clean state
Are you sure to proceed? (y/n)
            y
Force pulling submodules.
Framework branch name: official
source/libraries/axion[ OK ] (c72de7a)
source/libraries/geant4[ OK ] (9397cd2)
source/libraries/raw[ OK ] (b15d1cc)
source/libraries/track[ OK ] (9cf0fe3)
source/libraries/detector[ OK ] (54ef8f0)
source/libraries/connectors[ OK ] (de207df)
packages/restG4[ OK ] (cc7486c)
packages/restSQL[ OK ] (639a01a)
projects/basic-geometries[ OK ] (992517b)
projects/basic-readouts[ OK ] (a0767ae)
scripts/batch-scripts[ OK ] (c7e6c1c)
/home/ws/kx6822/workspace/rest-framework.git/doc/userguide[ Failed ]
source/libraries/axion/data[ OK ] (20da7fb)
source/libraries/axion/external/solarAxionFlux[ OK ] (82a7cf8)

$ git rev-parse HEAD                                                                                                                                                                                                         
0cf8f3c8dc84bae2d1f39aede01837dd202fd717

• With Detector/Raw/Track/Geant4/G4 the build still fails, same as before.

• With Detector/Raw/Track/Geant4 (suggested by @lobis) the build also fails (with a shorter error log). However the string problem and the "assignment of read-only location" remains: ErrorLog_noG4.txt

• I also tried to build with just the Geant4 module enabled, which has the same problems. I also tried this with GCC 8.4.0, but to no avail.

---

The next approach for me was to try older version of REST and its libraries. It turns out that the Geant4 module builds fine in v2.3.10 and v2.3.11, but not in v2.3.12.

On v2.3.11 I also tested the full build with Detector/Raw/Track/Geant4 and it works fine. When I enable G4 in addition, I'm back to compiler errors.

I then proceeded with some more older versions and tried my full build with Detector/Raw/Track/Geant4/G4.

• v2.3.10 -- fails (geant4lib v1.6 / restG4 v1.5)
• v2.3.9 -- fails (geant4lib v1.5 / restG4 v1.4)
• v2.3.7 -- fails (geant4lib v1.4)
• v2.3.6 -- fails (geant4lib v1.3 / restG4 v1.3)
• v2.3.5 -- fails (geant4lib v1.2 / restG4 v1.2)
• v2.3.0 -- fails

=> In summary: I can build with Geant4 on v2.3.11, but not v2.3.12. The build with G4 fails on all tested versions.

[jgalan]

Probably v2.3.10 and previous are not compatible with the latest Geant4 versions.

When I pull the latest master at the framework repository I am at the following commit

$ git rev-parse HEAD       
72a0c340453d8def38a43ddc8e7fd53c2cb73d5c

I cannot reproduce you commit hash 0cf8f3c8dc84bae2d1f39aede01837dd202fd717 with any v2.3.12, v2.3.11, v2.3.10, v2.3.9.

git checkout tags/v2.3.12
git rev-parse HEAD
792df26e4c8fcc8c058bf77e689ca0cc6ceb5a8e

git checkout tags/v2.3.11
git rev-parse HEAD
eeeaad8340c90f4ff66ae0fe6886acf137316aed

git checkout tags/v2.3.10
git rev-parse HEAD
7148a3e0ec915be05127e135f5838bbca5cdf2fd

...

[zykure]

Okay, that is strange. The commit belongs to v2.3.12 on my git clone.
The commit 0cf8f3c also agrees to what's shown here on GitHub for the tag.

git checkout tags/v2.3.12
git rev-parse HEAD
0cf8f3c8dc84bae2d1f39aede01837dd202fd717

git log
commit 0cf8f3c8dc84bae2d1f39aede01837dd202fd717 (HEAD, tag: v2.3.12, origin/alphaAna)
Merge: 792df26e 7a1761d4
Author: juanan <juanan318@gmail.com>
Date:   Mon Apr 25 16:40:49 2022 +0200

    Merge branch 'master' of github.com:rest-for-physics/framework into alphaAna

On v2.3.11 the commit is also different than yours:

git checkout tags/v2.3.11
git rev-parse HEAD
0c39835a72ffa398cb16d6192874aa9fdc3c9a22

git log
commit 0c39835a72ffa398cb16d6192874aa9fdc3c9a22 (HEAD, tag: v2.3.11, official)
Author: lobis <luis.antonio.obis@gmail.com>
Date:   Mon Mar 14 17:39:56 2022 +0100

    updated rawlib to 05b82eb5ff091128c134652dba8b21b58b687f22

[lobis]

@zykure I think if you use the pull-submodules.py with --latest option it should work.

Perhaps the issue is related to the C++ standard used to compile ROOT and Geant4. Currently the only supported standard is C++17, but it could work with a lower one, but its not garanteed.

REST automatically detects the C++ standard from root, but if you are not using C++17 this may be an issue for some libraries.

Can you please show the output of root-config --cflags and geant4-config --cflags? If they do not show c++17 I think you should recompile them with it.

[zykure]

@lobis Thanks, looks like that was an issue. The system version of ROOT I was using was compiled with C++11. I compiled ROOT (6.24) with C++17, and also compiled Geant4 (11.1) to make sure it's using the same compiler (although the binaries were using C++17 already).

root-config --cflags
-pthread -std=c++17 -m64 -I/kalinka/software/root-6.24.06_cpp17/include
root-config --version
6.24/06

geant4-config --cflags
-DG4VIS_USE_OPENGL -DG4UI_USE_TCSH -DG4VIS_USE_RAYTRACERX -DG4VIS_USE_OPENGLX -W -Wall -pedantic -Wno-non-virtual-dtor -Wno-long-long -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -Wno-variadic-macros -Wshadow -pipe -pthread -ftls-model=initial-exec -std=c++17 -I/kalinka/software/geant4.11.0.1/bin/../include/Geant4
geant4-config --version
11.0.1

The build now succeeds with v2.3.11 of REST when I enable Detector/Raw/Track/Geant4. Adding G4 in addition breaks the build because of a missing include file G4EmProcessOptions.hh which does not seem to be part of Geant4 (but maybe this works with Geant4 10.x). Otherwise there's only a few minor compiler warnings.

Now when I switch to v2.3.12 again and run pull-submodules.py --latest --clean, the build stills fails unfortunately. I still get the "assignment of read-only location" error, among others. This also happens on the master branch (I'm not sure if the --latest option is supposed to work with 2.3.12 so I checked this too.) On the master branch I get much less errors though, mostly about unimplemented virtual methods.

ErrorLog_v2.3.12.txt
ErrorLog_master.txt

Git status using master branch:

git checkout master
git rev-parse HEAD                                                                                                                                                                                           
4a21d782ececfab1d9f75b11663cb599a574be69

python3 ./pull-submodules.py --clean --latest --dontask
Framework branch name: master
source/libraries/axion --> Pulling branch : master  [ OK ] (1c3cc18)
source/libraries/geant4 --> Pulling branch : master  [ OK ] (41653ce)
source/libraries/raw --> Pulling branch : master  [ OK ] (7b1e403)
source/libraries/track --> Pulling branch : master  [ OK ] (2afeaa0)
source/libraries/detector --> Pulling branch : master  [ OK ] (0e023af)
source/libraries/connectors --> Pulling branch : master  [ OK ] (8399296)
packages/restG4 --> Pulling branch : master  [ OK ] (e481d8a)
packages/restSQL --> Pulling branch : master  [ OK ] (639a01a)
projects/basic-geometries --> Pulling branch : master  [ OK ] (080bd50)
projects/basic-readouts --> Pulling branch : master  [ OK ] (10f16e8)
scripts/batch-scripts --> Pulling branch : master  [ OK ] (c7e6c1c)
/home/ws/kx6822/workspace/rest-framework.git/doc/userguide[ Failed ]
source/libraries/axion/data --> Pulling branch : master  [ OK ] (22cf5a4)
source/libraries/axion/external/solarAxionFlux --> Pulling branch : master  [ OK ] (8f2d9bb)

[lobis]

@lobis Thanks, looks like that was an issue. The system version of ROOT I was using was compiled with C++11. I compiled ROOT (6.24) with C++17, and also compiled Geant4 (11.1) to make sure it's using the same compiler (although the binaries were using C++17 already).

root-config --cflags
-pthread -std=c++17 -m64 -I/kalinka/software/root-6.24.06_cpp17/include
root-config --version
6.24/06

geant4-config --cflags
-DG4VIS_USE_OPENGL -DG4UI_USE_TCSH -DG4VIS_USE_RAYTRACERX -DG4VIS_USE_OPENGLX -W -Wall -pedantic -Wno-non-virtual-dtor -Wno-long-long -Wwrite-strings -Wpointer-arith -Woverloaded-virtual -Wno-variadic-macros -Wshadow -pipe -pthread -ftls-model=initial-exec -std=c++17 -I/kalinka/software/geant4.11.0.1/bin/../include/Geant4
geant4-config --version
11.0.1

The build now succeeds with v2.3.11 of REST when I enable Detector/Raw/Track/Geant4. Adding G4 in addition breaks the build because of a missing include file G4EmProcessOptions.hh which does not seem to be part of Geant4 (but maybe this works with Geant4 10.x). Otherwise there's only a few minor compiler warnings.

Now when I switch to v2.3.12 again and run pull-submodules.py --latest --clean, the build stills fails unfortunately. I still get the "assignment of read-only location" error, among others. This also happens on the master branch (I'm not sure if the --latest option is supposed to work with 2.3.12 so I checked this too.) On the master branch I get much less errors though, mostly about unimplemented virtual methods.

ErrorLog_v2.3.12.txt
ErrorLog_master.txt

Git status using master branch:

git checkout master
git rev-parse HEAD                                                                                                                                                                                           
4a21d782ececfab1d9f75b11663cb599a574be69

python3 ./pull-submodules.py --clean --latest --dontask
Framework branch name: master
source/libraries/axion --> Pulling branch : master  [ OK ] (1c3cc18)
source/libraries/geant4 --> Pulling branch : master  [ OK ] (41653ce)
source/libraries/raw --> Pulling branch : master  [ OK ] (7b1e403)
source/libraries/track --> Pulling branch : master  [ OK ] (2afeaa0)
source/libraries/detector --> Pulling branch : master  [ OK ] (0e023af)
source/libraries/connectors --> Pulling branch : master  [ OK ] (8399296)
packages/restG4 --> Pulling branch : master  [ OK ] (e481d8a)
packages/restSQL --> Pulling branch : master  [ OK ] (639a01a)
projects/basic-geometries --> Pulling branch : master  [ OK ] (080bd50)
projects/basic-readouts --> Pulling branch : master  [ OK ] (10f16e8)
scripts/batch-scripts --> Pulling branch : master  [ OK ] (c7e6c1c)
/home/ws/kx6822/workspace/rest-framework.git/doc/userguide[ Failed ]
source/libraries/axion/data --> Pulling branch : master  [ OK ] (22cf5a4)
source/libraries/axion/external/solarAxionFlux --> Pulling branch : master  [ OK ] (8f2d9bb)

Glad that it worked. This problem you are describing is because of a missing Geant4 header that is no longer present in the latest version. However there are preprocessor directives in place to handle this and it should not give an error with the restG4 master branch. Perhaps the logic in the CMake is failing, I will look it up.

[jgalan]

Just a quick comment, I do not know what are the side effects of executing --clean and --latest arguments together. One should use one or the other.

--clean will pull the submodules that are presently linked to the framework master branch.
--latest will pull the commit found at master for any submodule.

--clean will pull then the "official submodules release", thats it when we decide that the submodule latest changes should be included in a new framework release. Otherwise, a submodule might continue its development without affecting the official release.

[zykure]

Thanks @jgalan ! I tried again just using --latest and I can confirm that it works now. I can build successfully with Detector/Raw/Track/Geant4/G4 modules enabled. There's only a bunch of warnings, but no errors anymore.

Looks like I'm all set now, so I will close this issue. Thanks again both of you!