A Geant4 simulation of the ATLAS Tile Calorimeter beam tests.
Fig. - 10 GeV muon passing through the ATLAS TileCal.Table of Contents
The project targets a standalone Geant4 simulation of the ATLAS Tile Calorimeter beam tests to perform Geant4 regression testing, physics lists comparison and validation against test-beam data.
- ⏰ Start date: 23 May 2022
- 📌 Status: from v1.0 results are deployed on geant-val
- 👨🔬 Lorenzo Pezzotti (CERN EP-SFT) - lorenzo.pezzotti@cern.ch
- 👨🔬 Stephan Lachnit (CERN EP-SFT Summer Student) - stephan.lachnit@cern.ch
- 👨🔬 Supervisor: Alberto Ribon (CERN EP-SFT)
- 🗣️ CERN EP-SFT Group Meeting 22/4/2024, End of Fellowship Report
- 🗣️ Geant4 Collaboration Meeting 2023 25-29/9/2023, Validation of Geant4 physics via calorimeter test-beams in geant-val
- 🗣️ ATLAS Simulation Group Meeting 5/9/2023 (ATLAS restricted), Testing ATLAS calorimeters with Geant4 and FLUKA.CERN
- 🗣️ CERN EP-SFT Simulation Meeting 5/9/2023, Testing ATLAS calorimeters with Geant4 and FLUKA.CERN
- 🗣️ 11th BTTB Workshop 19/4/2023, Geant4 Validation on Test-Beam Calorimetry Data
- 🗣️ Tile TB at the TileWeek 8/2/2023 (ATLAS restricted), Geant4 validation studies using the tile calorimeter beam tests
- 🗣️ Geant4 Collaboration Meeting 2022 28/9/2022, Physics validation of Geant4 via calorimeter test-beams
- 🗣️ ATLAS Simulation Group Meeting 20/9/2022, Standalone Geant4 validation on the ATLAS Tile Calorimeter beam test
- 📄 CERN summer student report 25/8/2022, Standalone Geant4 validation of the ATLAS Tile Calorimeter
- 🗣️ CERN EP-SFT Simulation Meeting 23/8/2022, Standalone Geant4 validation on the ATLAS Tile Calorimeter beam test
- 🗣️ CERN EP-SFT Group Meeting 22/8/2022, Simulating ATLAS TileCal for Geant4 validation
We provide datasets and ROOT analyses, as well as instructions for their reproducibility. Ask authors for access to datasets. Results are deployed on Geant Val.
Geant-val-results-table
ATLTileCalTB | Reproduce data | Reproduce analysis | Comments |
---|---|---|---|
v1.3 Geant4 11.2.p01 tag 1.3_1 FTFP_BERT(+tunes1,2,3), FTFP_BERT_ATL, QGSP_BERT, FTFP_INCLXX 300k events per run Added 19/4/2024 |
Run with Geant Val, OR ./ATLTileCalTB -m TBrun_all.mac -p $PHYSLIST |
Run with Geant Val, OR root ../analysis/TBrun_all.C |
First results produced on new el9 lxplus machine. |
v1.1 Geant4 11.1.p01 tag 1.1_2 FTFP_INCLXX Added 2/5/2023 |
Run with Geant Val, OR ./ATLTileCalTB -m TBrun_all.mac -p $PHYSLIST |
Run with Geant Val, OR root ../analysis/TBrun_all.C |
FTFP_INCLXX results only because results 1.1_1 do not contain FTFP_INCLXX |
v1.1 Geant4 11.1 tag 1.1_1 FTFP_BERT(+tunes1,2,3), FTFP_BERT_ATL, QGSP_BERT (tag v1.1_1) 300k events per run Added 13/2/2023 |
Run with Geant Val, OR ./ATLTileCalTB -m TBrun_all.mac -p $PHYSLIST |
Run with Geant Val, OR root ../analysis/TBrun_all.C |
FTFP_INCLXX results not included due to a crash, to be investigated (problem with merged root files by parser.py) |
v1.0 Geant4 10.4.p03, 10.5.p01, 10.6.p03, 10.7.p03, 11.0.p02 FTFP_BERT, FTFP_BERT_ATL, QGSP_BERT, FTFP_INCLXX 300k events per run Added 17/8/2022 |
Run with Geant Val, OR ./ATLTileCalTB -m TBrun_all.mac -p $PHYSLIST |
Run with Geant Val, OR root ../analysis/TBrun_all.C |
Adjusted events per run to 300k |
Other-results-table
ATLTileCalTB | Reproduce data | Reproduce analysis | Comments |
---|---|---|---|
v1.2 Dataset #3 Geant4.10.6.3 (modified) tag 1.2_3 FTFP_BERT Created on 3/9/2023 |
./ATLTileCalTB -m TBrun_all.mac -p FTFP_BERT |
root TBrun_all.C |
Results to test the new FTF parameters by A. Ribon to fix the fluctuations disprepancy with respect to ATLAS data. Excitation energy per wounded nucleon set to 50 MeV. |
v1.2 Dataset #2 Geant4.10.6.3 (modified) tag 1.2_2 FTFP_BERT Created on 17/8/2023 |
./ATLTileCalTB -m TBrun_all.mac -p FTFP_BERT |
root TBrun_all.C |
Results to test the new FTF parameters by A. Ribon to fix the fluctuations disprepancy with respect to ATLAS data. |
v1.2 Dataset #1 Geant4 11.1.ref05 & fluka4-3.3 tag 1.2_1 FTFP_BERT and FTFP_BERT+fluka.cern-interface Created on 5/8/2023 |
./ATLTileCalTB -m TBrun_all.mac -p FTFP_BERT |
root FLUKA_comparison.C |
Data produced to compare results between Geant4 and FLUKA by using the Fluka.Cern interface. FTFP_BERT was used and compared to its Fluka.Cern customized version available in this repo. The interface was compiled as provided in the Geant4 11.1.ref05 examples. See how to use it. |
- git clone the repo
git clone https://github.com/lopezzot/ATLTileCalTB.git
- source Geant4 env
source /relative_path_to/geant4.10.07_p03-install/bin/geant4.sh
- cmake build directory and make (using geant4.10.07_p03)
See CMake options for all build options
mkdir ATLTileCalTB-build; cd ATLTileCalTB-build/ cmake -DGeant4_DIR=/absolute_path_to/geant4.10.07_p03-install/lib/Geant4-10.7.3/ relative_path_to/ATLTileCalTB/ make
- execute (example with TBrun.mac macro card, 2 threads and FTFP_BERT physics list)
./ATLTileCalTB -m TBrun.mac -t 2 -p FTFP_BERT
Parser options
-m macro.mac
: pass a Geant4 macro card (example-m ATLTileCalTB_run.mac
available in source directory and automatically copied in build directory)-t integer
: pass number of threads for multi-thread execution (example-t 2
, default is the number of threads on the machine)-p Physics_List
: select Geant4 physics list (example-p FTFP_BERT
)- It is possible to select alternative FTF tunings with PL_tuneID (example -p FTFP_BERT_tune0) [only for Geant4-11.1.0 or higher]
- git clone the repo
git clone https://github.com/lopezzot/ATLTileCalTB.git
- cmake build directory and make (using geant4.10.07_p03, check for gcc and cmake dependencies for other versions)
mkdir ATLTileCalTB-build; cd ATLTileCalTB-build/ cp ../ATLTileCalTB/scripts/ATLTileCalTB_cvmfs_setup.sh ../ATLTileCalTB/scripts/ATLTileCalTB_lxplus_10.7.p03.sh . source ./ATLTileCalTB_lxplus_10.7.p03.sh -DBUILD_ANALYSIS=OFF
- execute (example with TBrun.mac macro card, 4 threads and FTFP_BERT physics list)
./ATLTileCalTB -m TBrun.mac -t 4 -p FTFP_BERT
- First follow the build instructions on lxplus
- prepare for HTCondor submission (example with Geant4.10.07_p03, TBrun.mac, 4 threads, FTFP_BERT physics list)
mkdir -p error log output cp ../ATLTileCalTB/scripts/ATLTileCalTB_HTCondor.sub ../ATLTileCalTB/scripts/ATLTileCalTB_HTCondor_10.7.p03.sh . sed -i "2 i cd $(pwd)" ATLTileCalTB_HTCondor_10.7.p03.sh echo ./ATLTileCalTB -m TBrun.mac -t 4 -p FTFP_BERT >> ATLTileCalTB_HTCondor_10.7.p03.sh sed -i "1 i executable = ATLTileCalTB_HTCondor_10.7.p03.sh" ATLTileCalTB_HTCondor.sub
- submit a job
condor_submit ATLTileCalTB_HTCondor.sub
- monitor the job
or (for persistency)
condor_q
condor_wait -status log/*.log
- additional info from HTCondor (optional)
rm all your jobsinspect your accounting groupcondor_rm username
display all accounting groupscondor_q owner $LOGNAME -long | grep '^AccountingGroup' | sort | uniq -c
display your accounting groupshaggis group list
check what accounting group a job hashaggis rights
specify the accounting group for yout job, to be added in the .sub filecondor_q jobid.0 -af AccountingGroup
check job CPU usage+AccountingGroup = "group_u_*"
ssh to machine where job is runningcondor_q -l jobid.0 | grep CPUsUsage
condor_ssh_to_job jobid.0
Geant4-11.1-ref05
introduces a Fluka.Cern interface to use the Fluka.Cern hadron inelastic process in any geant4 application as explained in examples/extended/hadronic/FlukaCern
. The following are my instructions to use this repo with a customized FTFP_BERT physics list using it. It assumes that cvmfs is mounted (e.g. usage on lxplus).
- Install Fluka.Cern from source code (example with fluka4-3.3)
source /cvmfs/sft.cern.ch/lcg/contrib/gcc/10.1.0/x86_64-centos7/setup.sh cd fluka4-3.3 && make -j 4 cd src/ && make cpp_headers mkdir /path-to/fluka4-3.3-install && make install DESTDIR=/path-to/fluka4-3.3-install/ PATH="/absolute-path-to/fluka4-3.3-install/bin/":$PATH
- Setup
geant4-11.1.ref05
and compile the fluka interface as in the examplesource /cvmfs/geant4.cern.ch/geant4/11.1.ref05/x86_64-centos7-gcc10-optdeb-MT/CMake-setup.sh source /cvmfs/geant4.cern.ch/geant4/11.1.ref05/x86_64-centos7-gcc10-optdeb-MT/bin/geant4.sh cd FlukaCern/FlukaInterface/ make interface make env source env_FLUKA_G4_interface.sh
- Build and execute ATLTileCalTB
NOTE: the Fluka.Cern interface can only be used in single-threaded mode.
git clone https://github.com/lopezzot/ATLTileCalTB.git mkdir ATLTileCalTB-build && cd ATLTileCalTB-build /cvmfs/sft.cern.ch/lcg/contrib/CMake/3.23.2/Linux-x86_64/bin/cmake -DG4_USE_FLUKA=1 ../ATLTileCalTB/ make
Geant Val is the Geant4 testing and validation suite. It is a project hosted on gitlab.cern.ch used to facilitate the maintenance and validation of Geant4 applications, referred to as tests.
The following are instructions to use ATLTileCalTB within Geant Val, from batch submission to website deployment.
- On lxplus, clone ATLTileCalTB and the Geant Val geant-config-generator
git clone https://github.com/lopezzot/ATLTileCalTB git clone https://gitlab.cern.ch/GeantValidation/geant-config-generator.git
- Follow the build instructions on lxplus
- Copy the ATLTileCalTB geant val scripts into
tests/geant4/
cp -r ATLTileCalTB/geantval_scripts/ATLTileCalTB/ geant-config-generator/tests/geant4/ mkdir -p geant-config-generator/tests/geant4/ATLTileCalTB/files cp ATLTileCalTB/TileTB_2B1EB_nobeamline.gdml ATLTileCalTB/analysis/TBrun_all.C geant-config-generator/tests/geant4/ATLTileCalTB/files/
- We will execute ATLTileCalTB via Geant Val using Geant4.10.7.p03, therefore we must make sure the file
10.7.p03.sh
exists inconfigs/geant/
. In the file10.7.p03.sh
we also export the path to the ATLTileCalTB executable (compiled with 10.7.p03).
Copy the config file using:./ATLTileCalTB/geantval_scripts/cpconf.sh \ ATLTileCalTB/geantval_scripts/configs/10.7.p03.sh \ geant-config-generator/configs/geant4/ \ $(pwd)/ATLTileCalTB-build
- Create macros and metadata for Geant Val execution
this command creates the Geant Val files for batch submission using HTCondor under the
cd geant-config-generator ./mc-config-generator.py submit -t ATLTileCalTB -d OUTPUT -v 10.7.p03 -q "testmatch" -r
OUTPUT
folder, using ATLTileCalTB, Geant4.10.7.p03 and thetestmatch
job flavour. - To monitor the jobs use
When the job execution ends, the root output files are stored in the corresponding job folder. Each job folder will look like this:
./mc-config-generator.py status -t ATLTileCalTB -d OUTPUT
ATLTileCalTB-env.log test_stderr.txt test_stdout.txt ATLTileCalTB.json ATLTileCalTB.mac ATLTileCalTBout_Run0.root ATLTileCalTB.sh bsub.sh config.sh
- Execute the analysis on the root files in the
OUTPUT
folder to create Geant Val JSON output filesMake sure to source ROOT on CVMFS before, for example by sourcing the provided lxplus scripts../mc-config-generator.py parse -t ATLTileCalTB -d OUTPUT
Besides the JSON files in theOUTPUTJSON
folder, the parser script will also create some temporary folder that contain the merged ROOT files for each run and the analysis output. - The last part is to deploy the results on Geant Val. The ATLTileCalTB layout on the Geant Val
website is defined in the ATLTileCalTB.xml file on gitlab.com/thegriglat/geant-val-layouts
(additional info are in the tags.json file).
Deploy JSON files on the Geant Val database
find . -name '*.json' | while read i; \ do curl -H "Content-Type: application/json" -H "token: askauthor" --data @$i https://geant-val.cern.ch/upload; \ echo; done
Custom options:
BUILD_ANALYSIS
: if set toON
(default), it will be an executable of the analysis, which is slightly faster. The analysis can also be run directly with theroot
executable (see Run the analysis), which is recommended if the compilation fails.WITH_ATLTileCalTB_PulseOutput
: if set toON
, the simulation will output the pulse response of the PMTs. These can be viewed by running./pulse_viewer.py
in the build directory. Since this slows the simulation considerably, it is recommended to leave this option disabled except for debugging purposes.WITH_ATLTileCalTB_NoNoise
: if set toON
, the simulation will not put electronic noise on the signal (per cell) and disable the 2 sigma noise cut. Only relevant for noise calibration.WITH_GEANT4_UIVIS
: if set toON
(default), build with UI and visualization drivers.G4_USE_FLUKA
: if set toON
build against the Fluka.Cern interface (defaultOFF
).WITH_LEAKAGEANALYSIS
: if set toON
build with leakage spectrum analyzer (defaultOFF
).
Relevant built-in options:
CMAKE_BUILD_TYPE
: set toDebug
for debugging and toRelease
for production (faster).CMAKE_EXPORT_COMPILE_COMMANDS
: ifON
, createscompile_commands.json
in the build folder. Might help some syntax highlighters to properly parse the analysis script if building withBUILD_ANALYSIS=ON
.Geant4_DIR
: set to the path containing theGeant4Config.cmake
file of your Geant4 installation.ROOT_DIR
: set to the path containing thROOTConfig.cmake
of your ROOT installation. Only used if building withBUILD_ANALYSIS=ON
.
- For the analysis both electron and pion data is required, which can be obtained by running for example
ATLTileCalTB -m TBrun_all.mac
. - After the run, all the ROOT files and their containing TTree need to merged into a single file called
ATLTileCalTBout_All.root
. This can be done withhadd
:hadd -f ATLTileCalTBout_RunAll.root ATLTileCalTBout_Run*.root
- To run the analysis, execute the analysis macro in the folder containing the root file:
Alternatively, the analysis marco can also be build as executable for slightly faster executation time.
root /path/to/ATLTileCalTB/analysis/TBrun_all.C
- The plots created during the analysis are stored in the
analysis.root
file.