The present document collects information about the ALICE detector simulation executable and digitization procedure used in LHC Run3.
Detector simulation, the simulation of detector response from virtual particle events, consists of essentialy 2 parts: a) the generation of simple (energy deposit) traces in the detector due to the passage of particles and the interaction with the detector material. b) the conversion of those traces into (electronic) signals in the detector readout (usually called digitization).
The first part is handled by the o2-sim executable (See SimSection). The second part is handled in the o2-sim-digitizer-workflow executable (See DigitSection). References to examples are collected here.
The Run3 simulation offers the following features
- distributed system based on FairMQ that is splitting event generation, particle transport and IO into separate asyncronous components that can be deployed on different machines
- sub-event parallelism making it possible to transport a single big event in a short time and to reduce memory consumption
- parallelism independent on transport engine
- configuration via pre-defined parameter classes and ini/text files
- clear separation of transport and digitization - each phase can be run fully independently
The purpose of the o2-sim executable is to simulate the passage of particles emerging from a collision inside the detector and to obtain their effect in terms of energy deposits (called hits) which could be converted into detectable signals. It is the driver executable which will spawn a topology of sub-processes that interact via messages in a distributed system.
-
Quick start example: A typical (exemplary) invocation is of the form
o2-sim -n 10 -g pythia8pp -e TGeant4 -j 2 --skipModules ZDC,PHSwhich would launch a simulation for 10 pythia8 events on the whole ALICE detector but ZDC and PHOS, using Geant4 on 2 parallel worker processes.
-
Generated output: The simulation creates the following output files:
| File | Description |
|---|---|
o2sim_Kine.root |
contains kinematics information (primaries and secondaries) and event meta information |
o2sim_geometry.root |
contains the final ROOT geometry created for simulation run |
o2sim_grp.root |
special global run parameters (grp) such as field |
o2sim_XXXHits.root |
hit file for each participating active detector XXX |
o2sim_configuration.ini |
summary of parameter values with which the simulation was done |
o2sim_serverlog |
log file produced from the particle generator server |
o2sim_workerlog |
log file produced form the transportation processes |
o2sim_hitmergerlog |
log file produced from the IO process |
- Main command line options: The following major options are available (incomplete):
| Option | Description |
|---|---|
| -h,--help | Prints the list of possible command line options and their default values. |
| -n,--number | The number of events to simulate. |
| -g,--generator | name of a predefined generator template to use (such as pythia8pp, pythia8hi). Configuration of generations is explained in a dedicated section. |
| -e,--engine | Select the VMC transport engine (TGeant4, TGeant3). |
| -m,--modules | List of modules/geometries to include (default is ALL); example -m PIPE ITS TPC |
| -j,--nworkers | Number of parallel simulation engine workers (default is half the number of hyperthread CPU cores) |
| --chunkSize | Size of a sub-event. This determines how many primary tracks will be sent to a simulation worker to process. |
| --skipModules | List of modules to skip / not to include (precedence over -m) |
| --configFile | A .ini file containing a list of (non-default) parameters to configure the simulation run. See section on configurable parameters for more details. |
| --configKeyValues | Like --configFile but allowing to set parameters on the command line as a string sequence. Example --configKeyValues "Stack.pruneKine=false". Takes precedence over --configFile. Parameters need to be known ConfigurableParams. |
| --seed | The initial seed to (all) random number instances. Default is -1 which leads to random behaviour. |
| -o,--outPrefix | How output files should be prefixed. Default is o2sim. Example -o mySignalProduction. |
| --noGeant | Switch off Geant transport. Just produce the generator kinematics. |
- Expert control via environment variables:
o2-simis sensitive to the following environment variables:
| Variable | Description |
|---|---|
| ALICE_O2SIM_DUMPLOG | When set, the output of all FairMQ components will be shown on the screen and can be piped into a user logfile. |
| ALICE_NOSIMSHM | When set, communication between simulation processes will not happen using a shared memory mechanism but using ROOT serialization. |
Simulation makes use of configurable parameters as described in the ConfigurableParam.md documentation.
Detector code as well as general simulation code declare such parameter and access them during runtime.
Once a parameter is declared, it can be influenced/set from the outside via configuration files or from the command line. See the --configFile as well as --configKeyValues command line options.
The complete list of parameters and their default values can be inspected in the file o2sim_configuration.ini that is produced by an empty run o2-sim -n 0 -m CAVE.
Important parameters influencing the transport simulation are:
| Main parameter key | Description |
|---|---|
| G4 | Parameters influencing the Geant4 engine, such as the physics list. Example "G4.physicsmode=kFTFP_BERT_optical_biasing" |
| Stack | Parameters influencing the particle stack. Example include whether the stack does kinematics pruning or whether it keeps secondaries at all. |
| SimCutParams | Parameters allowing to set some sime geometry stepping cuts in R, Z, etc. |
| Diamond | Parameter allowing to set the interaction vertex location and the spread/width. Is used in all event generators. |
| Pythia8 | Parameters that influence the pythia8 generator. |
| HepMC | Parameters that influence the HepMC generator. |
| TriggerParticle | Parameters influencing the trigger mechanism in particle generators. |
Detectors may also have parameters influencing various pieces such geometry layout, material composition etc.
Below some notes on example generators along with some usage info.
- Fwmugen
fwmugen is a lightweight and simple “box” generator for forward muons (1 muon / event)
o2-sim -m MFT -e TGeant3 -g fwmugen -n 10
- BoxGen
o2-sim -m PIPE ITS MFT -e TGeant3 -g boxgen -n 10 --configKeyValues 'BoxGun.pdg=13 ; BoxGun.eta[0]=-3.6 ; BoxGun.eta[1]=-2.45; BoxGun.number=100'
This command line will generate 10 events with 100 forward muons.
- PYTHIA 8
Configures pythia8 for min.bias pp collisions at 14 TeV
o2-sim -m PIPE ITS MFT -g pythia8pp -n 50
[Describe in detail the environment variables]
[Add something on data layout of hits file]
You may contribute to the documentation by asking a question
In order to access event generators from ALIROOT, such as THijing or TPyhtia6, you may use the -g external command line option followed by a ROOT macro setting up the event
generator. Examples thereof are available in the installation directory $O2_ROOT/share/Generators/external.
Users may write there own macros in order to customize to their needs.
Use the --modules or -m command line option. Example: o2-sim -m PIPE ITS TPC
will run the simulation on a geometry/material consinsting of PIPE, ITS, TPC.
One may perform any arbitrary simulation with AliRoot and reuse the kinematics information in form of Kinemtatics.root
produced. The file contains primary and possibly secondary particles (added by transportation).
When the file is passed to o2sim, the primary particles my be used as the initial event.
Use the -g extkin command line option:
o2-sim -g extkin --extKinFile Kinematics.root ...
4. How can I generate events (signal) using the vertex position of already-generated (background) events?
This process might be called embedding, where one wants to merge two events generated independenly. For that to be physically correct, both events have to originate from the same interaction vertex.
Assuming that your already-generated (background) events are stored in the o2sim.background.root file, you can force the interaction vertex for the generation of a new set of events to be the same as the one in the background with the following command line option:
o2-sim --embedIntoFile o2sim.background.root
Background events are sampled one-by-one until all events have been used. At that point the events start to be reused.
Run the simulation (currently only supported in combination with o2-sim-serial) with a preloaded library:
MCSTEPLOG_TTREE=1 LD_PRELOAD=$MCSTEPLOGGER_ROOT/lib/libMCStepLoggerInterceptSteps.so o2-sim-serial -j 1 -n 10
This will produce a file MCStepLoggerOutput.root containing detailed information about steps and processes (where, what, ...). The file can be analysed using a special analysis framework. See https://github.com/AliceO2Group/VMCStepLogger/blob/master/README.md for more documentation.
All event generator interfaces that comply with the o2::eventgen::Generator protocol can be triggered.
A basic 'particle trigger' is implemented in the o2::eventgen core and allows the user to define a trigger particle.
The definitions of the trigger particle can be expressed via command line arguments
o2-sim -g pythia8pp -t particle --configKeyValues "TriggerParticle.pdg=333;TriggerParticle.ptMin=5.;TriggerParticle.yMin=-0.5;TriggerParticle.yMax=0.5"
Custom triggers can also be constructed by the user to provide unlimited flexibility in the trigger needs. An external trigger function can be specified via command line arguments
o2-sim -g pythia8pp -t external --configKeyValues 'TriggerExternal.fileName=path_to_trigger_macro.C;TriggerExternal.funcName="the_function(some, parameters)"'
The function must comply with a simple protocol and return a lambda function defined as follows
o2::eventgen::Trigger the_function()
{
return [](const std::vector<TParticle>& particles) -> bool {
return true; // triggered
}
}
Within the lambda function the user receives the stack of generated particles and can inspect it to define a trigger at will.
The trigger is fired when the lambda function returns true and the simulation of the current event is subsequently started.
To allow users to define triggers that go beyond the particle stack generated by the event generator, another functionality is added. This allows the user to go deep into the core of the event generator, whenever this is possible. For this reason, this is called a 'DeepTrigger'. A 'DeepTrigger' is attached to the simulation in the same way as a normal trigger
o2-sim -g pythia8pp -t external --configKeyValues 'TriggerExternal.fileName=path_to_deep_trigger_macro.C;TriggerExternal.funcName="the_deep_function(some, parameters)"'
In this case the function must comply with a similar, but different protocol than before and return a lambda function defined as follows
{
return [mpiMin](void* interface, std::string name) -> bool {
return true;
};
}
Notice that in this case the user is presented with a pointer to the event-generator interface and a string that defines its name.
For the sake of generality, a void* has to be used in order to pass any possible types of event-generators, that are
normally othogonal one to another. The name encodes a string to identify what generator has been passed and perform the correct cast to use it.
Some medium parameter definitions are defined in $O2/Detectors/<detector>/simulation/data/simcuts.txt (or in $O2/Detectors/Passive/data/simcuts_<modulename>.txt for passive modules), others might be hard-coded in the source code. To extract/change/study any of those parameters on the fly, it is possible to write out all of them into a JSON file with
o2-sim <args> --configKeyValues "MaterialManagerParam.outputFile=medium_params.json"Parameters of interest can be changed in medium_params.json and it can be passed now as an input for the target simulation with
o2-sim <args> --configKeyValues "MaterialManagerParam.inputFile=medium_params.json"Note that process parameters have no effect when Geant4 is used for transport.
The MCReplay engine can be used to replay a simulation based on steps logged by the MCStepLogger (see also a more in-depth documentation).
To run it with O2, first follow the steps as explained in MCStepLoggerSection to produce a file containing logged steps. To replay, do
o2-sim-serial -n <ref_nevents> -e MCReplay -g extkinO2 --extKinFile o2sim_Kine.root -o replayIt is advisory to use another output prefix as done in this case since otherwise the hit files would be overwritten which might contain exactly the information one is interested in. Make sure to use/exclude the same modules as used in the reference run (-m and --skipModules flags). In case the reference run was done with another prefix, the kinematics file name is different, namely <prefix>_Kine.root.
If the name of the step log file is different, it can be passed with --configKeyValues="MCReplayParam.stepFilename=<path/step/file/name>". It is also possible to set a minimum energy (in units of GeV) cut particles have to have when produced. For that, use --configKeyValues="MCReplayParam.energyCut=0.1" if everything produced below 0.1 GeV should be dropped.
Comparing the produced hits with those from the reference run it is possible to omit steps/particle production which have a negligible impact on the hits and hence on digits. As a result, the detector simulation can be tuned to be faster and more efficient.
Deep triggers is just a name to a new functionality that allows the user to define custom functions that will have a direct handle on the event generator interface. The functionality follows the schema of the previous point, with the user providing a custom lambda function that will receive from the framework a pointer to the internal event-generator interface object (i.e. for Pythia8, a pointer to the Pythia object) and a tagname to identify the interface. This functionality might be useful to users who want to provide triggers based on information beyond the stack of the generated particles, based on more internal counters/information in the event generator machinery.
Here is an example of a deep trigger implementation in Pythia8.
// usage: o2sim --trigger external --configKeyValues 'TriggerExternal.fileName=trigger_mpi.C;TriggerExternal.funcName="trigger_mpi()"'
#include "Generators/Trigger.h"
#include "Pythia8/Pythia.h"
o2::eventgen::DeepTrigger
trigger_mpi(int mpiMin = 5)
{
return [mpiMin](void* interface, std::string name) -> bool {
if (!name.compare("pythia8")) {
auto py8 = reinterpret_cast<Pythia8::Pythia*>(interface);
return py8->info.nMPI() >= mpiMin;
}
LOG(fatal) << "Cannot define MPI for generator interface \'" << name << "\'";
return false;
};
}
Pythia8 machinery allows the user to hook some code at various stages of the event-generation process. For details, please look at Pyhia8 manual. http://home.thep.lu.se/~torbjorn/pythia82html/UserHooks.html
The interface is provided via a configuration macro, where the user will have to define a custom UserHooks according to the protocol defined by Pythia8. The macro will also have to provide a function to retrieve the pointer to the created custom UserHooks object.
This functionality might be of use for users who want to be able to steer the event-generation process from very deep inside the internal routines and want to veto some specific processes based on analysis of the status of Pythia8 at the various stages, i.e. veto events that do not have charm partons, before hadronisation of partons. This can save time in the event generation process as many steps can be skipped already at early time.
An example of a configuration macro is this one
// usage: o2sim -g pythia8pp --configKeyValues "GeneratorPythia8.hooksFileName=pythia8_userhooks_charm.C"
#include "Generators/Trigger.h"
#include "Pythia8/Pythia.h"
class UserHooksCharm : public Pythia8::UserHooks
{
public:
UserHooksCharm() = default;
~UserHooksCharm() = default;
bool canVetoPartonLevel() override { return true; };
bool doVetoPartonLevel(const Pythia8::Event& event) override
{
for (int ipa = 0; ipa < event.size(); ++ipa) {
if (abs(event[ipa].id()) != 4)
continue;
if (fabs(event[ipa].y()) > 1.5)
continue;
return false;
}
return true;
};
};
Pythia8::UserHooks*
pythia8_userhooks_charm()
{
return new UserHooksCharm();
Digitization - the transformation of hits produced in the transport simulation to electronics detector output - is steered by the o2-sim-digitizer-workflow executable. The executable is implemented as a [DPL workflow] (https://github.com/AliceO2Group/AliceO2/blob/dev/Framework/Core/README.md). The main components in this workflow are:
- A SimReader process, responsible to analyze available simulation information/kinematics and to setup the digitization context, which describes things such as the structure of the timeframe (bunch cross properties and interaction rate) as well as how to combine different background and signal hits.
- Digitizer processors per detector, responsible for the actual digitization upon receiving the digitization context from the SimReader.
- IO processors per detector, responsible to write digits to files.
- GRP updater, a process to update the GRP file with information aquired in digitization.
-
Quick start example: A minimal invocation is of the form
o2-sim-digitizer-workflow [--sims foo] -bwhich would launch the digitization phase for all detectors that took part in a simulation stored under simulation prefix
foo(default o2sim) and will digitize all events with a default bunch crossing structure. All digitizer will run in parallel to each other. -
A more advanced example:
o2-sim-digitizer-workflow --sims bkg,sgn --interactionRate 1e6 --onlyDet TPC,ITS -bwhich would launch the digitization phase for TPC and ITS with a custom LHC interactionRate. Moreover, this example does summation of digits coming from background (prefix bkg) as well as signal (prefix sgn) transport simulations.
-
Generated output: The digitization process creates the following output files:
| File | Description |
|---|---|
collisioncontext.root |
Contains information about the collision/digitization context used in this digitization. Keeps the list of input files and how collisions were composed for the digits embedding process and time stamps where assigned. |
XXXdigits.root |
Typically one digit file per detector XXX in a timeframe format. The file also typically contains mappings of digit indices to MC labels. |
o2simdigitizerworkflow_configuration.ini |
Summary of parameters used in the digitization process. |
- Main command line options: The following major options are available:
| Option | Description |
|---|---|
| -h,--help | Prints the list of possible command line options and their default values. |
| --sims | Comma separated list of simulation prefixes that should be overlaid/embedded. Example --sims background,signal where background and signal refer to transport simulation productions. Final collisions will be composed from both of them (in a round robin fashion). See separate section about Embedding for more details. If just one prefix is given, normal digitization without overlay will be done. |
| --tpc-lanes | Number of parallel digitizers for TPC, which has a special attention due an increased data rate compared to other detectors. |
| --interactionRate | Total hadronic interaction rate (Hz). |
| --bcPatternFile | Interacting BC pattern file chaning the default bunch crossing pattern, see macro/CreateBCPattern.C for details. |
| --onlyDet | Comma separated list of detectors to digitize. (Default is all) |
| --skipDet | Comma separed list of detectors to exclude. |
| --incontext | Name of context file. Useful for reusing a context from a previous run when we split the processing detectorwise. |
| --outcontext | Specify name of contextfile to produce. |
| --simFileQED | Optional special QED hit file to include effect from QED effects into digitization. |
The files produced by the o2-sim contain only a set of separate events, w/o any time stamp. The digitization will sample the vector of non-decreasing {BC/orbit} pairs according to bunch filling schema and requested interaction rate and provide it to digitizers.
The interaction sampling is steered by the HBFUtils configurable parameters, e.g.
o2-sim-digitizer-workflow --configKeyValues "HBFUtils.nHBFPerTF=128;HBFUtils.orbitFirst=123;HBFUtils.orbitFirstSampled=300"
The most important here is HBFUtils.orbitFirstSampled which tells to InteractionSampler the orbit from which the sampling should start.
Other parameters, nHBFPerTF and orbitFirst do not directly affect the digitization but they are stored in the grp object (o2sim_grp.root file) and the number of orbits per TF and the 1st orbit of the run.
The full content of the HBFUtils is stored in the o2simdigitizerworkflow_configuration.ini and can be used in reconstruction and MC->raw-data creation processes.
Particularly, when creating the raw data (see ...-digi2raw group of commands in O2/prodtests/full_system_test.sh) the content of the HBFUtils settings will be loaded from this ini file
(the name can be changed via --hbfutils-config <ini-file> option) and HBFUtils.orbitFirst will be used to define the start-of-the-run (SOX flag in the RDH) and HBFUtils.nHBFPerTF will be used to chop the digitized data to TFs.
Note that for the detectors in continuous readout mode all empty HBFs will be created for all orbits w/o detector data between the SOX and the last orbit of the TF for which data was received.
In case you want to create raw data starting from the 1st sampled TF, you should override the HBFUtils.orbitFirst by the wanted orbit, i.e. for the digitization done as in the example above
one can request
o2-ft0-digi2raw --configKeyValues "HBFUtils.orbitFirst=251" ...
to start the raw data not from the orbit 123 indicated during digitization but from the orbit 251 (i.e. the 2nd TF of the run). Additional setting HBFUtils.maxNOrbits can be provided
to limit the number of orbits stored (counted from the SOX) in the raw data (i.e. all data with orbit > HBFUtils.orbitFirst + HBFUtils.maxNOrbits will be ignored when creating the raw data.
Similarly, when running the reconstruction workflow using the output of the digitization, the HBFUtils settings used by o2-sim-digitizer-workflow will be loaded (again, they can be overridden
by either providing --hbfutils-config <ini-file> option or by explicit --configKeyValues "HBFUtils... setting) and the DPL will make sure that the content of the DataHeader.firstTForbit correspond
to the 1st orbit of the TF containing HBFUtils.orbitFirstSampled (then incremented by HBFUtils.nHBFPerTF for the following TFs if the digits files contain multiple TF entries).
We can associate digits to tracks/particles of the original transport simulation, so as to keep provenance information of how digits were triggered. This information can be passed forward to reconstruction and analysis and used to study reconstruction efficiencies etc.
To this end, a special data object MCCompLabel is offered, which allows to encapsulate the identifiers of track, event and source kinematics files.
MCCompLabel(int trackID, int evID, int srcID, bool fake = false)This information should be enough to lookup and load the precise Monte Carlo track (see here).
Association of digits to an arbitrary number of labels is done via filling a separate and dedicated container called MCTruthContainer which is written as a separate branch to the output file, next to the branch for digits. This has the advantage that digits may be kept as close as possible to the raw data and we can have arbitrary number of labels at a minimal memory cost.
The mechanics is as follows: For a collection of digits created for detector foo
std::vector<o2::foo::Digits> mDigits;we keep a separate container of labels of type:
o2::dataformats::MCTruthContainer<o2::dataformats::MCCompLabel> mLabelContainer;Querying the labels works by positional correspondance: Labels for the digit at position pos can be accessed in the following way:
const auto& digit = mDigits[pos];
// returns an iterable view of labels
const auto& labels_for_digit = mLabelContainer.getLabels(pos);
// iterate over labels
for (auto& label : labels_for_digit) {
// process label
}If positional correspondance is too weak, one may eventualy choose to record the corresponding data index in the labelcontainer inside the digit itself:
const auto& digit = mDigits[pos];
// returns an iterable view of labels
const auto& labels_for_digit = mLabelContainer.getLabels(digit.labelindex);
// iterate over labels
for (auto& label : labels_for_digit) {
// process label
}After digitization is done, one can use the MCKinematicsReader class to load and access the Monte Carlo tracks.
The MCKinematicsReader needs the digitization context file, generated during digitization. Once initialized it can return the tracks associated to a Monte Carlo label.
A typical code example may be
// init the reader from the context
o2::steer::MCKinematicsReader reader("collisioncontext.root");
// load digits from the digits file --> save in alldigits
// load the label container from the digits file --> save in labelcontainer
// this is simply iterating over all the digits and querying the tracks that contributed to these digits
for (int pos = 0; pos < alldigits.size(); ++pos) {
const auto& digit = alldigits[pos];
const auto& labels_for_digit = labelcontainer.getLabels(pos);
// iterate over labels
for (auto& label : labels_for_digit) {
track = reader.getTrack(label);
// do something with the track
}
}Note, that one can also access kinematics directly after the transport simulation. In this case, one needs to initialize the MCKinematicsReader in a different mode:
// init the reader from the transport kinematics file (assuming here prefix o2sim)
o2::steer::MCKinematicsReader reader("o2sim", o2::steer::MCKinematicsReader::Mode::kMCKine);
// loop over all events in the file
for (int event = 0; event < reader.getNEvents(0); ++event) {
// get all Monte Carlo tracks for this event
std::vector<MCTrack> const& tracks = reader.getTracks(event);
// analyse tracks
}Some examples for the usage of simulation and digitization are collected in an examples folder. Other helpful resources are the scripts used for regression testing in prodtests.
| Example | Short Description |
|---|---|
| HF_Embedding_Pythia8 | Example showing how to setup a complex HF simulation for embedding |
| AliRoot_Hijing | Example showing how to use Hijing from AliRoot for event generation |
| AliRoot_AMPT | Example showing how to use AMPT from AliRoot for event generation |
| Adaptive_Pythia8 | Complex example showing generator configuration for embedding that cat adapt the response based on the background event |
| Signal_ImpactB | Example showing generator configuration for embedding that cat adapt to the impact parameter of the background event |
| PrimaryKinematics | Example showing how to obtain only primary kinematics via transport configuration |
| HepMC_STARlight | Simple example showing generator configuration that runs a standalone STARlight generation that couples to the o2 via a HepMC file |
| Jet_Embedding_Pythia | Complex example showing generator configuration, digitization embedding, MCTrack access |
| Selective_Transport | Simple example showing how to run simulation transporting only a custumisable set of particles |
| Selective_Transport_pi0 | Complex example showing how to run simulation within the inhibit-pause-trigger-continue approach |
| Custom_EventInfo | Simple example showing how to add custom information to the MC event header |
| sim_challenge.sh | Basic example doing a simple transport, digitization, reconstruction pipeline on the full dectector. All stages use parallelism. |
| sim_performance.sh | Basic example for serial transport and linearized digitization sequence (one detector after the other). Serves as standard performance candle. |