feat: Allow volume constrain for propagation

[andiwand]

Add optional volume constrains to the propagation which are executed by the VolumeConstraintAborter. This is directly exercised in the track finding and wired to python.

This is generally useful if track finding / fitting should be done in a sub detector region. Same goes for simulation and extrapolation.

[andiwand]

I do not observe any performance changes with the ODD

[github-actions]

📊: Physics performance monitoring for bcb7b53

Full contents

physmon summary

• ✅ Particles fatras
• ✅ Particles geant4
• ✅ Particles ttbar
• ✅ Vertices ttbar
• ✅ Truth tracking (KF)
• ✅ Truth tracking (GSF)
• ✅ Truth tracking (GX2F)
• ✅ CKF | trackfinding | single muon | truth smeared seeding
• ✅ Track Summary CKF | trackfinding | single muon | truth smeared seeding
• ✅ Seeding trackfinding | single muon | truth estimated seeding
• ✅ CKF | trackfinding | single muon | truth estimated seeding
• ✅ Track Summary CKF | trackfinding | single muon | truth estimated seeding
• ✅ Seeding trackfinding | single muon | default seeding
• ✅ CKF | trackfinding | single muon | default seeding
• ✅ Track Summary CKF | trackfinding | single muon | default seeding
• ✅ Seeding trackfinding | single muon | orthogonal seeding
• ✅ CKF | trackfinding | single muon | orthogonal seeding
• ✅ Track Summary CKF | trackfinding | single muon | orthogonal seeding
• ✅ Seeding trackfinding | 4 muon x 50 vertices | default seeding
• ✅ CKF | trackfinding | 4 muon x 50 vertices | default seeding
• ✅ Track Summary CKF | trackfinding | 4 muon x 50 vertices | default seeding
• ✅ Ambisolver | trackfinding | 4 muon x 50 vertices | default seeding
• ✅ IVF notime | trackfinding | 4 muon x 50 vertices | default seeding
• ✅ AMVF gauss notime | trackfinding | 4 muon x 50 vertices | default seeding
• ✅ AMVF grid time | trackfinding | 4 muon x 50 vertices | default seeding
• ✅ Seeding trackfinding | ttbar with 200 pileup | default seeding
• ✅ CKF | trackfinding | ttbar with 200 pileup | default seeding
• ✅ Track Summary CKF | trackfinding | ttbar with 200 pileup | default seeding
• ✅ Ambisolver | trackfinding | ttbar with 200 pileup | default seeding
• ✅ AMVF gauss notime | trackfinding | ttbar with 200 pileup | default seeding
• ✅ AMVF grid time | trackfinding | ttbar with 200 pileup | default seeding

[timadye]

Sorry it took so long to get to this PR. I tested it with full_chain_itk.py. Unfortunately it failed right away:

RuntimeError: <class 'acts.ActsPythonBindings._examples.TrackFindingAlgorithm.Config'>: Failed to set pixelVolumeIds={8, 9, 10, 13, 14, 15, 16, 18, 19, 20}

I don't think we necessary need to setup constrainToVolumes for full_chain_itk.py (would be useful, but it's tricky), but it should be fixed to run without.

I noticed that full_chain_odd.py changed the Python type of pixelVolumes and stripVolumes from a set to a list. What's up with that?

[andiwand]

Sorry it took so long to get to this PR. I tested it with full_chain_itk.py. Unfortunately it failed right away:

RuntimeError: <class 'acts.ActsPythonBindings._examples.TrackFindingAlgorithm.Config'>: Failed to set pixelVolumeIds={8, 9, 10, 13, 14, 15, 16, 18, 19, 20}

I don't think we necessary need to setup constrainToVolumes for full_chain_itk.py (would be useful, but it's tricky), but it should be fixed to run without.

I will take a look

I noticed that full_chain_odd.py changed the Python type of pixelVolumes and stripVolumes from a set to a list. What's up with that?

I though I make this sound between this PR and the existing volume lists. A small vector is faster to search through and the interface should be more or less stable

[andiwand]

@timadye the ITk chain should work again now. The problem was just the change from set to list. Do you want to have another look?

Should we keep the list or revert back to a set? I think it makes sense to have them both use the same container type and a vector seems a bit more preferable for performance here. What do you think?

[timadye]

Should we keep the list or revert back to a set? I think it makes sense to have them both use the same container type and a vector seems a bit more preferable for performance here. What do you think?

Thinking about this, it would break any user scripts that happen to use pixelVolumes or stripVolumes. I expect hardly anyone uses those yet, so it should be OK.

In the end, I think these actually belong in the geometry setup, eg. itk.py so we will have to change again later.

[timadye]

This works. With 100 ttbar_pu200 events running full_chain_itk.py, #3470 TrackFindingAlgorithm is 1% slower than its base main (65aff94) and has a 0.7% drop in the fake rate. I don't see an appreciable change in the other tracking performance metrics.

Since we don't specify constrainToVolumes for the ITk, I don't know why the result isn't identical, but it's fine otherwise.

[andiwand]

This works. With 100 ttbar_pu200 events running full_chain_itk.py, #3470 TrackFindingAlgorithm is 1% slower than its base main (65aff94) and has a 0.7% drop in the fake rate. I don't see an appreciable change in the other tracking performance metrics.

Since we don't specify constrainToVolumes for the ITk, I don't know why the result isn't identical, but it's fine otherwise.

That is very odd. I just looked at the code and did not see a reason why this might be the case. I will merge main again maybe it got mixed up somehow

[timadye]

Since we don't specify constrainToVolumes for the ITk, I don't know why the result isn't identical, but it's fine otherwise.

That is very odd. I just looked at the code and did not see a reason why this might be the case. I will merge main again maybe it got mixed up somehow

I compared again the last update of #3470 against its last-merged main (0bf2465), using 8 threads as I did before. #3470 is now 1% faster and has a 0.6% increase in the fake rate, both the opposite to what I saw before. Unless this was a mistake on my part, it probably indicates a fluctuation.

I thought the two jobs should be running with the same events, since they have the same random number seed, so should have exactly identical results. But since I am running multi-threaded, there is scope for the events to be generated differently, and hence be subject to random fluctuations.

To test this, I ran again in single-threaded mode (20 events), and got exactly identical results #3470 vs main - even to the distributions being identical. #3470 was 1.6% slower, but I would have to run more longer tests to see if this is actually significant. I don't think it really matters at this level.

Sorry for delaying this PR with the spurious conundrum. It otherwise looks good to me.

Invalidated by push of 80406de

[paulgessinger]

@timadye the irreproducibility when running MT should be an Athena effect I guess? Otherwise, this would actually be a bit scary.

[timadye]

@timadye the irreproducibility when running MT should be an Athena effect I guess? Otherwise, this would actually be a bit scary.

I'm running the full chain, so could get events generated in a different order depending on the thread scheduling. Wouldn't that lead to a different random number sequence?

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
11.2% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[andiwand]

I'm running the full chain, so could get events generated in a different order depending on the thread scheduling. Wouldn't that lead to a different random number sequence?

Our RNG should give stable random numbers even tho it is scheduled differently. There is a seed for each event and this will be used consistently in MT or ST

The output order and the root binning might change though

[acts-project-service]

🔴 Athena integration test results [e7dceab]

🔴 Some tests have failed!

Please investigate the pipeline!

status	job	report
🟢	report_pull_request
🟢	run_art_test: test_data18_13TeV_1000evt
🔴	run_art_test: test_ttbarPU40_reco
🟢	run_workflow_tests_run4_mc
🟢	run_workflow_tests_run2_mc
🟢	run_workflow_tests_run2_data
🟢	run_workflow_tests_run3_mc
🟢	run_workflow_tests_run3_data
🟢	test_ActsDumpGeometryIdentifiers
🟢	test_ActsCheckObjectCountsCachedWorkflow
🟢	test_ActsCheckObjectCountsWorkflow
🟢	test_ActsEFTrackFit
🟢	test_ActsPersistifySeeds
🟢	test_ActsBenchmarkWithSpot
🟢	test_ActsAnalogueClustering
🟢	test_ActsWorkflowHeavyIons
🟢	test_ActsWorkflowFastTracking
🔴	test_ActsWorkflowCached
🔴	test_ActsWorkflow
🟢	test_ActsValidateAmbiguityResolution
🟢	test_ActsValidateResolvedTracks
🟢	test_ActsValidateTracks
🟢	test_ActsValidateActsCoreSpacePoints
🟢	test_ActsValidateActsSpacePoints
🟢	test_ActsValidateSeeds
🟢	test_ActsValidateOrthogonalSeeds
🟢	test_ActsValidateClusters
🟢	test_ActsPersistifyEDM
🟢	test_ActsGSFRefitting
🟢	test_ActsKfRefitting
🟢	test_ActsExtrapolationAlgTest
🟢	test_ActsITkTest
🟢	run_unit_tests

[paulgessinger]

Athena failures seem to be due to cvmfs issues