Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add processing of Interleaved events #358

Merged
merged 55 commits into from
May 2, 2020
Merged
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
ff66286
Add config for calibration calculator
FrancaCassol Apr 15, 2020
48f353b
Add new Component with performs camera calibration calculations
FrancaCassol Apr 15, 2020
5e4fe97
Change the name of the calibration tool: it works also for MC data
FrancaCassol Apr 15, 2020
ac174a9
Changed in order to use the new calibration calculator component
FrancaCassol Apr 15, 2020
c13ee32
Rename module because it works also for MC data
FrancaCassol Apr 15, 2020
99cf884
Minor changes
FrancaCassol Apr 15, 2020
e1a6ac9
Initialize all montoring containers (changes for using interleaved ca…
FrancaCassol Apr 15, 2020
d3b050e
Add function to write calibration events containers in DL1
FrancaCassol Apr 15, 2020
fe19206
Add the processing and writing of the interleaved calibration data
FrancaCassol Apr 15, 2020
dd4306c
Add ff time corrction at calibrator level
FrancaCassol Apr 20, 2020
ccdc50f
Add drs4 charge time correction
FrancaCassol Apr 20, 2020
e9cc86b
Add drs4 charge time corrections
FrancaCassol Apr 20, 2020
dbcba0c
Remove time flat-field correction at drs4 level
FrancaCassol Apr 20, 2020
fff004f
Add the possiblity to force calibration output
FrancaCassol Apr 20, 2020
b34bc22
Force interleaved calibraiton output at the end of file
FrancaCassol Apr 20, 2020
60403d0
First produce the time calibration file, which is now necessary for t…
FrancaCassol Apr 20, 2020
8748879
Container initalization changed
FrancaCassol Apr 20, 2020
6a2388a
Minor changes
FrancaCassol Apr 20, 2020
4771628
Ask high statistic for pedestals and flatfields, so only one calibrat…
FrancaCassol Apr 20, 2020
eb449b5
Eliminate the "fits" postfix in the h5 file output name
FrancaCassol Apr 20, 2020
814ee9d
Improve prints
FrancaCassol Apr 20, 2020
4bf3fdb
Changed name to script
FrancaCassol Apr 20, 2020
09a8a45
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 20, 2020
7461bff
Changed logic to assure that last interleaved statistics is written o…
FrancaCassol Apr 21, 2020
8fa4daa
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 21, 2020
4890cab
Minor changes
FrancaCassol Apr 21, 2020
b4218d7
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 21, 2020
cc92303
Correct index update and merge problem
FrancaCassol Apr 21, 2020
c633f05
Add trailet for excess noise factor. Improve function name
FrancaCassol Apr 21, 2020
e001a23
Require time calibation file as input
FrancaCassol Apr 21, 2020
3544032
Require time camibration file as input
FrancaCassol Apr 21, 2020
19fa103
Change function name. Correct trailet initialization
FrancaCassol Apr 21, 2020
10a3341
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 22, 2020
c2e6267
Add LSTEventType class to recognize the event type
FrancaCassol Apr 22, 2020
0abd613
Add "static" property to static class
FrancaCassol Apr 22, 2020
c5c5e3c
Add static property to static function
FrancaCassol Apr 22, 2020
39b9269
Replace kargs with config and parent in the CalibrationCalculator Com…
FrancaCassol Apr 22, 2020
4de59e4
Use the R1CameraContainer.selected_gain_chanel field
FrancaCassol Apr 22, 2020
412b3e2
Correct program flow in case of MC data
FrancaCassol Apr 24, 2020
1a05e04
Updated calibration json parameter files
FrancaCassol Apr 24, 2020
287f9a4
Allow time_calibration_path == "None" for simulation events
FrancaCassol Apr 24, 2020
ffc249c
Use null in jason file instead of "None"
FrancaCassol Apr 24, 2020
1e7d057
Correct config definition position
FrancaCassol Apr 28, 2020
e51c773
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 28, 2020
87bcec0
Delete unused script
FrancaCassol Apr 28, 2020
018163e
Correct import
FrancaCassol Apr 28, 2020
9887df3
Monir corrections
FrancaCassol Apr 29, 2020
f21f6c6
Update imports
FrancaCassol Apr 29, 2020
a6e8410
Merge branch 'master' into interleaved_events
maxnoe Apr 29, 2020
8d016d0
Skip the first 1000 events that badly drs4 corrected
FrancaCassol Apr 30, 2020
47d96df
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 30, 2020
0d4348b
Merge branch 'interleaved_events' of https://github.com/FrancaCassol/…
FrancaCassol Apr 30, 2020
a345aff
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol Apr 30, 2020
491fdb7
Set None as default for time calibration file
FrancaCassol May 2, 2020
82995c6
Merge branch 'master' of https://github.com/cta-observatory/cta-lstch…
FrancaCassol May 2, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Replace kargs with config and parent in the CalibrationCalculator Com…
…ponent

Replace kargs with config and parent in the CalibrationCalculator Component
  • Loading branch information
FrancaCassol committed Apr 22, 2020
commit 39b9269d71d96dc2f42dd7c4c2ee07547fe85670
34 changes: 6 additions & 28 deletions lstchain/calib/camera/calibration_calculator.py
Original file line number Diff line number Diff line change
@@ -61,8 +61,9 @@ class CalibrationCalculator(Component):

def __init__(
self,
parent=None,
config=None,
**kwargs

):

"""
@@ -80,19 +81,17 @@ def __init__(
The pedestal to use. If None, then
PedestalCalculator will be used by default.

kwargs

"""

super().__init__(**kwargs)
super().__init__(parent=parent, config=config,**kwargs)

self.flatfield = FlatFieldCalculator.from_name(
self.flatfield_product,
**kwargs
parent=self
)
self.pedestal = PedestalCalculator.from_name(
self.pedestal_product,
**kwargs
parent=self
)

msg = "tel_id not the same for all calibration components"
@@ -133,23 +132,6 @@ class LSTCalibrationCalculator(CalibrationCalculator):
help='Temporary cut on LG std against Lidar events till the calibox TIB do not work (default for filter 5.2) '
).tag(config=True)

def __init__(self, **kwargs):
"""
Calibration calculator for LST camera
Fills the MonitoringCameraContainer on the base of calibration events

Parameters:
----------
minimum_hg_charge_median :
Temporary cut on HG charge till the calibox TIB do not work
(default for filter 5.2)

maximum_lg_charge_std
Temporary cut on LG std against Lidar events till the calibox TIB do not work
(default for filter 5.2)

"""
super().__init__(**kwargs)

def calculate_calibration_coefficients(self, event):
"""
@@ -198,9 +180,7 @@ def calculate_calibration_coefficients(self, event):
camera_time_median = np.median(ff_data.time_median, axis=1)
calib_data.time_correction = -ff_data.relative_time_median - camera_time_median[:, np.newaxis]

ped_extractor_name = self.config.get("PedestalCalculator").get("charge_product")
ped_width = self.config.get(ped_extractor_name).get("window_width")
calib_data.pedestal_per_sample = ped_data.charge_median / ped_width
calib_data.pedestal_per_sample = ped_data.charge_median / self.pedestal.extractor.window_width

# put to zero unusable pixels
calib_data.dc_to_pe[calib_data.unusable_pixels] = 0
@@ -209,8 +189,6 @@ def calculate_calibration_coefficients(self, event):
# eliminate inf values id any (still necessary?)
calib_data.dc_to_pe[np.isinf(calib_data.dc_to_pe)] = 0



def process_interleaved(self, event):
"""
Process interleaved calibration events (pedestals and FF)
4 changes: 2 additions & 2 deletions lstchain/data/lstchain_standard_config.json
Original file line number Diff line number Diff line change
@@ -130,15 +130,15 @@
"maximum_lg_charge_std": 300,
"flatfield_product": "FlasherFlatFieldCalculator",
"pedestal_product": "PedestalIntegrator",
"PedestalCalculator":{
"PedestalIntegrator":{
"sample_size": 10000,
"sample_duration":100000,
"tel_id":1,
"charge_median_cut_outliers": [-10,10],
"charge_std_cut_outliers": [-10,10],
"charge_product":"FixedWindowSum"
},
"FlatFieldCalculator":{
"FlasherFlatFieldCalculator":{
"sample_size": 10000,
"sample_duration":100000,
"tel_id":1,
4 changes: 2 additions & 2 deletions lstchain/data/onsite_camera_calibration_param.json
Original file line number Diff line number Diff line change
@@ -10,14 +10,14 @@
"r0calibrator_product": "LSTR0Corrections",
"log_level":"DEBUG"
},
"PedestalCalculator":{
"PedestalIntegrator":{
"sample_duration":100000,
"tel_id":1,
"charge_median_cut_outliers": [-10,10],
"charge_std_cut_outliers": [-10,10],
"charge_product":"FixedWindowSum"
},
"FlatFieldCalculator":{
"FlasherFlatFieldCalculator":{
"sample_duration":100000,
"tel_id":1,
"charge_product":"LocalPeakWindowSum",
8 changes: 5 additions & 3 deletions lstchain/reco/r0_to_dl1.py
Original file line number Diff line number Diff line change
@@ -260,9 +260,11 @@ def r0_to_dl1(


# Component to process interleaved pedestal and flat-fields
calib_config = Config(config[config['calibration_product']])
# time calibration path to flatfield trailet
calib_config.FlatFieldCalculator.time_calibration_path = time_calibration_path
calib_config = Config(config)

# set time calibration path for flatfield trailet ()
calib_config.FlasherFlatFieldCalculator.time_calibration_path = time_calibration_path

calibration_calculator = CalibrationCalculator.from_name(
config['calibration_product'],
config=calib_config
4 changes: 2 additions & 2 deletions lstchain/scripts/onsite/onsite_create_calibration_file.py
Original file line number Diff line number Diff line change
@@ -136,8 +136,8 @@ def main():
# run lstchain script
cmd = f"lstchain_create_calibration_file " \
f"--input_file={input_file} --output_file={output_file} --pedestal_file={pedestal_file} " \
f"--FlatFieldCalculator.time_calibration_path={time_file} --FlatFieldCalculator.sample_size={stat_events} "\
f"--PedestalCalculator.sample_size={stat_events} " \
f"--FlasherFlatFieldCalculator.time_calibration_path={time_file} --FlasherFlatFieldCalculator.sample_size={stat_events} "\
f"--PedestalIntegrator.sample_size={stat_events} " \
f"--EventSource.max_events={max_events} --config={config_file} > {log_file} 2>&1"

print("\n--> RUNNING...")
9 changes: 4 additions & 5 deletions lstchain/tools/lstchain_create_calibration_file.py
Original file line number Diff line number Diff line change
@@ -75,7 +75,6 @@ class CalibrationHDF5Writer(Tool):
]
+ traits.classes_with_traits(CameraR0Calibrator)
+ traits.classes_with_traits(CalibrationCalculator)

)

def __init__(self, **kwargs):
@@ -97,9 +96,9 @@ def __init__(self, **kwargs):
self.simulation = False

def setup(self):
kwargs = dict(parent=self)

self.log.debug(f"Open file")
self.eventsource = EventSource.from_config(**kwargs)
self.eventsource = EventSource.from_config(parent=self)

# if data remember how many event in the files
if "LSTEventSource" in str(type(self.eventsource)):
@@ -112,13 +111,13 @@ def setup(self):

self.processor = CalibrationCalculator.from_name(
self.calibration_product,
**kwargs
parent=self
)

if self.r0calibrator_product:
self.r0calibrator = CameraR0Calibrator.from_name(
self.r0calibrator_product,
**kwargs
parent=self
)