Telescope pointing from first event may be badly mistaken

[moralejo]

cta-lstchain/lstchain/high_level/hdu_table.py

Lines 270 to 272 in e4a5b7a

	pnt_icrs = SkyCoord(
	alt=pointing_alt[0],
	az=pointing_az[0],

@juanjq has seen some instance of the telescope pointing (RA,dec) for the first event in a DL2 file being quite off from the true pointing for (most of) the run. Example: run 17821. This is the only pointing stored in the DL3 file for the whole run, as RA_PNT, DEC_PNT

We think this has consequences for the ring background model: the model for such runs is badly mistaken, probably because symmetry is assumed around a wrong sky position.

[morcuended]

Are these pointing coordinates from the first event internally used in Gammapy for ring background calculation? Or is it whatever coordinates you define? What pointing information do you suggest including the HDU table?

[maxnoe]

@moralejo Do you understand why this happens? Is the timestamp wrong or the interpolation or the drive log?

Is this only for runs without a valid dragon reference where the camera config time is used?

[juanjq]

In principle it seems that gammapy assumes symmetry around pointing, that is written in the DL3 and it does not need to be input. And @maxnoe, I don't know 100%, but I don't think the issue is related to the dragon timestamps, we think this happens because sometimes when wobbling, LST can oscillate in the first instants of a run, resulting in a error of few 0.1 deg approx, then stabilizes. So then taking the first pointing as general is not that accurate. So I don't know which option can be better but the mean position can be taken, or maybe excluding start and end of the run, I guess there is not a unique solution for that.

[SeiyaNozaki]

I asked Armand before and it seems this is a real feature. You can also see this mispointing at the beginning of each tracking in the drive log. I had similar issues for source-dep analysis (fixed in #1211), so I agree with the suggestion by @juanjq (not take pointing info excluding the start of run)
https://www.lst1.iac.es/operations/drive/Log_cmd.2024-06-15.html

[morcuended]

Indeed when checking the center of off regions in Gammapy we can see this dispersion. Agree as well with @juanjq proposal of excluding the start/end and then taking the average

[maxnoe]

Ok, so the issue is not that the pointing is wrong for those events, it is that it is not yet stable on the wanted position and only wrong to use it as the one pointing position for the DL3 file.

Maybe we should identify the time range of this unstable pointing and exclude it via the GTI.

[maxnoe]

We (@juanjq and me) just had a look into the pointing of one of the runs, and it is very clear that it is not-yet-stable pointing from the drive in the first ~35 seconds of the run:

So for the fix, I would propose:

• Compute the mean pointing starting from something like 60s, maybe with an exceptional case for runs that are shorted.
• Identify the start of stable pointing by looking at e.g. by the RMSE from this mean pointing.
• Exclude the "unstable" part in the GTI of the DL3 files.

[moralejo]

Apologies for long silence on this. Completely excluding the unstable part is risky, we may e.g. miss some relevant GRB data - which can be properly reconstructed because we do have the event-wise reconstructed coordinates (and also event-wise pointing, although this is currently available in DL2, not in DL3).

Why is the pointing not included event-wise in DL3? Although in general we will track a RA-dec sky position, there will also be occasional "drift scans". And if one has full-enclosure IRFs it should be possible to analyze all data together despite the changing pointing. Can we simply add more entries in the pnt_table (see below) and then gammapy will do some interpolation for each event when needed?

cta-lstchain/lstchain/high_level/hdu_table.py

Line 417 in e4a5b7a

pnt_table = QTable(

Order-0 solution however is indeed writing the "correct value" and excluding periods which deviate significantly. As usual with this sort of thing, the hard part is to decide on reasonable cut values, deal with exceptions, etc.

[morcuended]

The pointing is included event-wise in the DL3 files, isn't it?

[moralejo]

The pointing is included event-wise in the DL3 files, isn't it?

Those are reconstructed RA, dec, right?

[morcuended]

Those are reconstructed RA, dec, right?

Yes

[moralejo]

Those are reconstructed RA, dec, right?

Yes

So, not the telescope pointing. I am not sure I am following, sorry...

[maxnoe]

GADF has the mechanism of storing the monitoring pointing table, however noting that it is preliminary and not implemented in science tools:
https://gamma-astro-data-formats.readthedocs.io/en/v0.3/events/pointing.html

And indeed, Gammapy does not support it:
gammapy/gammapy#4860

The simple solution for a high priority alert would be to cut the observations into multiple files that can have a constant ra/dec pointing and analyze them together.

[morcuended]

So, not the telescope pointing. I am not sure I am following, sorry...

My bad, it was me not following it. Disregard my comment.

[maxnoe]

Why is the pointing not included event-wise in DL3? Although in general we will track a RA-dec sky position, there will also be occasional "drift scans".

Just for completeness, this is supported by a fixed pointing in alt AZ with obsmode drift.

The only thing that is currently not supported is pointing not stable in either icrs or altaz.

[moralejo]

Why is the pointing not included event-wise in DL3? Although in general we will track a RA-dec sky position, there will also be occasional "drift scans".

Just for completeness, this is supported by a fixed pointing in alt AZ with obsmode drift.

Ah, ok, thanks! I guess we also have to update our DL2 to DL3 tool to allow setting obs_mode to drift upon request.

[morcuended]

Ah, ok, thanks! I guess we also have to update our DL2 to DL3 tool to allow setting obs_mode to drift upon request.

I've just created a new issue #1301 to remember this point

[moralejo]

@SeiyaNozaki I do not understand the fix you mentioned for source-dependent analysis.
With the current code, I think the problematic runs will have wrong expected_src_x, y, computed with the first event's pointing :

cta-lstchain/lstchain/high_level/hdu_table.py

Lines 324 to 325 in 620d0a9

	expected_src_x = data["expected_src_x"][0] * u.m
	expected_src_y = data["expected_src_y"][0] * u.m

I am not sure though what that is used for (since we have the src-dep parameters already computed at DL1/DL2 level with the event-wise pointing)

[moralejo]

Opened PR: #1302

[morcuended]

#1302 excluded the run's initial part and then took the mean pointing coordinates of the rest of the events.

For the long run, the suggestion was to define GTIs based on this pointing information that could be used later in Gammapy.

Can we close this?