BUG: pd.Timestamp loses fold when removing timezone which is inconsistent with datetime.datetime

[AlexeyDmitriev]

• [+] I have checked that this issue has not already been reported.

• [+] I have confirmed this bug exists on the latest version of pandas.
  I have tested on {'dirty': False, 'error': None, 'full-revisionid': '67a3d4241ab84419856b84fc3ebc9abcbe66c6b3', 'version': '1.1.4'}

• [-] (optional) I have confirmed this bug exists on the master branch of pandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Problem description

datetime.datetime saves the fold when replacing tzinfo, but pandas.Timestamp doesn't

Code Sample, a copy-pastable example

import dateutil.tz
import pandas
from datetime import datetime as dt
print(pandas.Timestamp(1256427000000000000, tz=dateutil.tz.gettz('Europe/Moscow'), unit='ns'))
# Timestamp('2009-10-25 02:30:00+0300', tz='dateutil//usr/share/zoneinfo/Europe/Moscow')
print(pandas.Timestamp(1256427000000000000, tz=dateutil.tz.gettz('Europe/Moscow'), unit='ns').fold)
# 1
print(pandas.Timestamp(1256427000000000000, tz=dateutil.tz.gettz('Europe/Moscow'), unit='ns').replace(tzinfo=None).fold)
# 0
print(dt(2009, 10, 25, 2,30, fold=1, tzinfo=dateutil.tz.gettz('Europe/Moscow')).replace(tzinfo=None).fold)
# 1

Expected Output

Timestamp('2009-10-25 02:30:00+0300', tz='dateutil//usr/share/zoneinfo/Europe/Moscow')
1
1
1

Output of pd.show_versions()

commit : 67a3d42
python : 3.8.4.final.0
python-bits : 64
OS : Darwin
OS-release : 19.5.0
Version : Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.1.4
numpy : 1.16.4
pytz : 2020.4
dateutil : 2.8.0
pip : 20.1.1
setuptools : 39.0.1
Cython : 3.0a6
pytest : 5.0.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.8.3 (dt dec pq3 ext lo64)
jinja2 : 2.10.1
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.0.3
numexpr : None
odfpy : None
openpyxl : 3.0.5
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : 0.8.6
xarray : None
xlrd : 1.2.0
xlwt : None
numba : None

[AlexKirko]

Hello, Alexey, thanks for noticing this!

The example doesn't necessarily have to work, because we don't quite support fold for anything except dateutil timezones, so losing fold after destroying timezone information isn't that horrible.

However, this also doesn't work and is definitely a bug:

import dateutil.tz
import pandas

print(pandas.Timestamp(1256427000000000000, tz=dateutil.tz.gettz('Europe/Moscow'), unit='ns').replace(tzinfo=dateutil.tz.gettz('Europe/Moscow')).fold)
OUT:
0

The cause is clear: it's because in pandas/_libs/tslibs/timestamp.pyx we don't pull fold from self in replace. Instead we transform self into a datetime structure that loses the fold information and then set fold to whatever the user supplied.

This should be easily fixed by setting the default to None and then replacing with whatever was in self, if the user didn't supply anything.

Taking a look. The fix I'm trying preserves fold even when the timezone info is destroyed, fold just doesn't do anything in that case.

[AlexeyDmitriev]

I'd say I don't really have a good use case.
I just noticed that my tests started to fail when I changed from datetime.datetime to pd.Timestamp
I think, it's not desired for a class with a description "Timestamp is the pandas equivalent of python’s Datetime and is interchangeable with it in most cases" unless there's a good reason.

Also, from replace I expect that all the fields that are not replaced stay the same as they were.

Anyway, yours is certainly a bug, and I think if you fix it, the None version will be also fixed

Thanks