Can't use astype() to assign a series of dtypes to a frame.

[mvashishtha]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS Monterey
• Modin version (modin.__version__): latest source
• Python version: 3.9.10
• Code we can use to reproduce:

import modin.pandas as pd
df = pd.DataFrame([[0]])
df.astype(df.dtypes)

Describe the problem

If I try to pass a series of dtypes to astype, I get ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all(). However, I can do that in pandas.

Source code / logs

Show stack trace

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [10], in <module>
      1 import modin.pandas as pd
      3 df = pd.DataFrame([[0]])
----> 4 df.astype(df.dtypes)

File ~/modin/modin/pandas/base.py:921, in BasePandasDataset.astype(self, dtype, copy, errors)
    918     for column in self._query_compiler.columns:
    919         col_dtypes[column] = dtype
--> 921 new_query_compiler = self._query_compiler.astype(col_dtypes)
    922 return self._create_or_update_from_compiler(new_query_compiler, not copy)

File ~/modin/modin/core/storage_formats/pandas/query_compiler.py:1508, in PandasQueryCompiler.astype(self, col_dtypes, **kwargs)
   1507 def astype(self, col_dtypes, **kwargs):
-> 1508     return self.__constructor__(self._modin_frame.astype(col_dtypes))

File ~/modin/modin/core/dataframe/pandas/dataframe/dataframe.py:114, in lazy_metadata_decorator.<locals>.decorator.<locals>.run_f_on_minimally_updated_metadata(self, *args, **kwargs)
    112     elif apply_axis == "rows":
    113         obj._propagate_index_objs(axis=0)
--> 114 result = f(self, *args, **kwargs)
    115 if apply_axis is None and not transpose:
    116     result._deferred_index = self._deferred_index

File ~/modin/modin/core/dataframe/pandas/dataframe/dataframe.py:1003, in PandasDataframe.astype(self, col_dtypes)
   1000 except TypeError:
   1001     new_dtype = dtype
-> 1003 if dtype != np.int32 and new_dtype == np.int32:
   1004     new_dtypes[column] = np.dtype("int64")
   1005 elif dtype != np.float32 and new_dtype == np.float32:

File /usr/local/lib/python3.9/site-packages/pandas/core/generic.py:1535, in NDFrame.__nonzero__(self)
   1533 @final
   1534 def __nonzero__(self):
-> 1535     raise ValueError(
   1536         f"The truth value of a {type(self).__name__} is ambiguous. "
   1537         "Use a.empty, a.bool(), a.item(), a.any() or a.all()."
   1538     )

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

[mvashishtha]

Now that I think about it, this feature may be a bug in pandas caused by attempting to convert a dict of types to a Series here. I asked that in pandas-dev/pandas#46353. I'll make the fix #4318 a draft until I find out.

[mvashishtha]

I got confirmation from a pandas maintainer in pandas-dev/pandas#46353 that this is a feature, not a bug, in pandas. However, I asked for clarification as to the behavior when the series has a non-unique index. For now, I think #4318 can proceed.