Don't force conversion if dtype is object

[hoxbro]

Fixes #7234

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.23%. Comparing base (0453e9c) to head (eabef78).
Report is 4 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7242   +/-   ##
=======================================
  Coverage   82.23%   82.23%           
=======================================
  Files         337      337           
  Lines       50251    50258    +7     
=======================================
+ Hits        41325    41331    +6     
- Misses       8926     8927    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.