chore: Change arrow scalar ids usage

[camenares]

[PENDING] 2. Run unit tests and ensure that they are passing: https://github.com/feast-dev/feast/blob/master/CONTRIBUTING.md#unit-tests

What this PR does / why we need it:

Changes where _ARROW_SCALAR_IDS_TO_BQ is imported, so that google-cloud-bigquery can be bumped. This PR only bumps to 3.13.0, but unblocks further bumping. _ARROW_SCALAR_IDS_TO_BQ was moved here.

Which issue(s) this PR fixes:

Related to #3823, but provides a different solution than downgrading.

[tokoko]

This will break all versions below 3.14, won't it? Maybe we should just copy the mapping over instead... Not ideal, but might be better than to depend on an internal helper function. @sudohainguyen wdyt?

[sudohainguyen]

good fix!
@camenares could you help re-generate the requirements txt files?
to make sure all other packages are compatible 😄

[sudohainguyen]

please also check that the change is backward compatible for those who using version less that 3.12

[camenares]

@tokoko that's a good point, I think you're right, it would not work 3.12 (but will double check).

I think there's three options:

1. Change setup.py to also strict pin to 3.13.0
2. Do a try/catch and try to import from each helper
3. Copy the mapping over.

I will work on regenerating the requirements.txt's

[tokoko]

@camenares option 2 also sounds good to me. We can resort to copying the mapping if some other issue arises in the future. Any reason for the upper bound set at <3.14. Why can't we change to <4? Are there some other known issues?

P.S. You need to sign all your commits (git commit -s). Follow the instructions here to fix the DCO check.

[tokoko]

Can you run make lint-python and make format-python commands?

[camenares]

@tokoko yep! Fixing that now, I believe it's related to python/mypy#1153 but testing this locally

[camenares]

@tokoko I also noticed that allowing the bound to be <4 failed the full integration tests (local were fine). Restricting to <3.14.0 seems to let those pass though.

[tokoko]

@camenares integration tests failure was unrelated, sorry about that. We accidentally merged a failing PR to master.

[camenares]

@tokoko Oh! Thanks for that, I just figured it was worth trying <3.14. Let me bump to <4.0 again then.

[tokoko]

You will have to regenerate requirements files for the bump to take effect in ci.

[camenares]

@tokoko <4.0 worked! I see no additional changes after running make lock-python-dependencies-all.

[tokoko]

right, looks like firebase-admin lib is pinning with an upper bound, I'll look into that later. lgtm otherwise, thanks a lot

[camenares]

Thanks a lot for the guidance @tokoko , looking forward to more Feast contributions in the future!

EDIT: I see integration-tests-and-build / integration-test-python (3.10, ubuntu-latest) failed on the commit, but it looks like that (or the test for another python version) regularly fails on some commits.

[tokoko]

we have some strict rate limits on cloud services that our tests are often hitting, you can ignore it.