[Tune] Unpinned pydantic dependency

[yinweisu]

What happened + What you expected to happen

pydantic just did a major release, which causes ray tune to return AttributeError: module 'pydantic.fields' has no attribute 'ModelField'

Versions / Dependencies

2.3.1
I've looked at setup.py in the master branch of ray, this issue should be on the latest version too

Reproduction script

NA

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[yinweisu]

A more general comment: I see there are many dependencies that are not capped in ray. I would recommend pin them correctly to avoid things like this constantly happen. Customers who consume ray would have to do hot-patches too because of this

[pcmoritz]

Thanks for the bug report, see also #37000

What is the right way to handle these problems depends a lot on the dependency -- some do semantic versioning, some do not, sometimes we need lower pins, sometimes we need upper pins, sometimes we need both -- in general we try to be as lenient as possible since Ray is used with many many different packages and package versions (in fact that's the point of Ray -- to allow people distributing their Python applications).

We are trying to remove dependencies since that is just the best solution for everybody. For packages we cannot remove, we decide on a case-by-case basis what the best way forward is. I agree with your assessment that for pydantic we should put an upper pin on the major version (which will be 3 after we fixed the problems with 2).

See also

#35472
#36609
#36333

and related PRs.

[yinweisu]

Thanks! I understand your point totally and our project faces similar issues too in a sense that we have tons of dependencies. What we found useful is try to add lower and upper bounds before each release to ensure the environment is being controlled. I know there will be nuance for different cases. But in general, the goal is to avoid an upgrade of a dependency, which is outside what we can control, to break our own package. Glad to see that you guys are working toward that goal!

[woshiyyya]

Seems that it has been fixed by this PR #37000. I'll close this issue.