Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attrs dependency requirement conflicts with Airflow #37

Closed
kemics opened this issue Sep 7, 2021 · 6 comments · Fixed by #38 or #39
Closed

Attrs dependency requirement conflicts with Airflow #37

kemics opened this issue Sep 7, 2021 · 6 comments · Fixed by #38 or #39
Labels
enhancement New feature or request

Comments

@kemics
Copy link

kemics commented Sep 7, 2021

Hi, thank you for the helpful library

I am trying to use toloka-kit in Airflow environment. However, when I am installing both apache-airflow and toloka-kit packages it fails with dependency conflict error. Something like this:

apache-airflow 2.1.2+composer has requirement attrs<21.0,>=20.0, but you have attrs 21.2.0.

I tried different recent versions of Airflow (both v1 and v2 and master branch) and all are failing with a similar error

And it looks like that toloka-kit relies on API of new 21 version of attrs. Because I ran toloka-kit tests in virtual env with attrs==20.3.0 and 26 tests failed.

I understand that it would be unpleasant to downgrade attrs package version requirement here. However, Airflow is a very popular solution and I assume that I am not the only one who manages or will manage Toloka projects via Airflow. Moreover, I doubt that Airflow will update the dependency version soon. Currently this is a blocker for me and maybe to somebody else to migrate from bare API calls to toloka-kit. For example, I am not able to create a hook, PythonOperator or PythonSensor for crowdsourcing purposes via toloka-kit

What do you think?

@Losik
Copy link
Contributor

Losik commented Sep 9, 2021

The reason for attrs>=21.2.0 is this feature python-attrs/attrs#710 that we rely on for correct signatures iand correct stub generation.This feature should have appeared in 20.4.0 but the only unyanked release after 20.3.0 is 21.2.0.

I guess we can downgrade to attrs>=20.3.0. If we use later versions of attrs in our environment during stub-generation process. The only thing this would affect is inspect.signature(toloka_client.<method_name>) which would produce correct but less detailed signatures (having Any on positions that would otherwise be more specific). IDE hints will stay detailed, though.

long story short, we'll downgrade to attrs>=20.3.0 in the next release

@Losik Losik added the enhancement New feature or request label Sep 9, 2021
@Losik Losik linked a pull request Sep 10, 2021 that will close this issue
@Losik Losik linked a pull request Sep 10, 2021 that will close this issue
@kemics
Copy link
Author

kemics commented Sep 13, 2021

@Losik thank you again for solving this so fast. Can you please push tag (looks like 0.1.15)?

@Losik
Copy link
Contributor

Losik commented Sep 13, 2021

Yes, I must do one more commit with changelog updates and version increase, merge it with main branch and then create a new release. This commit have to sync with GitHub from our internal repo and this action is triggered once an hour. Unfortunately, this functionality is maintained by another team and I do not have a way to trigger this synchronization manually at the moment =(

Long story short, I expect new PyPI release to be ready in about an hour

@Losik Losik linked a pull request Sep 13, 2021 that will close this issue
@Losik Losik closed this as completed in #39 Sep 13, 2021
@Losik
Copy link
Contributor

Losik commented Sep 13, 2021

@kemics, toloka-kit==0.1.15 is in PyPI :)

@kemics
Copy link
Author

kemics commented Sep 13, 2021

Thanks a lot, this change really helps to use this package in my production 🎉

@Pocoder
Copy link
Contributor

Pocoder commented Jan 25, 2022

@kemics, if you continue to use toloka-kit in airflow, you can be early adopter and try out our new library for airflow. We will be very grateful for your feedback =)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
3 participants