Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Python 3.10 support #19059

Closed
1 of 2 tasks
mik-laj opened this issue Oct 19, 2021 · 10 comments · Fixed by #22050
Closed
1 of 2 tasks

Add Python 3.10 support #19059

mik-laj opened this issue Oct 19, 2021 · 10 comments · Fixed by #22050
Assignees
@potiuk
Labels
kind:feature Feature Requests

Comments

@mik-laj
Copy link
Member

mik-laj commented Oct 19, 2021

Description

According to README.md - "Support for Python and Kubernetes versions":

We support a new version of Python/Kubernetes in main after they are officially released, as soon as we make them work in our CI pipeline (which might not be immediate due to dependencies catching up with new versions of Python mostly) we release new images/support in Airflow based on the working CI setup.

Python 3.10 has been released on October 18, 2021, so we should take steps to test Airflow with this release.

Use case/motivation

Support for new versions of Python is expected by users who wish to test the new features that this version added. After that, some users install this release by mistake and then have trouble using Airflow e.g. https://stackoverflow.com/questions/69615585/unable-to-start-airflow-webserver-with-python-3-10

Related issues

#15515

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@mik-laj mik-laj added the kind:feature Feature Requests label Oct 19, 2021
@uranusjr
Copy link
Member

The ModuleNotFoundError: No module named 'pwd' error the user gets is weird though. Airflow should have no problem running on 3.10 (albeit not officially supported), and the pwd stdlib did not receive any significant change. That problem is likely separated from 3.10 support.

I'll submit a fix to deal with the distutils deprecation warning.

@mik-laj mik-laj mentioned this issue Oct 19, 2021
Merged
@uranusjr uranusjr mentioned this issue Oct 19, 2021
Merged
@auvipy
Copy link
Contributor

auvipy commented Dec 22, 2021

did python 3.10.1 release solved anything?

@potiuk potiuk self-assigned this Dec 22, 2021
@potiuk
Copy link
Member

potiuk commented Dec 22, 2021

Python 3.10 support is missing for other reasons - unrelated to version support. it was missing because some dependencies did not supported it. But this might be a good thing to try again maybe we can get it workin. I am planning to do some further cleanups with images after Xms and possibly attemp to create even an ARM image (and there python 3.10 might actually be better than older versions).

@potiuk
Copy link
Member

potiuk commented Jan 23, 2022

Just try to build Python 3.10 - we are rather close - I think all our deps are almost ready - the last one is numpy limited by `apache-beam'. This is also blocker for ARM images. However it's been alrady merged in 'beam' repo:

apache/beam@d845a00#diff-1275c48808de339ef6f282d844c83ec441b5cfa0debc373fdcb7dba497da4fc8

And the next release should build both for Python 3.10 and allow us to build ARM images.

@potiuk
Copy link
Member

potiuk commented Jan 24, 2022

FYI. Apache Bem release that should be our "last" serious blocker is planned for first week of February. I could make some exclusions and make it, independently of Beam, but since it is only weeks away, I think it is better to wait.

This was referenced Jan 24, 2022
Closed
Closed
@AetherUnbound AetherUnbound mentioned this issue Feb 23, 2022
Merged
7 tasks
@hozn
Copy link

hozn commented Mar 3, 2022

It looks like new Apache Beam is released (assuming that is https://beam.apache.org/blog/beam-2.36.0/). Was that the last dependency holdout for this?

@potiuk
Copy link
Member

potiuk commented Mar 6, 2022

Yep. One more small things with #22017 and it shoud be possible.

@potiuk potiuk mentioned this issue Mar 7, 2022
Merged
@potiuk
Copy link
Member

potiuk commented Mar 7, 2022

All right. Now the only thing I need to do is to release all providers with version 3.10 support :).

potiuk added a commit to potiuk/airflow that referenced this issue Mar 20, 2022
@potiuk
Add Python 3.10 support
03bcfea 
Python 3.10 support has been long missing because a number of our
dependencies had problems with it. It seems that last problems
remaining should be fixed now, and we should be able to get
a proper Python 3.10 support.

Closes: apache#19059
potiuk added a commit to potiuk/airflow that referenced this issue Mar 21, 2022
@potiuk
Add Python 3.10 support
9c140ce 
Python 3.10 support has been long missing because a number of our
dependencies had problems with it. It seems that last problems
remaining should be fixed now, and we should be able to get
a proper Python 3.10 support.

Closes: apache#19059
potiuk added a commit that referenced this issue Mar 21, 2022
@potiuk
Add Python 3.10 support (#22050)
4c28c2a 
Python 3.10 support has been long missing because a number of our
dependencies had problems with it. It seems that last problems
remaining should be fixed now, and we should be able to get
a proper Python 3.10 support.

Closes: #19059
@Jorricks
Copy link
Contributor

@potiuk should we update the README.md?

Note: Python v3.10 is not supported yet. For details, see #19059.

@potiuk
Copy link
Member

potiuk commented Mar 28, 2022

It's already updated https://github.com/apache/airflow/blob/main/README.md . Only main version supports Python 3.10. What other update would you need? If it is missing somewhere in other readmes you see - feel free to open PR :)

kaxil added a commit to astronomer/airflow that referenced this issue Mar 29, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0
d805e84 
Based on the feedback in apache#19059 (comment) -- this PR makes it explicit
@kaxil kaxil mentioned this issue Mar 29, 2022
Merged
kaxil added a commit that referenced this issue Mar 29, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 (#…
c58b5e5 
…22602)

Based on the feedback in #19059 (comment) -- this PR makes it explicit
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Jul 10, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
77a832b 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
@EMCP EMCP mentioned this issue Jul 30, 2022
Closed
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Aug 30, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
2139b66 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Oct 4, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
7497eee 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
aglipska pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Oct 8, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
6e65f1d 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Dec 7, 2022
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
cc0a76f 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
leahecole pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Jan 27, 2023
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
b0110ce 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
kosteev pushed a commit to kosteev/composer-airflow-test-copybara that referenced this issue Sep 12, 2024
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
d1284f4 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Sep 18, 2024
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
87a98dd 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this issue Nov 7, 2024
@kaxil
Doc: Explicitly specify Py 3.10 will be supported from Airflow 2.3.0 …
99618ef 
…(#22602)

Based on the feedback in apache/airflow#19059 (comment) -- this PR makes it explicit

GitOrigin-RevId: c58b5e583bcad83048fdee72bc9ddf6c3a7b4229
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@potiuk potiuk

Labels
kind:feature Feature Requests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Python 3.10 support potiuk/airflow
6 participants
@hozn @potiuk @uranusjr @auvipy @mik-laj @Jorricks