Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Apache Airflow "latest" image is still Python 3.8 in Airflow 2.8.0 #36740

Closed
1 task done
potiuk opened this issue Jan 11, 2024 · 3 comments · Fixed by #36779
Closed
1 task done

Apache Airflow "latest" image is still Python 3.8 in Airflow 2.8.0 #36740

potiuk opened this issue Jan 11, 2024 · 3 comments · Fixed by #36779
Assignees
@potiuk
Labels
kind:meta High-level information important to the community

Comments

@potiuk
Copy link
Member

potiuk commented Jan 11, 2024

Body

Despite what is documented in Changelog and what we agreed in the lazy consensus , the default docker imagest (latest/latest-slim) are still pointing to Python 3.8.

The changelog should be corrected and that move should happen in Airflow 2.9

Committer

  • I acknowledge that I am a maintainer/committer of the Apache Airflow project.
@potiuk potiuk added the kind:meta High-level information important to the community label Jan 11, 2024
@potiuk potiuk self-assigned this Jan 11, 2024
@potiuk
Copy link
Member Author

potiuk commented Jan 11, 2024

Looks strange.. It looks like the code is there and fixed .... puzzled a bit... But I will find out.

@hussein-awala
Copy link
Member

Good catch! but yeah, it's very strange. Based in the CI log, it should be ok:

Replacing latest images with links to the newly created images.
Creating apache/airflow:latest-python3.11 alias for apache/airflow:2.8.0-python3.11
Running command: <regctl image copy apache/airflow:2.8.0-python3.11 apache/airflow:latest-python3.11>
  
  Working directory /home/runner/actions-runner/_work/airflow/airflow
  
  
  regctl image copy --force-recursive --digest-tags apache/airflow:2.8.0-python3.11 apache/airflow:latest-python3.11
  
Creating apache/airflow:latest alias for apache/airflow:2.8.0
Running command: <regctl image copy apache/airflow:2.8.0 apache/airflow:latest>
  
  Working directory /home/runner/actions-runner/_work/airflow/airflow
  
  
  regctl image copy --force-recursive --digest-tags apache/airflow:2.8.0 apache/airflow:latest

I checked the sha256 of the images we released one month ago to ensure that it is not a new issue, but it looks like we had the same issue with the RC images.

@potiuk
Copy link
Member Author

potiuk commented Jan 14, 2024

Ah... I know .. my mistake. If you look here - it only changese at which python version the "2.8.0" alias points to latest one. There is no python used in this if, so the only thing it change is when latest becomes aliased to2.8.0.

            if python == ALLOWED_PYTHON_MAJOR_MINOR_VERSIONS[-1]:
                # only tag latest "default" image when we build the latest allowed python version
                if slim_images:
                    alias_image(
                        f"{dockerhub_repo}:slim-{airflow_version}",
                        f"{dockerhub_repo}:slim-latest",
                    )
                else:
                    alias_image(
                        f"{dockerhub_repo}:{airflow_version}",
                        f"{dockerhub_repo}:latest",
                    )

But what should be changed is the line before that:

            if python == DEFAULT_PYTHON_MAJOR_MINOR_VERSION:
                alias_image(image_name, f"{dockerhub_repo}:{airflow_version}")

(and similar one in slim image) - > because THERE the "airflow:2.8.0-python3.8" gets the airflow:2.8.0 alias - which what it was about.

I will announce it at devlist that we are delaying it and remove the log in dockerfile changelog.

potiuk added a commit to potiuk/airflow that referenced this issue Jan 14, 2024
@potiuk
Remove wrong changelog about default Python version in the image
99b3107 
In apache#36003 we **thought** we changed default "version" image to
point to "newest" python version not to the "oldest" supported
one - as agreed in https://lists.apache.org/thread/0oxnvct24xlqsj76z42w2ttw2d043oy3

However as observed and tracked in apache#36740 the change was not effective.
We only changed the moment at which latest image is pointing to
2.8.0 but not whether 2.8.0 points to `python-3.8` or `python-3.11'.

This means that we should only do that change for Python 3.9 qnd
revert the changelog (and cherry-pick it to 2.8.1)
@potiuk potiuk mentioned this issue Jan 14, 2024
Merged
potiuk added a commit to potiuk/airflow that referenced this issue Jan 14, 2024
@potiuk
Properly swith the "2.X.Y" image to point to "newest" Python
a7273ac 
The change apache#36003 attempted to change which Python version is used
for the "default" image ("2.8.0" and effectively "latest as well).

But it failed to do so - because of a bug. The changelog about this
have been removed in apache#36777 and we are bringing the change back
(this time properly implemented) and changelog - with aim to
release it in Airflow 2.9.0.

Fixes: apache#36740
@potiuk potiuk mentioned this issue Jan 14, 2024
Merged
potiuk added a commit to potiuk/airflow that referenced this issue Jan 14, 2024
@potiuk
Properly switch the "2.X.Y" image to point to "newest" Python
f0a0d9b 
The change apache#36003 attempted to change which Python version is used
for the "default" image ("2.8.0" and effectively "latest as well).

But it failed to do so - because of a bug. The changelog about this
have been removed in apache#36777 and we are bringing the change back
(this time properly implemented) and changelog - with aim to
release it in Airflow 2.9.0.

Fixes: apache#36740
potiuk added a commit that referenced this issue Jan 14, 2024
@potiuk
Remove wrong changelog about default Python version in the image (#36777
270b112 
)

In #36003 we **thought** we changed default "version" image to
point to "newest" python version not to the "oldest" supported
one - as agreed in https://lists.apache.org/thread/0oxnvct24xlqsj76z42w2ttw2d043oy3

However as observed and tracked in #36740 the change was not effective.
We only changed the moment at which latest image is pointing to
2.8.0 but not whether 2.8.0 points to `python-3.8` or `python-3.11'.

This means that we should only do that change for Python 3.9 qnd
revert the changelog (and cherry-pick it to 2.8.1)
potiuk added a commit that referenced this issue Jan 14, 2024
@potiuk
Remove wrong changelog about default Python version in the image (#36777
903e32c 
)

In #36003 we **thought** we changed default "version" image to
point to "newest" python version not to the "oldest" supported
one - as agreed in https://lists.apache.org/thread/0oxnvct24xlqsj76z42w2ttw2d043oy3

However as observed and tracked in #36740 the change was not effective.
We only changed the moment at which latest image is pointing to
2.8.0 but not whether 2.8.0 points to `python-3.8` or `python-3.11'.

This means that we should only do that change for Python 3.9 qnd
revert the changelog (and cherry-pick it to 2.8.1)

(cherry picked from commit 270b112)
potiuk added a commit that referenced this issue Jan 15, 2024
@potiuk
Properly switch the "2.X.Y" image to point to "newest" Python (#36779)
bd21dec 
The change #36003 attempted to change which Python version is used
for the "default" image ("2.8.0" and effectively "latest as well).

But it failed to do so - because of a bug. The changelog about this
have been removed in #36777 and we are bringing the change back
(this time properly implemented) and changelog - with aim to
release it in Airflow 2.9.0.

Fixes: #36740
ephraimbuddy pushed a commit that referenced this issue Jan 15, 2024
@potiuk @ephraimbuddy
Remove wrong changelog about default Python version in the image (#36777
48331ec 
)

In #36003 we **thought** we changed default "version" image to
point to "newest" python version not to the "oldest" supported
one - as agreed in https://lists.apache.org/thread/0oxnvct24xlqsj76z42w2ttw2d043oy3

However as observed and tracked in #36740 the change was not effective.
We only changed the moment at which latest image is pointing to
2.8.0 but not whether 2.8.0 points to `python-3.8` or `python-3.11'.

This means that we should only do that change for Python 3.9 qnd
revert the changelog (and cherry-pick it to 2.8.1)

(cherry picked from commit 270b112)
@ephraimbuddy ephraimbuddy mentioned this issue Jan 16, 2024
Closed
68 tasks
abhishekbhakat pushed a commit to abhishekbhakat/my_airflow that referenced this issue Mar 5, 2024
@potiuk @abhishekbhakat
Remove wrong changelog about default Python version in the image (apa…
fb4ea7b 
…che#36777)

In apache#36003 we **thought** we changed default "version" image to
point to "newest" python version not to the "oldest" supported
one - as agreed in https://lists.apache.org/thread/0oxnvct24xlqsj76z42w2ttw2d043oy3

However as observed and tracked in apache#36740 the change was not effective.
We only changed the moment at which latest image is pointing to
2.8.0 but not whether 2.8.0 points to `python-3.8` or `python-3.11'.

This means that we should only do that change for Python 3.9 qnd
revert the changelog (and cherry-pick it to 2.8.1)
abhishekbhakat pushed a commit to abhishekbhakat/my_airflow that referenced this issue Mar 5, 2024
@potiuk @abhishekbhakat
Properly switch the "2.X.Y" image to point to "newest" Python (apache…
be80532 
…#36779)

The change apache#36003 attempted to change which Python version is used
for the "default" image ("2.8.0" and effectively "latest as well).

But it failed to do so - because of a bug. The changelog about this
have been removed in apache#36777 and we are bringing the change back
(this time properly implemented) and changelog - with aim to
release it in Airflow 2.9.0.

Fixes: apache#36740
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@potiuk potiuk

Labels
kind:meta High-level information important to the community
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Properly switch the "2.X.Y" image to point to "newest" Python potiuk/airflow
2 participants
@potiuk @hussein-awala