Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Update Airflow next steps to indicate Python version support #696

Closed
EMCP opened this issue Jul 30, 2022 · 3 comments · Fixed by #972
Closed

docs: Update Airflow next steps to indicate Python version support #696

EMCP opened this issue Jul 30, 2022 · 3 comments · Fixed by #972
Assignees
Labels
kind/Bug Something isn't working valuestream/Meltano

Comments

@EMCP
Copy link

EMCP commented Jul 30, 2022

Meltano Version

2.3.0

Python Version

3.10

Bug scope

CLI (options, error messages, logging, etc.)

Operating System

Kubuntu 22.04

Description

while attempting to install .. got stuck when attempting to add airflow will put replication steps here

  1. start a blank new project with the latest meltano available..
$ meltano discover orchestrators
2022-07-30T18:38:08.881040Z [info     ] Environment 'dev' is active
Orchestrators
airflow

$ meltano add orchestrators airflow
2022-07-30T18:38:18.506731Z [info     ] Environment 'dev' is active
Added orchestrator 'airflow' to your Meltano project
Variant:        apache (default)
Repository:     https://github.com/apache/airflow
Documentation:  https://docs.meltano.com/guide/orchestration

Added required file bundle 'files-airflow' to your Meltano project
Variant:        meltano (default)
Repository:     https://github.com/meltano/files-airflow
Documentation:  https://hub.meltano.com/files/files-airflow

Installing orchestrator 'airflow'...
Installing file bundle 'files-airflow'...
Orchestrator 'airflow' could not be installed: failed to install plugin 'airflow'.
ERROR: 404 Client Error: Not Found for url: https://raw.githubusercontent.com/apache/airflow/constraints-2.1.2/constraints-3.10.txt

Installed file bundle 'files-airflow'
Adding 'files-airflow' files to project...
Created orchestrate/dags/meltano.py
Installed 1/2 plugins
Failed to install plugin(s)

this might be a hinderance to meltano fully supporting 3.10 at this time.. I understand though and will back off to 3.9 or lower

Code

No response

@EMCP EMCP added kind/Bug Something isn't working valuestream/Meltano labels Jul 30, 2022
@EMCP
Copy link
Author

EMCP commented Jul 30, 2022

seems according to apache/airflow#19059 , support for py 3.10 will come in v2.3.0 of airflow

@EMCP
Copy link
Author

EMCP commented Jul 30, 2022

can confirm .. https://raw.githubusercontent.com/apache/airflow/constraints-2.3.0/constraints-3.10.txt is a valid path

is there a troubleshooting area I can add this tip to? or how can I help smooth this out for future users?

  orchestrators:
  - name: airflow
    variant: apache
    pip_url: apache-airflow==2.3.0 --constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.3.0/constraints-${MELTANO__PYTHON_VERSION}.txt

@tayloramurphy
Copy link
Collaborator

@EMCP this makes me think of this issue #126

Related to this too, we have an issue to get Airflow upgraded to 2.3 #658 but there's some work in Meltano we'd need to do to make that happen.

That said, we could add a note in the next_steps in https://github.com/meltano/hub/blob/main/_data/meltano/orchestrators/airflow/apache.yml#L20 to alert users to the Python/Airflow mismatch.

@tayloramurphy tayloramurphy transferred this issue from meltano/meltano Aug 1, 2022
@tayloramurphy tayloramurphy changed the title Getting error when attempting to use airflow with meltano and py3.10 docs: Update Airflow next steps to indicate Python version support Aug 1, 2022
@tayloramurphy tayloramurphy self-assigned this Aug 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/Bug Something isn't working valuestream/Meltano
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants