Error "TypeError: object of type 'map' has no len()" when running "airflow db init" #13711
Comments
|
Thanks for opening your first issue here! Be sure to follow the issue template! |
|
More info: My [tool.poetry]
name = "airflow-test"
version = "0.1.0"
description = ""
authors = ["Matt Welke <REDACTED>"]
[tool.poetry.dependencies]
python = "^3.8"
apache-airflow = "^2.0.0"
[tool.poetry.dev-dependencies]
pylint = "^2.6.0"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api" |
|
This is problem with poetry solving dependencies in a wrong way (comparing to PIP at least). However if you want to get some workaround, see ##13558 and especially #13149. I will also add some extra limitations in 2.0.1 for python3-openid as it seems this is what causes the problem. |
potiuk
added a commit
to PolideaInternal/airflow
that referenced
this issue
Jan 16, 2021
Seems that python3-openid dependency is not properly solved by tools like poetry (it is properly resolved by pip). The result is that old version of python3-openid is installed when poetry is used and errors when initdb is run. While we do not use poetry as an official installation mechanism this happens frequently enought and it is easy enough to fix that we can add this dependency to make it easier for poetry users. Related to apache#13711 apache#13558 apache#13149
potiuk
added a commit
that referenced
this issue
Jan 16, 2021
* Adds python3-openid requirement Seems that python3-openid dependency is not properly solved by tools like poetry (it is properly resolved by pip). The result is that old version of python3-openid is installed when poetry is used and errors when initdb is run. While we do not use poetry as an official installation mechanism this happens frequently enought and it is easy enough to fix that we can add this dependency to make it easier for poetry users. Related to #13711 #13558 #13149 * Update setup.cfg
|
@potiuk Thanks for the quick response, and for the links. It looks like someone already reported my issue. Sorry for the duplicate. I did search, but I didn't find it. It looks like Poetry considers this a "won't fix" so I'll keep in mind that I can't use it for Airflow. Some feedback on how I read the docs: The reason I haven't read the installation page with all that info yet is because I was following the table of contents in the docs. The quickstart page comes before the installation page, and the quickstart page includes installation instructions. Because I saw installation instructions in the quickstart, I assumed they were all I needed to know for installation. If I saw installation in the table of contents when I was deciding what to read, I must have assumed it was about installation for production. |
Thanks for that explanation! That makes a lot of sense and I will update the quickstart documentation in a moment to make sure others who follow the same path will be warned! |
|
CC: @mattwelke please take a look at #13727 and see if you think this change addresses the earlier confusion created by inconsistent documentation. |
potiuk
added a commit
to PolideaInternal/airflow
that referenced
this issue
Jan 17, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache#13711 (comment)
potiuk
added a commit
that referenced
this issue
Jan 19, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in #13711 (comment)
kaxil
pushed a commit
that referenced
this issue
Jan 21, 2021
* Adds python3-openid requirement Seems that python3-openid dependency is not properly solved by tools like poetry (it is properly resolved by pip). The result is that old version of python3-openid is installed when poetry is used and errors when initdb is run. While we do not use poetry as an official installation mechanism this happens frequently enought and it is easy enough to fix that we can add this dependency to make it easier for poetry users. Related to #13711 #13558 #13149 * Update setup.cfg (cherry picked from commit df73edf)
kaxil
pushed a commit
that referenced
this issue
Jan 21, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in #13711 (comment) (cherry picked from commit c82f89f)
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Sep 16, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) (cherry picked from commit c82f89f52cb057f6c3771b5f6aef45a5dde0db48) GitOrigin-RevId: f2674f4baec057516c640216127fd17d2619a3d7
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Sep 17, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Sep 23, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Nov 27, 2021
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Mar 10, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Jun 4, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Jun 7, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) (cherry picked from commit c82f89f52cb057f6c3771b5f6aef45a5dde0db48) GitOrigin-RevId: f2674f4baec057516c640216127fd17d2619a3d7
kosteev
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Jul 9, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Aug 27, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Oct 4, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
aglipska
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Oct 7, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Dec 7, 2022
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
leahecole
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Jan 27, 2023
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
kosteev
pushed a commit
to kosteev/composer-airflow-test-copybara
that referenced
this issue
Sep 12, 2024
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
kosteev
pushed a commit
to kosteev/composer-airflow-test-copybara
that referenced
this issue
Sep 12, 2024
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
kosteev
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Sep 17, 2024
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
kosteev
pushed a commit
to GoogleCloudPlatform/composer-airflow
that referenced
this issue
Nov 7, 2024
We have recently seen a number of issues created by users who tried to install airflow with poetry or pip-tools or who had successes with using the latest pip 20.3.3. This change aims to update the 'note' content and make sure installation instructions are consistent everywhere, so that new users are warned against using anything else than PIP and that they are aware about potential problems with 'pip 20.3' and ways to mitigate the problems. This responds to the needs of confused users such as one in apache/airflow#13711 (comment) GitOrigin-RevId: c82f89f52cb057f6c3771b5f6aef45a5dde0db48
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Apache Airflow version: 2.0.0
Environment: Ubuntu 20.04 LTS, Python 3.8.6 via pyenv, in a virtual env via Poetry
uname -a): Linux DESKTOP-QBFDUA0 4.19.104-microsoft-standard Improving the search functionality in the graph view #1 SMP Wed Feb 19 06:37:35 UTC 2020 x86_64 x86_64 x86_64 GNU/LinuxWhat happened:
Following the quick start (https://airflow.apache.org/docs/apache-airflow/stable/start.html), I encountered an error. My steps:
pyenv global 3.8.6to set my Python version to 3.8.6, since I see that Airflow doesn't yet support 3.9 explicitly.poetry initto begin the project. During the interactive prompt, addapache-airflowas a dependency.poetry installto install the Airflow dependency.poetry run airflow db initto run the Airflow init command in a new virtual env managed by Poetry.I saw the following error at the end:
The error was at the end of a long stack trace. The complete output of running that command was:
(sorry, code formatting doesn't appear to work with
<details>)Notably, if I repeat the above steps except that instead of using pyenv to control my Python version and Poetry to control a virtual environment, I use pyenv to control my Python version but do not use a virtual environment, instead installing the Airflow dependency to my Python 3.8.6 shim managed by pyenv, it works. I can run
airflow db initsuccessfully:We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.
flask-jwt-extended 3.25.0 requires PyJWT<2.0,>=1.6.4, but you'll have pyjwt 2.0.0 which is incompatible.
Successfully installed Babel-2.9.0 Flask-Babel-1.0.0 Flask-JWT-Extended-3.25.0 Flask-OpenID-1.2.5 Flask-SQLAlchemy-2.4.4 Mako-1.1.4 PyJWT-2.0.0 PyYAML-5.3.1 WTForms-2.3.3 alembic-1.4.3 apache-airflow-2.0.0 apache-airflow-providers-ftp-1.0.0 apache-airflow-providers-http-1.0.0 apache-airflow-providers-imap-1.0.0 apache-airflow-providers-sqlite-1.0.0 apispec-3.3.2 argcomplete-1.12.2 attrs-20.3.0 cached-property-1.5.2 cattrs-1.1.2 certifi-2020.12.5 cffi-1.14.4 chardet-3.0.4 click-7.1.2 clickclick-20.10.2 colorama-0.4.4 colorlog-4.0.2 commonmark-0.9.1 connexion-2.7.0 croniter-0.3.37 cryptography-3.3.1 defusedxml-0.6.0 dill-0.3.3 dnspython-2.1.0 docutils-0.16 email-validator-1.1.2 flask-1.1.2 flask-appbuilder-3.1.1 flask-caching-1.9.0 flask-login-0.4.1 flask-swagger-0.2.13 flask-wtf-0.14.3 funcsigs-1.0.2 graphviz-0.16 gunicorn-19.10.0 idna-2.10 importlib-metadata-1.7.0 importlib-resources-1.5.0 inflection-0.5.1 iso8601-0.1.13 itsdangerous-1.1.0 jinja2-2.11.2 json-merge-patch-0.2 jsonschema-3.2.0 lazy-object-proxy-1.4.3 lockfile-0.12.2 markdown-3.3.3 markupsafe-1.1.1 marshmallow-3.10.0 marshmallow-enum-1.5.1 marshmallow-oneofschema-2.1.0 marshmallow-sqlalchemy-0.23.1 natsort-7.1.0 numpy-1.19.5 openapi-spec-validator-0.2.9 pandas-1.2.0 pendulum-2.1.2 prison-0.1.3 psutil-5.8.0 pycparser-2.20 pygments-2.7.4 pyrsistent-0.17.3 python-daemon-2.2.4 python-dateutil-2.8.1 python-editor-1.0.4 python-nvd3-0.15.0 python-slugify-4.0.1 python3-openid-3.2.0 pytz-2020.5 pytzdata-2020.1 requests-2.23.0 rich-9.2.0 setproctitle-1.2.1 six-1.15.0 sqlalchemy-1.3.22 sqlalchemy-jsonfield-1.0.0 sqlalchemy-utils-0.36.8 swagger-ui-bundle-0.0.8 tabulate-0.8.7 tenacity-6.2.0 termcolor-1.1.0 text-unidecode-1.3 thrift-0.13.0 typing-extensions-3.7.4.3 tzlocal-1.5.1 unicodecsv-0.14.1 urllib3-1.25.11 werkzeug-1.0.1 zipp-3.4.0
WARNING: You are using pip version 20.2.1; however, version 20.3.3 is available.
You should consider upgrading via the '/home/matt/.pyenv/versions/3.8.6/bin/python3.8 -m pip install --upgrade pip' command.
~ > which airflow
/home/matt/.pyenv/shims/airflow
~ > cd airflow-test-no-virtual-env/
~/airflow-test-no-virtual-env > airflow db init
DB: sqlite:////home/matt/airflow-test-no-virtual-env//airflow.db
[2021-01-15 19:46:59,757] {db.py:678} INFO - Creating tables
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade -> e3a246e0dc1, current schema
INFO [alembic.runtime.migration] Running upgrade e3a246e0dc1 -> 1507a7289a2f, create is_encrypted
/home/matt/.pyenv/versions/3.8.6/lib/python3.8/site-packages/alembic/ddl/sqlite.py:43 UserWarning: Skipping unsupported ALTER for creation of implicit constraintPlease refer to the batch mode feature which allows for SQLite migrations using a copy-and-move strategy.
INFO [alembic.runtime.migration] Running upgrade 1507a7289a2f -> 13eb55f81627, maintain history for compatibility with earlier migrations
INFO [alembic.runtime.migration] Running upgrade 13eb55f81627 -> 338e90f54d61, More logging into task_instance
INFO [alembic.runtime.migration] Running upgrade 338e90f54d61 -> 52d714495f0, job_id indices
INFO [alembic.runtime.migration] Running upgrade 52d714495f0 -> 502898887f84, Adding extra to Log
INFO [alembic.runtime.migration] Running upgrade 502898887f84 -> 1b38cef5b76e, add dagrun
INFO [alembic.runtime.migration] Running upgrade 1b38cef5b76e -> 2e541a1dcfed, task_duration
INFO [alembic.runtime.migration] Running upgrade 2e541a1dcfed -> 40e67319e3a9, dagrun_config
INFO [alembic.runtime.migration] Running upgrade 40e67319e3a9 -> 561833c1c74b, add password column to user
INFO [alembic.runtime.migration] Running upgrade 561833c1c74b -> 4446e08588, dagrun start end
INFO [alembic.runtime.migration] Running upgrade 4446e08588 -> bbc73705a13e, Add notification_sent column to sla_miss
INFO [alembic.runtime.migration] Running upgrade bbc73705a13e -> bba5a7cfc896, Add a column to track the encryption state of the 'Extra' field in connection
INFO [alembic.runtime.migration] Running upgrade bba5a7cfc896 -> 1968acfc09e3, add is_encrypted column to variable table
INFO [alembic.runtime.migration] Running upgrade 1968acfc09e3 -> 2e82aab8ef20, rename user table
INFO [alembic.runtime.migration] Running upgrade 2e82aab8ef20 -> 211e584da130, add TI state index
INFO [alembic.runtime.migration] Running upgrade 211e584da130 -> 64de9cddf6c9, add task fails journal table
INFO [alembic.runtime.migration] Running upgrade 64de9cddf6c9 -> f2ca10b85618, add dag_stats table
INFO [alembic.runtime.migration] Running upgrade f2ca10b85618 -> 4addfa1236f1, Add fractional seconds to mysql tables
INFO [alembic.runtime.migration] Running upgrade 4addfa1236f1 -> 8504051e801b, xcom dag task indices
INFO [alembic.runtime.migration] Running upgrade 8504051e801b -> 5e7d17757c7a, add pid field to TaskInstance
INFO [alembic.runtime.migration] Running upgrade 5e7d17757c7a -> 127d2bf2dfa7, Add dag_id/state index on dag_run table
INFO [alembic.runtime.migration] Running upgrade 127d2bf2dfa7 -> cc1e65623dc7, add max tries column to task instance
WARNI [unusual_prefix_7c969f2cbb12c110385be97d88cb24052dba3ae5_example_kubernetes_executor_config] Could not import DAGs in example_kubernetes_executor_config.py: No module named 'kubernetes'
WARNI [unusual_prefix_7c969f2cbb12c110385be97d88cb24052dba3ae5_example_kubernetes_executor_config] Install kubernetes dependencies with: pip install apache-airflow['cncf.kubernetes']
INFO [alembic.runtime.migration] Running upgrade cc1e65623dc7 -> bdaa763e6c56, Make xcom value column a large binary
INFO [alembic.runtime.migration] Running upgrade bdaa763e6c56 -> 947454bf1dff, add ti job_id index
INFO [alembic.runtime.migration] Running upgrade 947454bf1dff -> d2ae31099d61, Increase text size for MySQL (not relevant for other DBs' text types)
INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 0e2a74e0fc9f, Add time zone awareness
INFO [alembic.runtime.migration] Running upgrade d2ae31099d61 -> 33ae817a1ff4, kubernetes_resource_checkpointing
INFO [alembic.runtime.migration] Running upgrade 33ae817a1ff4 -> 27c6a30d7c24, kubernetes_resource_checkpointing
INFO [alembic.runtime.migration] Running upgrade 27c6a30d7c24 -> 86770d1215c0, add kubernetes scheduler uniqueness
INFO [alembic.runtime.migration] Running upgrade 86770d1215c0, 0e2a74e0fc9f -> 05f30312d566, merge heads
INFO [alembic.runtime.migration] Running upgrade 05f30312d566 -> f23433877c24, fix mysql not null constraint
INFO [alembic.runtime.migration] Running upgrade f23433877c24 -> 856955da8476, fix sqlite foreign key
INFO [alembic.runtime.migration] Running upgrade 856955da8476 -> 9635ae0956e7, index-faskfail
INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> dd25f486b8ea, add idx_log_dag
INFO [alembic.runtime.migration] Running upgrade dd25f486b8ea -> bf00311e1990, add index to taskinstance
INFO [alembic.runtime.migration] Running upgrade 9635ae0956e7 -> 0a2a5b66e19d, add task_reschedule table
INFO [alembic.runtime.migration] Running upgrade 0a2a5b66e19d, bf00311e1990 -> 03bc53e68815, merge_heads_2
INFO [alembic.runtime.migration] Running upgrade 03bc53e68815 -> 41f5f12752f8, add superuser field
INFO [alembic.runtime.migration] Running upgrade 41f5f12752f8 -> c8ffec048a3b, add fields to dag
INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> dd4ecb8fbee3, Add schedule interval to dag
INFO [alembic.runtime.migration] Running upgrade dd4ecb8fbee3 -> 939bb1e647c8, task reschedule fk on cascade delete
INFO [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 6e96a59344a4, Make TaskInstance.pool not nullable
INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> d38e04c12aa2, add serialized_dag table
INFO [alembic.runtime.migration] Running upgrade d38e04c12aa2 -> b3b105409875, add root_dag_id to DAG
INFO [alembic.runtime.migration] Running upgrade 6e96a59344a4 -> 74effc47d867, change datetime to datetime2(6) on MSSQL tables
INFO [alembic.runtime.migration] Running upgrade 939bb1e647c8 -> 004c1210f153, increase queue name size limit
INFO [alembic.runtime.migration] Running upgrade c8ffec048a3b -> a56c9515abdc, Remove dag_stat table
INFO [alembic.runtime.migration] Running upgrade a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875 -> 08364691d074, Merge the four heads back together
INFO [alembic.runtime.migration] Running upgrade 08364691d074 -> fe461863935f, increase_length_for_connection_password
INFO [alembic.runtime.migration] Running upgrade fe461863935f -> 7939bcff74ba, Add DagTags table
INFO [alembic.runtime.migration] Running upgrade 7939bcff74ba -> a4c2fd67d16b, add pool_slots field to task_instance
INFO [alembic.runtime.migration] Running upgrade a4c2fd67d16b -> 852ae6c715af, Add RenderedTaskInstanceFields table
INFO [alembic.runtime.migration] Running upgrade 852ae6c715af -> 952da73b5eff, add dag_code table
INFO [alembic.runtime.migration] Running upgrade 952da73b5eff -> a66efa278eea, Add Precision to execution_date in RenderedTaskInstanceFields table
INFO [alembic.runtime.migration] Running upgrade a66efa278eea -> da3f683c3a5a, Add dag_hash Column to serialized_dag table
INFO [alembic.runtime.migration] Running upgrade da3f683c3a5a -> 92c57b58940d, Create FAB Tables
INFO [alembic.runtime.migration] Running upgrade 92c57b58940d -> 03afc6b6f902, Increase length of FAB ab_view_menu.name column
INFO [alembic.runtime.migration] Running upgrade 03afc6b6f902 -> cf5dc11e79ad, drop_user_and_chart
INFO [alembic.runtime.migration] Running upgrade cf5dc11e79ad -> bbf4a7ad0465, Remove id column from xcom
INFO [alembic.runtime.migration] Running upgrade bbf4a7ad0465 -> b25a55525161, Increase length of pool name
INFO [alembic.runtime.migration] Running upgrade b25a55525161 -> 3c20cacc0044, Add DagRun run_type
INFO [alembic.runtime.migration] Running upgrade 3c20cacc0044 -> 8f966b9c467a, Set conn_type as non-nullable
INFO [alembic.runtime.migration] Running upgrade 8f966b9c467a -> 8d48763f6d53, add unique constraint to conn_id
INFO [alembic.runtime.migration] Running upgrade 8d48763f6d53 -> e38be357a868, Add sensor_instance table
INFO [alembic.runtime.migration] Running upgrade e38be357a868 -> b247b1e3d1ed, Add queued by Job ID to TI
INFO [alembic.runtime.migration] Running upgrade b247b1e3d1ed -> e1a11ece99cc, Add external executor ID to TI
INFO [alembic.runtime.migration] Running upgrade e1a11ece99cc -> bef4f3d11e8b, Drop KubeResourceVersion and KubeWorkerId
INFO [alembic.runtime.migration] Running upgrade bef4f3d11e8b -> 98271e7606e2, Add scheduling_decision to DagRun and DAG
INFO [alembic.runtime.migration] Running upgrade 98271e7606e2 -> 52d53670a240, fix_mssql_exec_date_rendered_task_instance_fields_for_MSSQL
INFO [alembic.runtime.migration] Running upgrade 52d53670a240 -> 364159666cbd, Add creating_job_id to DagRun table
INFO [alembic.runtime.migration] Running upgrade 364159666cbd -> 45ba3f1493b9, add-k8s-yaml-to-rendered-templates
INFO [alembic.runtime.migration] Running upgrade 45ba3f1493b9 -> 849da589634d, Prefix DAG permissions.
INFO [alembic.runtime.migration] Running upgrade 849da589634d -> 2c6edca13270, Resource based permissions.
[2021-01-15 19:47:01,953] {manager.py:727} WARNING - No user yet created, use flask fab command to do it.
[2021-01-15 19:47:04,784] {migration.py:517} INFO - Running upgrade 2c6edca13270 -> 61ec73d9401f, Add description field to connection
[2021-01-15 19:47:04,793] {migration.py:517} INFO - Running upgrade 61ec73d9401f -> 64a7d6477aae, fix description field in connection to be text
[2021-01-15 19:47:04,798] {migration.py:517} INFO - Running upgrade 64a7d6477aae -> e959f08ac86c, Change field in DagCode to MEDIUMTEXT for MySql
[2021-01-15 19:47:05,097] {dagbag.py:440} INFO - Filling up the DagBag from /home/matt/airflow-test-no-virtual-env//dags
[2021-01-15 19:47:05,116] {example_kubernetes_executor_config.py:174} WARNING - Could not import DAGs in example_kubernetes_executor_config.py: No module named 'kubernetes'
[2021-01-15 19:47:05,117] {example_kubernetes_executor_config.py:175} WARNING - Install kubernetes dependencies with: pip install apache-airflow['cncf.kubernetes']
[2021-01-15 19:47:05,133] {dag.py:1813} INFO - Sync 28 DAGs
[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_subdag_operator.section-1
[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_branch_dop_operator_v3
[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_xcom_args
[2021-01-15 19:47:05,136] {dag.py:1832} INFO - Creating ORM DAG for example_external_task_marker_parent
[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for tutorial_taskflow_api_etl
[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_xcom
[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_xcom_args_with_operators
[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_kubernetes_executor
[2021-01-15 19:47:05,137] {dag.py:1832} INFO - Creating ORM DAG for example_python_operator
[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for example_branch_operator
[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for tutorial
[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for test_utils
[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for example_passing_params_via_test_command
[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for latest_only_with_trigger
[2021-01-15 19:47:05,138] {dag.py:1832} INFO - Creating ORM DAG for example_short_circuit_operator
[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_dag_decorator
[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_skip_dag
[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_subdag_operator.section-2
[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_bash_operator
[2021-01-15 19:47:05,139] {dag.py:1832} INFO - Creating ORM DAG for example_subdag_operator
[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_task_group
[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for latest_only
[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_nested_branch_dag
[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_external_task_marker_child
[2021-01-15 19:47:05,140] {dag.py:1832} INFO - Creating ORM DAG for example_trigger_controller_dag
[2021-01-15 19:47:05,141] {dag.py:1832} INFO - Creating ORM DAG for tutorial_etl_dag
[2021-01-15 19:47:05,141] {dag.py:1832} INFO - Creating ORM DAG for example_complex
[2021-01-15 19:47:05,141] {dag.py:1832} INFO - Creating ORM DAG for example_trigger_target_dag
[2021-01-15 19:47:05,146] {dag.py:2266} INFO - Setting next_dagrun for example_bash_operator to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,156] {dag.py:2266} INFO - Setting next_dagrun for example_branch_dop_operator_v3 to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,159] {dag.py:2266} INFO - Setting next_dagrun for example_branch_operator to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_complex to None
[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_dag_decorator to None
[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_external_task_marker_child to None
[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_external_task_marker_parent to None
[2021-01-15 19:47:05,160] {dag.py:2266} INFO - Setting next_dagrun for example_kubernetes_executor to None
[2021-01-15 19:47:05,164] {dag.py:2266} INFO - Setting next_dagrun for example_nested_branch_dag to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_passing_params_via_test_command to 2021-01-15 00:00:00+00:00
[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_python_operator to None
[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_short_circuit_operator to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_skip_dag to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,174] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-1 to None
[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-2 to None
[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_task_group to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_trigger_controller_dag to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_trigger_target_dag to None
[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_xcom to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,175] {dag.py:2266} INFO - Setting next_dagrun for example_xcom_args to None
[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for example_xcom_args_with_operators to None
[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for latest_only to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for latest_only_with_trigger to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for test_utils to None
[2021-01-15 19:47:05,176] {dag.py:2266} INFO - Setting next_dagrun for tutorial to 2021-01-14 00:00:00+00:00
[2021-01-15 19:47:05,177] {dag.py:2266} INFO - Setting next_dagrun for tutorial_etl_dag to None
[2021-01-15 19:47:05,177] {dag.py:2266} INFO - Setting next_dagrun for tutorial_taskflow_api_etl to None
[2021-01-15 19:47:05,192] {dag.py:1813} INFO - Sync 2 DAGs
[2021-01-15 19:47:05,195] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-1 to None
[2021-01-15 19:47:05,195] {dag.py:2266} INFO - Setting next_dagrun for example_subdag_operator.section-2 to None
Initialization done
What you expected to happen: The
airflow db initcommand succeeds when run within my virtual env (using the commandpoetry run airflow db init).What do you think went wrong?
Googling the error let me to find a Stack Overflow page that describes it as ultimately an error related to Python 2 vs. Python 3. Something to do with something called "map". I looked at Airflow supported versions though and found that it supports Python 3, so I found it strange since I was using Python 3. I also tried using the command
pyenv local 3.8.6to try to be extra thorough in locking in a particular supported Python version.Because it doesn't work inside my virtual environment, but works outside it, I think there's something wrong with how all the tooling is working together. I could just stop using a virtual environment to continue. However, what I've learned so far as I learn how to develop in Python is that using virtual environments is a good idea to make my code portable and keep it separate from other stuff on my machine. For example, as I learn Airflow, I might want to bounce back and forth between Airflow 1 and Airflow 2 to see how the installation methods (non-modular vs. modular) work.
How to reproduce it: Run the steps I described above on Ubuntu 20.04 LTS or a similar Linux operating system, using Python 3.
How often does this problem occur? Once? Every time etc?
Every time I try to run the
airflow db initcommand to finish the quickstart.The text was updated successfully, but these errors were encountered: