worker service not starting on cdh6.3 #10
Comments
|
I've faced the same issue. I've found out that my rabbitMq broker wasn't set up correctly on the cloudera interface |
Thank you it worked for my after starting RabbitMQ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
[13/Jul/2020 00:12:41 +0000] 117396 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/csd/csd.sh start_worker
[13/Jul/2020 00:12:41 +0000] 117396 MainThread redactor ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/12038-airflow-AIRFLOW_WORKER
[13/Jul/2020 00:12:41 +0000] 117396 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 117423
Mon Jul 13 00:12:42 EDT 2020
'
'
++ seq 1 1
seq 1 $COUNT'++ AIRFLOW_DIRNAME=AIRFLOW-1.10.3-python2.7.15_b2
++ export AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ export PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ '[' -z '' ']'
++ export AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ printf '! -name %s ' cloudera-config.sh hue.sh impala.sh sqoop.sh supervisor.conf config.zip proc.json '.log' '*.keytab' '*jceks'
Mon Jul 13 00:12:42 EDT 2020
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F: '{print $1}'
++ sed -e 's|AIRFLOW_HOME=/var/lib/airflow:||'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ AIRFLOW_HOME=/var/lib/airflow
++ awk -F: '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ sed -e 's|CELERY_BROKER=RabbitMQ:||'
++ echo CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER=RabbitMQ
++ awk -F= '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ awk -F= '{print $2}'
++ CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER_HOST=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_HOST=
++ sed -e 's|CELERY_BROKER_HOST=:||'
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_HOST=
++ echo CELERY_BROKER_HOST=
++ awk -F= '{print $2}'
++ CELERY_BROKER_HOST=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PASS=
++ echo CELERY_BROKER_PASS=
++ sed -e 's|CELERY_BROKER_PASS=:||'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $2}'
++ CELERY_BROKER_PASS=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PORT=
++ echo CELERY_BROKER_PORT=
++ sed -e 's|CELERY_BROKER_PORT=:||'
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PORT=
++ echo CELERY_BROKER_PORT=
++ awk -F= '{print $2}'
++ CELERY_BROKER_PORT=
++ echo CELERY_BROKER_USER=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_USER=
++ sed -e 's|CELERY_BROKER_USER=:||'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $2}'
++ CELERY_BROKER_USER=
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F: '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ sed -e 's|DB_HOST=usadc-vsedgdh03:||'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $2}'
++ DB_HOST=usadc-vsedgdh03
++ echo DB_NAME=airflow
++ awk -F: '{print $1}'
++ echo DB_NAME=airflow
++ sed -e 's|DB_NAME=airflow:||'
++ echo DB_NAME=airflow
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo DB_NAME=airflow
++ DB_NAME=airflow
++ echo DB_PASS=p@ssw0rd123
++ awk -F: '{print $1}'
++ sed -e 's|DB_PASS=p@ssw0rd123:||'
++ echo DB_PASS=p@ssw0rd123
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $2}'
++ DB_PASS=p@ssw0rd123
++ echo DB_PORT=3306
++ awk -F: '{print $1}'
++ echo DB_PORT=3306
++ sed -e 's|DB_PORT=3306:||'
++ awk -F= '{print $1}'
++ echo DB_PORT=3306
++ echo DB_PORT=3306
++ awk -F= '{print $2}'
++ DB_PORT=3306
++ echo DB_TYPE=MySQL
++ awk -F: '{print $1}'
++ echo DB_TYPE=MySQL
++ sed -e 's|DB_TYPE=MySQL:||'
++ echo DB_TYPE=MySQL
++ awk -F= '{print $1}'
++ echo DB_TYPE=MySQL
++ awk -F= '{print $2}'
++ DB_TYPE=MySQL
++ echo DB_USER=airflow
++ awk -F: '{print $1}'
++ echo DB_USER=airflow
++ sed -e 's|DB_USER=airflow:||'
++ echo DB_USER=airflow
++ awk -F= '{print $1}'
++ echo DB_USER=airflow
++ awk -F= '{print $2}'
++ DB_USER=airflow
++ echo api:auth_backend=airflow.api.auth.backend.default
++ awk -F: '{print $1}'
++ echo api:auth_backend=airflow.api.auth.backend.default
++ sed -e 's|api:||'
++ awk -F= '{print $1}'
++ echo auth_backend=airflow.api.auth.backend.default
++ awk -F= '{print $2}'
++ echo auth_backend=airflow.api.auth.backend.default
++ echo celery:default_queue=default
++ awk -F: '{print $1}'
++ echo celery:default_queue=default
++ sed -e 's|celery:||'
++ awk -F= '{print $1}'
++ echo default_queue=default
++ echo default_queue=default
++ awk -F= '{print $2}'
++ echo celery:worker_concurrency=16
++ awk -F: '{print $1}'
++ sed -e 's|celery:||'
++ echo celery:worker_concurrency=16
++ echo worker_concurrency=16
++ awk -F= '{print $1}'
++ echo worker_concurrency=16
++ awk -F= '{print $2}'
++ echo celery:worker_log_server_port=8793
++ awk -F: '{print $1}'
++ echo celery:worker_log_server_port=8793
++ sed -e 's|celery:||'
++ awk -F= '{print $1}'
++ echo worker_log_server_port=8793
++ awk -F= '{print $2}'
++ echo worker_log_server_port=8793
++ echo cli:api_client=airflow.api.client.local_client
++ awk -F: '{print $1}'
++ echo cli:api_client=airflow.api.client.local_client
++ sed -e 's|cli:||'
++ awk -F= '{print $1}'
++ echo api_client=airflow.api.client.local_client
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $2}'
++ echo cli:endpoint_url=http://localhost:8080
++ awk -F: '{print $1}'
++ echo cli:endpoint_url=http://localhost:8080
++ sed -e 's|cli:||'
++ awk -F= '{print $1}'
++ echo endpoint_url=http://localhost:8080
++ echo endpoint_url=http://localhost:8080
++ awk -F= '{print $2}'
++ echo core:base_log_folder=/var/log/airflow
++ awk -F: '{print $1}'
++ echo core:base_log_folder=/var/log/airflow
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo base_log_folder=/var/log/airflow
++ echo base_log_folder=/var/log/airflow
++ awk -F= '{print $2}'
++ echo core:dag_concurrency=4
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:dag_concurrency=4
++ awk -F= '{print $1}'
++ echo dag_concurrency=4
++ awk -F= '{print $2}'
++ echo dag_concurrency=4
++ awk -F: '{print $1}'
++ echo core:dagbag_import_timeout=30
++ sed -e 's|core:||'
++ echo core:dagbag_import_timeout=30
++ echo dagbag_import_timeout=30
++ awk -F= '{print $1}'
++ echo dagbag_import_timeout=30
++ awk -F= '{print $2}'
++ echo core:dags_are_paused_at_creation=true
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:dags_are_paused_at_creation=true
++ awk -F= '{print $1}'
++ echo dags_are_paused_at_creation=true
++ echo dags_are_paused_at_creation=true
++ awk -F= '{print $2}'
++ echo core:dags_folder=/var/lib/airflow/dags
++ awk -F: '{print $1}'
++ echo core:dags_folder=/var/lib/airflow/dags
++ sed -e 's|core:||'
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $1}'
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $2}'
++ echo core:default_impersonation=
++ awk -F: '{print $1}'
++ echo core:default_impersonation=
++ sed -e 's|core:||'
++ echo default_impersonation=
++ awk -F= '{print $1}'
++ echo default_impersonation=
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:donot_pickle=false
++ sed -e 's|core:||'
++ echo core:donot_pickle=false
++ echo donot_pickle=false
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo donot_pickle=false
++ awk -F: '{print $1}'
++ echo core:executor=LocalExecutor
++ sed -e 's|core:||'
++ echo core:executor=LocalExecutor
++ echo executor=LocalExecutor
++ awk -F= '{print $1}'
++ echo executor=LocalExecutor
++ awk -F= '{print $2}'
++ echo core:fernet_key=
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:fernet_key=
++ awk -F= '{print $1}'
++ echo fernet_key=
++ echo fernet_key=
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:load_examples=false
++ echo core:load_examples=false
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo load_examples=false
++ echo load_examples=false
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:max_active_runs_per_dag=16
++ sed -e 's|core:||'
++ echo core:max_active_runs_per_dag=16
++ echo max_active_runs_per_dag=16
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo max_active_runs_per_dag=16
++ awk -F: '{print $1}'
++ echo core:non_pooled_task_slot_count=128
++ sed -e 's|core:||'
++ echo core:non_pooled_task_slot_count=128
++ awk -F= '{print $1}'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $2}'
++ echo non_pooled_task_slot_count=128
++ awk -F: '{print $1}'
++ echo core:parallelism=8
++ echo core:parallelism=8
++ sed -e 's|core:||'
++ echo parallelism=8
++ awk -F= '{print $1}'
++ echo parallelism=8
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ sed -e 's|core:||'
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $1}'
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_enabled=true
++ sed -e 's|core:||'
++ echo core:sql_alchemy_pool_enabled=true
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_enabled=true
++ awk -F= '{print $2}'
++ echo sql_alchemy_pool_enabled=true
++ echo core:sql_alchemy_pool_recycle=1800
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_recycle=1800
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_recycle=1800
++ awk -F= '{print $2}'
++ echo sql_alchemy_pool_recycle=1800
++ echo core:sql_alchemy_pool_size=5
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_size=5
++ sed -e 's|core:||'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_reconnect_timeout=300
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_reconnect_timeout=300
++ sed -e 's|core:||'
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo sql_alchemy_reconnect_timeout=300
++ echo core:task_runner=BashTaskRunner
++ awk -F: '{print $1}'
++ echo core:task_runner=BashTaskRunner
++ sed -e 's|core:||'
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $1}'
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $2}'
++ echo hive:default_hive_mapred_queue=
++ awk -F: '{print $1}'
++ echo hive:default_hive_mapred_queue=
++ sed -e 's|hive:||'
++ echo default_hive_mapred_queue=
++ awk -F= '{print $1}'
++ echo default_hive_mapred_queue=
++ awk -F= '{print $2}'
++ echo log_dir=/var/log/airflow
++ awk -F: '{print $1}'
++ echo log_dir=/var/log/airflow
++ sed -e 's|log_dir=/var/log/airflow:||'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $1}'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $2}'
++ log_dir=/var/log/airflow
++ echo operators:default_cpus=1
++ awk -F: '{print $1}'
++ echo operators:default_cpus=1
++ sed -e 's|operators:||'
++ echo default_cpus=1
++ awk -F= '{print $1}'
++ echo default_cpus=1
++ awk -F= '{print $2}'
++ echo operators:default_disk=512
++ awk -F: '{print $1}'
++ echo operators:default_disk=512
++ sed -e 's|operators:||'
++ echo default_disk=512
++ awk -F= '{print $1}'
++ echo default_disk=512
++ awk -F= '{print $2}'
++ echo operators:default_gpus=0
++ awk -F: '{print $1}'
++ echo operators:default_gpus=0
++ sed -e 's|operators:||'
++ echo default_gpus=0
++ awk -F= '{print $1}'
++ echo default_gpus=0
++ awk -F= '{print $2}'
++ echo operators:default_owner=Airflow
++ awk -F: '{print $1}'
++ echo operators:default_owner=Airflow
++ sed -e 's|operators:||'
++ echo default_owner=Airflow
++ awk -F= '{print $1}'
++ echo default_owner=Airflow
++ awk -F= '{print $2}'
++ echo operators:default_ram=512
++ awk -F: '{print $1}'
++ echo operators:default_ram=512
++ sed -e 's|operators:||'
++ echo default_ram=512
++ awk -F= '{print $1}'
++ echo default_ram=512
++ awk -F= '{print $2}'
++ echo smtp:smtp_host=localhost
++ awk -F: '{print $1}'
++ echo smtp:smtp_host=localhost
++ sed -e 's|smtp:||'
++ awk -F= '{print $1}'
++ echo smtp_host=localhost
++ awk -F= '{print $2}'
++ echo smtp_host=localhost
++ awk -F: '{print $1}'
++ echo smtp:smtp_mail_from=airflow@localhost
++ sed -e 's|smtp:||'
++ echo smtp:smtp_mail_from=airflow@localhost
++ awk -F= '{print $1}'
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $2}'
++ echo smtp_mail_from=airflow@localhost
++ echo smtp:smtp_password=
++ awk -F: '{print $1}'
++ echo smtp:smtp_password=
++ sed -e 's|smtp:||'
++ echo smtp_password=
++ awk -F= '{print $1}'
++ echo smtp_password=
++ awk -F= '{print $2}'
++ echo smtp:smtp_port=25
++ awk -F: '{print $1}'
++ echo smtp:smtp_port=25
++ sed -e 's|smtp:||'
++ echo smtp_port=25
++ awk -F= '{print $1}'
++ echo smtp_port=25
++ awk -F= '{print $2}'
++ echo smtp:smtp_ssl=false
++ awk -F: '{print $1}'
++ echo smtp:smtp_ssl=false
++ sed -e 's|smtp:||'
++ echo smtp_ssl=false
++ awk -F= '{print $1}'
++ echo smtp_ssl=false
++ awk -F= '{print $2}'
++ echo smtp:smtp_starttls=true
++ awk -F: '{print $1}'
++ echo smtp:smtp_starttls=true
++ sed -e 's|smtp:||'
++ echo smtp_starttls=true
++ awk -F= '{print $1}'
++ echo smtp_starttls=true
++ awk -F= '{print $2}'
++ echo smtp:smtp_user=
++ awk -F: '{print $1}'
++ echo smtp:smtp_user=
++ sed -e 's|smtp:||'
++ echo smtp_user=
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo smtp_user=
++ date '+%Y-%m-%d %H:%M:%S'
/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/configuration.py:214: FutureWarning: The task_runner setting in [core] has the old default value of u'BashTaskRunner'. This value has been changed to u'StandardTaskRunner' in the running config, but please update your config before Apache Airflow 2.0.
FutureWarning,
Traceback (most recent call last):
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/.airflow", line 32, in
args.func(args)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper
return f(*args, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/bin/cli.py", line 1090, in worker
worker.run(**options)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in run
**kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 101, in init
self.setup_instance(**self.prepare_args(**kwargs))
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 122, in setup_instance
self._conninfo = self.app.connection_for_read()
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 743, in connection_for_read
return self._connection(url or self.conf.broker_read_url, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 819, in _connection
'broker_connection_timeout', connect_timeout
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/connection.py", line 189, in init
url_params = parse_url(hostname)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 47, in parse_url
scheme, host, port, user, password, path, query = _parse_url(url)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 81, in url_to_parts
parts.port,
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 162, in port
port = int(port, 10)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/types/newint.py", line 76, in new
return super(newint, cls).new(cls, val)
ValueError: invalid literal for long() with base 10: ''
[13/Jul/2020 00:12:48 +0000] 120062 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/csd/csd.sh start_worker
[13/Jul/2020 00:12:48 +0000] 120062 MainThread redactor ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/12038-airflow-AIRFLOW_WORKER
[13/Jul/2020 00:12:48 +0000] 120062 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 120079
Mon Jul 13 00:12:48 EDT 2020
'
'
++ seq 1 1
seq 1 $COUNT'++ AIRFLOW_DIRNAME=AIRFLOW-1.10.3-python2.7.15_b2
++ export AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ export PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ '[' -z '' ']'
++ export AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ printf '! -name %s ' cloudera-config.sh hue.sh impala.sh sqoop.sh supervisor.conf config.zip proc.json '.log' '*.keytab' '*jceks'
Mon Jul 13 00:12:49 EDT 2020
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ awk -F: '{print $1}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ echo AIRFLOW_HOME=/var/lib/airflow
++ sed -e 's|AIRFLOW_HOME=/var/lib/airflow:||'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F= '{print $1}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F= '{print $2}'
++ AIRFLOW_HOME=/var/lib/airflow
++ echo CELERY_BROKER=RabbitMQ
++ awk -F: '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ sed -e 's|CELERY_BROKER=RabbitMQ:||'
++ awk -F= '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER=RabbitMQ
++ awk -F= '{print $2}'
++ CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER_HOST=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_HOST=
++ sed -e 's|CELERY_BROKER_HOST=:||'
++ echo CELERY_BROKER_HOST=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_HOST=
++ awk -F= '{print $2}'
++ CELERY_BROKER_HOST=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PASS=
++ sed -e 's|CELERY_BROKER_PASS=:||'
++ echo CELERY_BROKER_PASS=
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $2}'
++ CELERY_BROKER_PASS=
++ echo CELERY_BROKER_PORT=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PORT=
++ sed -e 's|CELERY_BROKER_PORT=:||'
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PORT=
++ awk -F= '{print $2}'
++ echo CELERY_BROKER_PORT=
++ CELERY_BROKER_PORT=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_USER=
++ echo CELERY_BROKER_USER=
++ sed -e 's|CELERY_BROKER_USER=:||'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $2}'
++ CELERY_BROKER_USER=
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F: '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ sed -e 's|DB_HOST=usadc-vsedgdh03:||'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $2}'
++ DB_HOST=usadc-vsedgdh03
++ echo DB_NAME=airflow
++ awk -F: '{print $1}'
++ echo DB_NAME=airflow
++ sed -e 's|DB_NAME=airflow:||'
++ echo DB_NAME=airflow
++ awk -F= '{print $1}'
++ echo DB_NAME=airflow
++ awk -F= '{print $2}'
++ DB_NAME=airflow
++ echo DB_PASS=p@ssw0rd123
++ awk -F: '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ sed -e 's|DB_PASS=p@ssw0rd123:||'
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $2}'
++ DB_PASS=p@ssw0rd123
++ echo DB_PORT=3306
++ awk -F: '{print $1}'
++ echo DB_PORT=3306
++ sed -e 's|DB_PORT=3306:||'
++ awk -F= '{print $1}'
++ echo DB_PORT=3306
++ echo DB_PORT=3306
++ awk -F= '{print $2}'
++ DB_PORT=3306
++ awk -F: '{print $1}'
++ echo DB_TYPE=MySQL
++ sed -e 's|DB_TYPE=MySQL:||'
++ echo DB_TYPE=MySQL
++ echo DB_TYPE=MySQL
++ awk -F= '{print $1}'
++ echo DB_TYPE=MySQL
++ awk -F= '{print $2}'
++ DB_TYPE=MySQL
++ awk -F: '{print $1}'
++ echo DB_USER=airflow
++ sed -e 's|DB_USER=airflow:||'
++ echo DB_USER=airflow
++ echo DB_USER=airflow
++ awk -F= '{print $1}'
++ echo DB_USER=airflow
++ awk -F= '{print $2}'
++ DB_USER=airflow
++ echo api:auth_backend=airflow.api.auth.backend.default
++ awk -F: '{print $1}'
++ echo api:auth_backend=airflow.api.auth.backend.default
++ sed -e 's|api:||'
++ echo auth_backend=airflow.api.auth.backend.default
++ awk -F= '{print $1}'
++ echo auth_backend=airflow.api.auth.backend.default
++ awk -F= '{print $2}'
++ echo celery:default_queue=default
++ awk -F: '{print $1}'
++ echo celery:default_queue=default
++ sed -e 's|celery:||'
++ echo default_queue=default
++ awk -F= '{print $1}'
++ echo default_queue=default
++ awk -F= '{print $2}'
++ echo celery:worker_concurrency=16
++ awk -F: '{print $1}'
++ echo celery:worker_concurrency=16
++ sed -e 's|celery:||'
++ echo worker_concurrency=16
++ awk -F= '{print $1}'
++ echo worker_concurrency=16
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo celery:worker_log_server_port=8793
++ echo celery:worker_log_server_port=8793
++ sed -e 's|celery:||'
++ echo worker_log_server_port=8793
++ awk -F= '{print $1}'
++ echo worker_log_server_port=8793
++ awk -F= '{print $2}'
++ echo cli:api_client=airflow.api.client.local_client
++ awk -F: '{print $1}'
++ echo cli:api_client=airflow.api.client.local_client
++ sed -e 's|cli:||'
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $1}'
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $2}'
++ echo cli:endpoint_url=http://localhost:8080
++ awk -F: '{print $1}'
++ echo cli:endpoint_url=http://localhost:8080
++ sed -e 's|cli:||'
++ echo endpoint_url=http://localhost:8080
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo endpoint_url=http://localhost:8080
++ awk -F: '{print $1}'
++ echo core:base_log_folder=/var/log/airflow
++ echo core:base_log_folder=/var/log/airflow
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo base_log_folder=/var/log/airflow
++ echo base_log_folder=/var/log/airflow
++ awk -F= '{print $2}'
++ echo core:dag_concurrency=4
++ awk -F: '{print $1}'
++ echo core:dag_concurrency=4
++ sed -e 's|core:||'
++ echo dag_concurrency=4
++ awk -F= '{print $1}'
++ echo dag_concurrency=4
++ awk -F= '{print $2}'
++ echo core:dagbag_import_timeout=30
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:dagbag_import_timeout=30
++ echo dagbag_import_timeout=30
++ awk -F= '{print $1}'
++ echo dagbag_import_timeout=30
++ awk -F= '{print $2}'
++ echo core:dags_are_paused_at_creation=true
++ awk -F: '{print $1}'
++ echo core:dags_are_paused_at_creation=true
++ sed -e 's|core:||'
++ echo dags_are_paused_at_creation=true
++ awk -F= '{print $1}'
++ echo dags_are_paused_at_creation=true
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:dags_folder=/var/lib/airflow/dags
++ echo core:dags_folder=/var/lib/airflow/dags
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo dags_folder=/var/lib/airflow/dags
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $2}'
++ echo core:default_impersonation=
++ awk -F: '{print $1}'
++ echo core:default_impersonation=
++ sed -e 's|core:||'
++ echo default_impersonation=
++ awk -F= '{print $1}'
++ echo default_impersonation=
++ awk -F= '{print $2}'
++ echo core:donot_pickle=false
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:donot_pickle=false
++ echo donot_pickle=false
++ awk -F= '{print $1}'
++ echo donot_pickle=false
++ awk -F= '{print $2}'
++ echo core:executor=LocalExecutor
++ awk -F: '{print $1}'
++ echo core:executor=LocalExecutor
++ sed -e 's|core:||'
++ echo executor=LocalExecutor
++ awk -F= '{print $1}'
++ echo executor=LocalExecutor
++ awk -F= '{print $2}'
++ echo core:fernet_key=
++ awk -F: '{print $1}'
++ echo core:fernet_key=
++ sed -e 's|core:||'
++ echo fernet_key=
++ awk -F= '{print $1}'
++ echo fernet_key=
++ awk -F= '{print $2}'
++ echo core:load_examples=false
++ awk -F: '{print $1}'
++ echo core:load_examples=false
++ sed -e 's|core:||'
++ echo load_examples=false
++ awk -F= '{print $1}'
++ echo load_examples=false
++ awk -F= '{print $2}'
++ echo core:max_active_runs_per_dag=16
++ awk -F: '{print $1}'
++ echo core:max_active_runs_per_dag=16
++ sed -e 's|core:||'
++ echo max_active_runs_per_dag=16
++ awk -F= '{print $1}'
++ echo max_active_runs_per_dag=16
++ awk -F= '{print $2}'
++ echo core:non_pooled_task_slot_count=128
++ awk -F: '{print $1}'
++ echo core:non_pooled_task_slot_count=128
++ sed -e 's|core:||'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $1}'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $2}'
++ echo core:parallelism=8
++ awk -F: '{print $1}'
++ echo core:parallelism=8
++ sed -e 's|core:||'
++ echo parallelism=8
++ awk -F= '{print $1}'
++ echo parallelism=8
++ awk -F= '{print $2}'
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ awk -F: '{print $1}'
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ sed -e 's|core:||'
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $1}'
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_pool_enabled=true
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_enabled=true
++ sed -e 's|core:||'
++ echo sql_alchemy_pool_enabled=true
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_enabled=true
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_pool_recycle=1800
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_recycle=1800
++ sed -e 's|core:||'
++ echo sql_alchemy_pool_recycle=1800
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_recycle=1800
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_pool_size=5
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_size=5
++ sed -e 's|core:||'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_reconnect_timeout=300
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_reconnect_timeout=300
++ sed -e 's|core:||'
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $1}'
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:task_runner=BashTaskRunner
++ echo core:task_runner=BashTaskRunner
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $2}'
++ echo task_runner=BashTaskRunner
++ awk -F: '{print $1}'
++ echo hive:default_hive_mapred_queue=
++ sed -e 's|hive:||'
++ echo hive:default_hive_mapred_queue=
++ echo default_hive_mapred_queue=
++ awk -F= '{print $1}'
++ echo default_hive_mapred_queue=
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo log_dir=/var/log/airflow
++ sed -e 's|log_dir=/var/log/airflow:||'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $1}'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $2}'
++ echo log_dir=/var/log/airflow
++ log_dir=/var/log/airflow
++ echo operators:default_cpus=1
++ awk -F: '{print $1}'
++ echo operators:default_cpus=1
++ sed -e 's|operators:||'
++ awk -F= '{print $1}'
++ echo default_cpus=1
++ awk -F= '{print $2}'
++ echo default_cpus=1
++ awk -F: '{print $1}'
++ echo operators:default_disk=512
++ echo operators:default_disk=512
++ sed -e 's|operators:||'
++ echo default_disk=512
++ awk -F= '{print $1}'
++ echo default_disk=512
++ awk -F= '{print $2}'
++ echo operators:default_gpus=0
++ awk -F: '{print $1}'
++ echo operators:default_gpus=0
++ sed -e 's|operators:||'
++ echo default_gpus=0
++ awk -F= '{print $1}'
++ echo default_gpus=0
++ awk -F= '{print $2}'
++ echo operators:default_owner=Airflow
++ awk -F: '{print $1}'
++ echo operators:default_owner=Airflow
++ sed -e 's|operators:||'
++ echo default_owner=Airflow
++ awk -F= '{print $1}'
++ echo default_owner=Airflow
++ awk -F= '{print $2}'
++ echo operators:default_ram=512
++ awk -F: '{print $1}'
++ echo operators:default_ram=512
++ sed -e 's|operators:||'
++ echo default_ram=512
++ awk -F= '{print $1}'
++ echo default_ram=512
++ awk -F= '{print $2}'
++ echo smtp:smtp_host=localhost
++ awk -F: '{print $1}'
++ echo smtp:smtp_host=localhost
++ sed -e 's|smtp:||'
++ echo smtp_host=localhost
++ awk -F= '{print $1}'
++ echo smtp_host=localhost
++ awk -F= '{print $2}'
++ echo smtp:smtp_mail_from=airflow@localhost
++ awk -F: '{print $1}'
++ echo smtp:smtp_mail_from=airflow@localhost
++ sed -e 's|smtp:||'
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $1}'
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $2}'
++ echo smtp:smtp_password=
++ awk -F: '{print $1}'
++ echo smtp:smtp_password=
++ sed -e 's|smtp:||'
++ echo smtp_password=
++ awk -F= '{print $1}'
++ echo smtp_password=
++ awk -F= '{print $2}'
++ echo smtp:smtp_port=25
++ awk -F: '{print $1}'
++ echo smtp:smtp_port=25
++ sed -e 's|smtp:||'
++ echo smtp_port=25
++ awk -F= '{print $1}'
++ echo smtp_port=25
++ awk -F= '{print $2}'
++ echo smtp:smtp_ssl=false
++ awk -F: '{print $1}'
++ echo smtp:smtp_ssl=false
++ sed -e 's|smtp:||'
++ echo smtp_ssl=false
++ awk -F= '{print $1}'
++ echo smtp_ssl=false
++ awk -F= '{print $2}'
++ echo smtp:smtp_starttls=true
++ awk -F: '{print $1}'
++ echo smtp:smtp_starttls=true
++ sed -e 's|smtp:||'
++ echo smtp_starttls=true
++ awk -F= '{print $1}'
++ echo smtp_starttls=true
++ awk -F= '{print $2}'
++ echo smtp:smtp_user=
++ awk -F: '{print $1}'
++ echo smtp:smtp_user=
++ sed -e 's|smtp:||'
++ echo smtp_user=
++ awk -F= '{print $1}'
++ echo smtp_user=
++ awk -F= '{print $2}'
++ date '+%Y-%m-%d %H:%M:%S'
/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/configuration.py:214: FutureWarning: The task_runner setting in [core] has the old default value of u'BashTaskRunner'. This value has been changed to u'StandardTaskRunner' in the running config, but please update your config before Apache Airflow 2.0.
FutureWarning,
Traceback (most recent call last):
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/.airflow", line 32, in
args.func(args)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper
return f(*args, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/bin/cli.py", line 1090, in worker
worker.run(**options)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in run
**kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 101, in init
self.setup_instance(**self.prepare_args(**kwargs))
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 122, in setup_instance
self._conninfo = self.app.connection_for_read()
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 743, in connection_for_read
return self._connection(url or self.conf.broker_read_url, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 819, in _connection
'broker_connection_timeout', connect_timeout
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/connection.py", line 189, in init
url_params = parse_url(hostname)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 47, in parse_url
scheme, host, port, user, password, path, query = _parse_url(url)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 81, in url_to_parts
parts.port,
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 162, in port
port = int(port, 10)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/types/newint.py", line 76, in new
return super(newint, cls).new(cls, val)
ValueError: invalid literal for long() with base 10: ''
[13/Jul/2020 00:12:55 +0000] 120851 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/csd/csd.sh start_worker
[13/Jul/2020 00:12:55 +0000] 120851 MainThread redactor ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/12038-airflow-AIRFLOW_WORKER
[13/Jul/2020 00:12:55 +0000] 120851 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 120860
Mon Jul 13 00:12:56 EDT 2020
'
'
++ seq 1 1
seq 1 $COUNT'++ AIRFLOW_DIRNAME=AIRFLOW-1.10.3-python2.7.15_b2
++ export AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ export PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ '[' -z '' ']'
++ export AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ printf '! -name %s ' cloudera-config.sh hue.sh impala.sh sqoop.sh supervisor.conf config.zip proc.json '.log' '*.keytab' '*jceks'
Mon Jul 13 00:12:56 EDT 2020
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F: '{print $1}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ sed -e 's|AIRFLOW_HOME=/var/lib/airflow:||'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F= '{print $1}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F= '{print $2}'
++ AIRFLOW_HOME=/var/lib/airflow
++ echo CELERY_BROKER=RabbitMQ
++ awk -F: '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ sed -e 's|CELERY_BROKER=RabbitMQ:||'
++ echo CELERY_BROKER=RabbitMQ
++ awk -F= '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ awk -F= '{print $2}'
++ CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER_HOST=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_HOST=
++ sed -e 's|CELERY_BROKER_HOST=:||'
++ echo CELERY_BROKER_HOST=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_HOST=
++ awk -F= '{print $2}'
++ CELERY_BROKER_HOST=
++ echo CELERY_BROKER_PASS=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PASS=
++ sed -e 's|CELERY_BROKER_PASS=:||'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $2}'
++ CELERY_BROKER_PASS=
++ echo CELERY_BROKER_PORT=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PORT=
++ sed -e 's|CELERY_BROKER_PORT=:||'
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PORT=
++ awk -F= '{print $2}'
++ echo CELERY_BROKER_PORT=
++ CELERY_BROKER_PORT=
++ echo CELERY_BROKER_USER=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_USER=
++ sed -e 's|CELERY_BROKER_USER=:||'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $2}'
++ CELERY_BROKER_USER=
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F: '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ sed -e 's|DB_HOST=usadc-vsedgdh03:||'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $2}'
++ DB_HOST=usadc-vsedgdh03
++ echo DB_NAME=airflow
++ awk -F: '{print $1}'
++ echo DB_NAME=airflow
++ sed -e 's|DB_NAME=airflow:||'
++ echo DB_NAME=airflow
++ awk -F= '{print $1}'
++ echo DB_NAME=airflow
++ awk -F= '{print $2}'
++ DB_NAME=airflow
++ echo DB_PASS=p@ssw0rd123
++ awk -F: '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ sed -e 's|DB_PASS=p@ssw0rd123:||'
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $2}'
++ DB_PASS=p@ssw0rd123
++ echo DB_PORT=3306
++ awk -F: '{print $1}'
++ sed -e 's|DB_PORT=3306:||'
++ echo DB_PORT=3306
++ echo DB_PORT=3306
++ awk -F= '{print $1}'
++ echo DB_PORT=3306
++ awk -F= '{print $2}'
++ DB_PORT=3306
++ echo DB_TYPE=MySQL
++ awk -F: '{print $1}'
++ echo DB_TYPE=MySQL
++ sed -e 's|DB_TYPE=MySQL:||'
++ echo DB_TYPE=MySQL
++ awk -F= '{print $1}'
++ echo DB_TYPE=MySQL
++ awk -F= '{print $2}'
++ DB_TYPE=MySQL
++ echo DB_USER=airflow
++ awk -F: '{print $1}'
++ echo DB_USER=airflow
++ sed -e 's|DB_USER=airflow:||'
++ echo DB_USER=airflow
++ awk -F= '{print $1}'
++ echo DB_USER=airflow
++ awk -F= '{print $2}'
++ DB_USER=airflow
++ echo api:auth_backend=airflow.api.auth.backend.default
++ awk -F: '{print $1}'
++ echo api:auth_backend=airflow.api.auth.backend.default
++ sed -e 's|api:||'
++ echo auth_backend=airflow.api.auth.backend.default
++ awk -F= '{print $1}'
++ echo auth_backend=airflow.api.auth.backend.default
++ awk -F= '{print $2}'
++ echo celery:default_queue=default
++ awk -F: '{print $1}'
++ echo celery:default_queue=default
++ sed -e 's|celery:||'
++ echo default_queue=default
++ awk -F= '{print $1}'
++ echo default_queue=default
++ awk -F= '{print $2}'
++ echo celery:worker_concurrency=16
++ awk -F: '{print $1}'
++ echo celery:worker_concurrency=16
++ sed -e 's|celery:||'
++ echo worker_concurrency=16
++ awk -F= '{print $1}'
++ echo worker_concurrency=16
++ awk -F= '{print $2}'
++ echo celery:worker_log_server_port=8793
++ awk -F: '{print $1}'
++ echo celery:worker_log_server_port=8793
++ sed -e 's|celery:||'
++ echo worker_log_server_port=8793
++ awk -F= '{print $1}'
++ echo worker_log_server_port=8793
++ awk -F= '{print $2}'
++ echo cli:api_client=airflow.api.client.local_client
++ awk -F: '{print $1}'
++ sed -e 's|cli:||'
++ echo cli:api_client=airflow.api.client.local_client
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $1}'
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $2}'
++ echo cli:endpoint_url=http://localhost:8080
++ awk -F: '{print $1}'
++ echo cli:endpoint_url=http://localhost:8080
++ sed -e 's|cli:||'
++ awk -F= '{print $1}'
++ echo endpoint_url=http://localhost:8080
++ echo endpoint_url=http://localhost:8080
++ awk -F= '{print $2}'
++ echo core:base_log_folder=/var/log/airflow
++ awk -F: '{print $1}'
++ echo core:base_log_folder=/var/log/airflow
++ sed -e 's|core:||'
++ echo base_log_folder=/var/log/airflow
++ awk -F= '{print $1}'
++ echo base_log_folder=/var/log/airflow
++ awk -F= '{print $2}'
++ echo core:dag_concurrency=4
++ awk -F: '{print $1}'
++ echo core:dag_concurrency=4
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo dag_concurrency=4
++ echo dag_concurrency=4
++ awk -F= '{print $2}'
++ echo core:dagbag_import_timeout=30
++ awk -F: '{print $1}'
++ echo core:dagbag_import_timeout=30
++ sed -e 's|core:||'
++ echo dagbag_import_timeout=30
++ awk -F= '{print $1}'
++ echo dagbag_import_timeout=30
++ awk -F= '{print $2}'
++ echo core:dags_are_paused_at_creation=true
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:dags_are_paused_at_creation=true
++ echo dags_are_paused_at_creation=true
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo dags_are_paused_at_creation=true
++ echo core:dags_folder=/var/lib/airflow/dags
++ awk -F: '{print $1}'
++ echo core:dags_folder=/var/lib/airflow/dags
++ sed -e 's|core:||'
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $1}'
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $2}'
++ echo core:default_impersonation=
++ awk -F: '{print $1}'
++ echo core:default_impersonation=
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo default_impersonation=
++ awk -F= '{print $2}'
++ echo default_impersonation=
++ awk -F: '{print $1}'
++ echo core:donot_pickle=false
++ echo core:donot_pickle=false
++ sed -e 's|core:||'
++ echo donot_pickle=false
++ awk -F= '{print $1}'
++ echo donot_pickle=false
++ awk -F= '{print $2}'
++ echo core:executor=LocalExecutor
++ awk -F: '{print $1}'
++ echo core:executor=LocalExecutor
++ sed -e 's|core:||'
++ echo executor=LocalExecutor
++ awk -F= '{print $1}'
++ echo executor=LocalExecutor
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:fernet_key=
++ echo core:fernet_key=
++ sed -e 's|core:||'
++ echo fernet_key=
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo fernet_key=
++ echo core:load_examples=false
++ awk -F: '{print $1}'
++ echo core:load_examples=false
++ sed -e 's|core:||'
++ echo load_examples=false
++ awk -F= '{print $1}'
++ echo load_examples=false
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:max_active_runs_per_dag=16
++ echo core:max_active_runs_per_dag=16
++ sed -e 's|core:||'
++ echo max_active_runs_per_dag=16
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo max_active_runs_per_dag=16
++ echo core:non_pooled_task_slot_count=128
++ awk -F: '{print $1}'
++ echo core:non_pooled_task_slot_count=128
++ sed -e 's|core:||'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $1}'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:parallelism=8
++ echo core:parallelism=8
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo parallelism=8
++ echo parallelism=8
++ awk -F= '{print $2}'
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ awk -F: '{print $1}'
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo plugins_folder=/var/lib/airflow/plugins
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_pool_enabled=true
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:sql_alchemy_pool_enabled=true
++ echo sql_alchemy_pool_enabled=true
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_enabled=true
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_pool_recycle=1800
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_recycle=1800
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_recycle=1800
++ echo sql_alchemy_pool_recycle=1800
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_pool_size=5
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_size=5
++ sed -e 's|core:||'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $2}'
++ echo core:sql_alchemy_reconnect_timeout=300
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:sql_alchemy_reconnect_timeout=300
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $1}'
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:task_runner=BashTaskRunner
++ echo core:task_runner=BashTaskRunner
++ sed -e 's|core:||'
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $1}'
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo hive:default_hive_mapred_queue=
++ echo hive:default_hive_mapred_queue=
++ sed -e 's|hive:||'
++ echo default_hive_mapred_queue=
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo default_hive_mapred_queue=
++ awk -F: '{print $1}'
++ echo log_dir=/var/log/airflow
++ echo log_dir=/var/log/airflow
++ sed -e 's|log_dir=/var/log/airflow:||'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo log_dir=/var/log/airflow
++ log_dir=/var/log/airflow
++ echo operators:default_cpus=1
++ awk -F: '{print $1}'
++ echo operators:default_cpus=1
++ sed -e 's|operators:||'
++ awk -F= '{print $1}'
++ echo default_cpus=1
++ echo default_cpus=1
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo operators:default_disk=512
++ echo operators:default_disk=512
++ sed -e 's|operators:||'
++ awk -F= '{print $1}'
++ echo default_disk=512
++ echo default_disk=512
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo operators:default_gpus=0
++ sed -e 's|operators:||'
++ echo operators:default_gpus=0
++ awk -F= '{print $1}'
++ echo default_gpus=0
++ awk -F= '{print $2}'
++ echo default_gpus=0
++ echo operators:default_owner=Airflow
++ awk -F: '{print $1}'
++ sed -e 's|operators:||'
++ echo operators:default_owner=Airflow
++ echo default_owner=Airflow
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo default_owner=Airflow
++ echo operators:default_ram=512
++ awk -F: '{print $1}'
++ echo operators:default_ram=512
++ sed -e 's|operators:||'
++ echo default_ram=512
++ awk -F= '{print $1}'
++ echo default_ram=512
++ awk -F= '{print $2}'
++ echo smtp:smtp_host=localhost
++ awk -F: '{print $1}'
++ sed -e 's|smtp:||'
++ echo smtp:smtp_host=localhost
++ awk -F= '{print $1}'
++ echo smtp_host=localhost
++ awk -F= '{print $2}'
++ echo smtp_host=localhost
++ echo smtp:smtp_mail_from=airflow@localhost
++ awk -F: '{print $1}'
++ sed -e 's|smtp:||'
++ echo smtp:smtp_mail_from=airflow@localhost
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $1}'
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $2}'
++ echo smtp:smtp_password=
++ awk -F: '{print $1}'
++ echo smtp:smtp_password=
++ sed -e 's|smtp:||'
++ echo smtp_password=
++ awk -F= '{print $1}'
++ echo smtp_password=
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo smtp:smtp_port=25
++ echo smtp:smtp_port=25
++ sed -e 's|smtp:||'
++ awk -F= '{print $1}'
++ echo smtp_port=25
++ awk -F= '{print $2}'
++ echo smtp_port=25
++ echo smtp:smtp_ssl=false
++ awk -F: '{print $1}'
++ echo smtp:smtp_ssl=false
++ sed -e 's|smtp:||'
++ echo smtp_ssl=false
++ awk -F= '{print $1}'
++ echo smtp_ssl=false
++ awk -F= '{print $2}'
++ echo smtp:smtp_starttls=true
++ awk -F: '{print $1}'
++ echo smtp:smtp_starttls=true
++ sed -e 's|smtp:||'
++ awk -F= '{print $1}'
++ echo smtp_starttls=true
++ echo smtp_starttls=true
++ awk -F= '{print $2}'
++ echo smtp:smtp_user=
++ awk -F: '{print $1}'
++ sed -e 's|smtp:||'
++ echo smtp:smtp_user=
++ echo smtp_user=
++ awk -F= '{print $1}'
++ echo smtp_user=
++ awk -F= '{print $2}'
++ date '+%Y-%m-%d %H:%M:%S'
/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/configuration.py:214: FutureWarning: The task_runner setting in [core] has the old default value of u'BashTaskRunner'. This value has been changed to u'StandardTaskRunner' in the running config, but please update your config before Apache Airflow 2.0.
FutureWarning,
Traceback (most recent call last):
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/.airflow", line 32, in
args.func(args)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper
return f(*args, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/bin/cli.py", line 1090, in worker
worker.run(**options)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in run
**kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 101, in init
self.setup_instance(**self.prepare_args(**kwargs))
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 122, in setup_instance
self._conninfo = self.app.connection_for_read()
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 743, in connection_for_read
return self._connection(url or self.conf.broker_read_url, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 819, in _connection
'broker_connection_timeout', connect_timeout
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/connection.py", line 189, in init
url_params = parse_url(hostname)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 47, in parse_url
scheme, host, port, user, password, path, query = _parse_url(url)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 81, in url_to_parts
parts.port,
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 162, in port
port = int(port, 10)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/types/newint.py", line 76, in new
return super(newint, cls).new(cls, val)
ValueError: invalid literal for long() with base 10: ''
[13/Jul/2020 00:13:04 +0000] 121677 MainThread redactor INFO Started launcher: /opt/cloudera/cm-agent/service/csd/csd.sh start_worker
[13/Jul/2020 00:13:04 +0000] 121677 MainThread redactor ERROR Redaction rules file doesn't exist, not redacting logs. file: redaction-rules.json, directory: /run/cloudera-scm-agent/process/12038-airflow-AIRFLOW_WORKER
[13/Jul/2020 00:13:04 +0000] 121677 MainThread redactor INFO Re-exec watcher: /opt/cloudera/cm-agent/bin/cm proc_watcher 121689
Mon Jul 13 00:13:04 EDT 2020
'
'
++ seq 1 1
seq 1 $COUNT'++ AIRFLOW_DIRNAME=AIRFLOW-1.10.3-python2.7.15_b2
++ export AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ AIRFLOW_DIR=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ PATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
++ export PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ PYTHONHOME=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2
++ export PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ PYTHONPATH=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7
++ '[' -z '' ']'
++ export AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ AIRFLOW_PYTHON=/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/python2.7
++ printf '! -name %s ' cloudera-config.sh hue.sh impala.sh sqoop.sh supervisor.conf config.zip proc.json '.log' '*.keytab' '*jceks'
Mon Jul 13 00:13:05 EDT 2020
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ date '+%Y-%m-%d %H:%M:%S'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F: '{print $1}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ sed -e 's|AIRFLOW_HOME=/var/lib/airflow:||'
++ awk -F= '{print $1}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ awk -F= '{print $2}'
++ echo AIRFLOW_HOME=/var/lib/airflow
++ AIRFLOW_HOME=/var/lib/airflow
++ awk -F: '{print $1}'
++ echo CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER=RabbitMQ
++ sed -e 's|CELERY_BROKER=RabbitMQ:||'
++ echo CELERY_BROKER=RabbitMQ
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo CELERY_BROKER=RabbitMQ
++ CELERY_BROKER=RabbitMQ
++ echo CELERY_BROKER_HOST=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_HOST=
++ sed -e 's|CELERY_BROKER_HOST=:||'
++ echo CELERY_BROKER_HOST=
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo CELERY_BROKER_HOST=
++ CELERY_BROKER_HOST=
++ echo CELERY_BROKER_PASS=
++ awk -F: '{print $1}'
++ sed -e 's|CELERY_BROKER_PASS=:||'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PASS=
++ awk -F= '{print $2}'
++ echo CELERY_BROKER_PASS=
++ CELERY_BROKER_PASS=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_PORT=
++ echo CELERY_BROKER_PORT=
++ sed -e 's|CELERY_BROKER_PORT=:||'
++ awk -F= '{print $1}'
++ echo CELERY_BROKER_PORT=
++ echo CELERY_BROKER_PORT=
++ awk -F= '{print $2}'
++ CELERY_BROKER_PORT=
++ awk -F: '{print $1}'
++ echo CELERY_BROKER_USER=
++ echo CELERY_BROKER_USER=
++ sed -e 's|CELERY_BROKER_USER=:||'
++ echo CELERY_BROKER_USER=
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo CELERY_BROKER_USER=
++ CELERY_BROKER_USER=
++ awk -F: '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ sed -e 's|DB_HOST=usadc-vsedgdh03:||'
++ echo DB_HOST=usadc-vsedgdh03
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $1}'
++ echo DB_HOST=usadc-vsedgdh03
++ awk -F= '{print $2}'
++ DB_HOST=usadc-vsedgdh03
++ echo DB_NAME=airflow
++ awk -F: '{print $1}'
++ sed -e 's|DB_NAME=airflow:||'
++ echo DB_NAME=airflow
++ echo DB_NAME=airflow
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo DB_NAME=airflow
++ DB_NAME=airflow
++ awk -F: '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ echo DB_PASS=p@ssw0rd123
++ sed -e 's|DB_PASS=p@ssw0rd123:||'
++ awk -F= '{print $1}'
++ echo DB_PASS=p@ssw0rd123
++ awk -F= '{print $2}'
++ echo DB_PASS=p@ssw0rd123
++ DB_PASS=p@ssw0rd123
++ echo DB_PORT=3306
++ awk -F: '{print $1}'
++ sed -e 's|DB_PORT=3306:||'
++ echo DB_PORT=3306
++ echo DB_PORT=3306
++ awk -F= '{print $1}'
++ echo DB_PORT=3306
++ awk -F= '{print $2}'
++ DB_PORT=3306
++ echo DB_TYPE=MySQL
++ awk -F: '{print $1}'
++ echo DB_TYPE=MySQL
++ sed -e 's|DB_TYPE=MySQL:||'
++ echo DB_TYPE=MySQL
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo DB_TYPE=MySQL
++ DB_TYPE=MySQL
++ echo DB_USER=airflow
++ awk -F: '{print $1}'
++ sed -e 's|DB_USER=airflow:||'
++ echo DB_USER=airflow
++ echo DB_USER=airflow
++ awk -F= '{print $1}'
++ echo DB_USER=airflow
++ awk -F= '{print $2}'
++ DB_USER=airflow
++ echo api:auth_backend=airflow.api.auth.backend.default
++ awk -F: '{print $1}'
++ echo api:auth_backend=airflow.api.auth.backend.default
++ sed -e 's|api:||'
++ awk -F= '{print $1}'
++ echo auth_backend=airflow.api.auth.backend.default
++ echo auth_backend=airflow.api.auth.backend.default
++ awk -F= '{print $2}'
++ echo celery:default_queue=default
++ awk -F: '{print $1}'
++ sed -e 's|celery:||'
++ echo celery:default_queue=default
++ echo default_queue=default
++ awk -F= '{print $1}'
++ echo default_queue=default
++ awk -F= '{print $2}'
++ echo celery:worker_concurrency=16
++ awk -F: '{print $1}'
++ echo celery:worker_concurrency=16
++ sed -e 's|celery:||'
++ awk -F= '{print $1}'
++ echo worker_concurrency=16
++ awk -F= '{print $2}'
++ echo worker_concurrency=16
++ awk -F: '{print $1}'
++ echo celery:worker_log_server_port=8793
++ echo celery:worker_log_server_port=8793
++ sed -e 's|celery:||'
++ echo worker_log_server_port=8793
++ awk -F= '{print $1}'
++ echo worker_log_server_port=8793
++ awk -F= '{print $2}'
++ echo cli:api_client=airflow.api.client.local_client
++ awk -F: '{print $1}'
++ echo cli:api_client=airflow.api.client.local_client
++ sed -e 's|cli:||'
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $1}'
++ echo api_client=airflow.api.client.local_client
++ awk -F= '{print $2}'
++ echo cli:endpoint_url=http://localhost:8080
++ awk -F: '{print $1}'
++ echo cli:endpoint_url=http://localhost:8080
++ sed -e 's|cli:||'
++ awk -F= '{print $1}'
++ echo endpoint_url=http://localhost:8080
++ echo endpoint_url=http://localhost:8080
++ awk -F= '{print $2}'
++ echo core:base_log_folder=/var/log/airflow
++ awk -F: '{print $1}'
++ echo core:base_log_folder=/var/log/airflow
++ sed -e 's|core:||'
++ echo base_log_folder=/var/log/airflow
++ awk -F= '{print $1}'
++ echo base_log_folder=/var/log/airflow
++ awk -F= '{print $2}'
++ echo core:dag_concurrency=4
++ awk -F: '{print $1}'
++ echo core:dag_concurrency=4
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo dag_concurrency=4
++ awk -F= '{print $2}'
++ echo dag_concurrency=4
++ awk -F: '{print $1}'
++ echo core:dagbag_import_timeout=30
++ sed -e 's|core:||'
++ echo core:dagbag_import_timeout=30
++ awk -F= '{print $1}'
++ echo dagbag_import_timeout=30
++ echo dagbag_import_timeout=30
++ awk -F= '{print $2}'
++ echo core:dags_are_paused_at_creation=true
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:dags_are_paused_at_creation=true
++ awk -F= '{print $1}'
++ echo dags_are_paused_at_creation=true
++ awk -F= '{print $2}'
++ echo dags_are_paused_at_creation=true
++ awk -F: '{print $1}'
++ echo core:dags_folder=/var/lib/airflow/dags
++ echo core:dags_folder=/var/lib/airflow/dags
++ sed -e 's|core:||'
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $1}'
++ echo dags_folder=/var/lib/airflow/dags
++ awk -F= '{print $2}'
++ echo core:default_impersonation=
++ awk -F: '{print $1}'
++ echo core:default_impersonation=
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo default_impersonation=
++ awk -F= '{print $2}'
++ echo default_impersonation=
++ echo core:donot_pickle=false
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:donot_pickle=false
++ awk -F= '{print $1}'
++ echo donot_pickle=false
++ echo donot_pickle=false
++ awk -F= '{print $2}'
++ echo core:executor=LocalExecutor
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:executor=LocalExecutor
++ awk -F= '{print $1}'
++ echo executor=LocalExecutor
++ awk -F= '{print $2}'
++ echo executor=LocalExecutor
++ awk -F: '{print $1}'
++ echo core:fernet_key=
++ echo core:fernet_key=
++ sed -e 's|core:||'
++ echo fernet_key=
++ awk -F= '{print $1}'
++ echo fernet_key=
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:load_examples=false
++ sed -e 's|core:||'
++ echo core:load_examples=false
++ awk -F= '{print $1}'
++ echo load_examples=false
++ awk -F= '{print $2}'
++ echo load_examples=false
++ echo core:max_active_runs_per_dag=16
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:max_active_runs_per_dag=16
++ echo max_active_runs_per_dag=16
++ awk -F= '{print $1}'
++ echo max_active_runs_per_dag=16
++ awk -F= '{print $2}'
++ echo core:non_pooled_task_slot_count=128
++ awk -F: '{print $1}'
++ echo core:non_pooled_task_slot_count=128
++ sed -e 's|core:||'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $1}'
++ echo non_pooled_task_slot_count=128
++ awk -F= '{print $2}'
++ echo core:parallelism=8
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:parallelism=8
++ awk -F= '{print $1}'
++ echo parallelism=8
++ awk -F= '{print $2}'
++ echo parallelism=8
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ awk -F: '{print $1}'
++ echo core:plugins_folder=/var/lib/airflow/plugins
++ sed -e 's|core:||'
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $1}'
++ echo plugins_folder=/var/lib/airflow/plugins
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_enabled=true
++ echo core:sql_alchemy_pool_enabled=true
++ sed -e 's|core:||'
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_enabled=true
++ awk -F= '{print $2}'
++ echo sql_alchemy_pool_enabled=true
++ echo core:sql_alchemy_pool_recycle=1800
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:sql_alchemy_pool_recycle=1800
++ echo sql_alchemy_pool_recycle=1800
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo sql_alchemy_pool_recycle=1800
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_pool_size=5
++ echo core:sql_alchemy_pool_size=5
++ sed -e 's|core:||'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $1}'
++ echo sql_alchemy_pool_size=5
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo core:sql_alchemy_reconnect_timeout=300
++ echo core:sql_alchemy_reconnect_timeout=300
++ sed -e 's|core:||'
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $1}'
++ echo sql_alchemy_reconnect_timeout=300
++ awk -F= '{print $2}'
++ echo core:task_runner=BashTaskRunner
++ awk -F: '{print $1}'
++ sed -e 's|core:||'
++ echo core:task_runner=BashTaskRunner
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $1}'
++ echo task_runner=BashTaskRunner
++ awk -F= '{print $2}'
++ echo hive:default_hive_mapred_queue=
++ awk -F: '{print $1}'
++ echo hive:default_hive_mapred_queue=
++ sed -e 's|hive:||'
++ echo default_hive_mapred_queue=
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo default_hive_mapred_queue=
++ echo log_dir=/var/log/airflow
++ awk -F: '{print $1}'
++ echo log_dir=/var/log/airflow
++ sed -e 's|log_dir=/var/log/airflow:||'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $1}'
++ echo log_dir=/var/log/airflow
++ awk -F= '{print $2}'
++ log_dir=/var/log/airflow
++ echo operators:default_cpus=1
++ awk -F: '{print $1}'
++ echo operators:default_cpus=1
++ sed -e 's|operators:||'
++ echo default_cpus=1
++ awk -F= '{print $1}'
++ echo default_cpus=1
++ awk -F= '{print $2}'
++ echo operators:default_disk=512
++ awk -F: '{print $1}'
++ sed -e 's|operators:||'
++ echo operators:default_disk=512
++ awk -F= '{print $1}'
++ echo default_disk=512
++ awk -F= '{print $2}'
++ echo default_disk=512
++ echo operators:default_gpus=0
++ awk -F: '{print $1}'
++ echo operators:default_gpus=0
++ sed -e 's|operators:||'
++ echo default_gpus=0
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo default_gpus=0
++ awk -F: '{print $1}'
++ echo operators:default_owner=Airflow
++ echo operators:default_owner=Airflow
++ sed -e 's|operators:||'
++ awk -F= '{print $1}'
++ echo default_owner=Airflow
++ echo default_owner=Airflow
++ awk -F= '{print $2}'
++ echo operators:default_ram=512
++ awk -F: '{print $1}'
++ echo operators:default_ram=512
++ sed -e 's|operators:||'
++ echo default_ram=512
++ awk -F= '{print $1}'
++ echo default_ram=512
++ awk -F= '{print $2}'
++ echo smtp:smtp_host=localhost
++ awk -F: '{print $1}'
++ echo smtp:smtp_host=localhost
++ sed -e 's|smtp:||'
++ echo smtp_host=localhost
++ awk -F= '{print $1}'
++ echo smtp_host=localhost
++ awk -F= '{print $2}'
++ echo smtp:smtp_mail_from=airflow@localhost
++ awk -F: '{print $1}'
++ echo smtp:smtp_mail_from=airflow@localhost
++ sed -e 's|smtp:||'
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $1}'
++ echo smtp_mail_from=airflow@localhost
++ awk -F= '{print $2}'
++ awk -F: '{print $1}'
++ echo smtp:smtp_password=
++ sed -e 's|smtp:||'
++ echo smtp:smtp_password=
++ echo smtp_password=
++ awk -F= '{print $1}'
++ echo smtp_password=
++ awk -F= '{print $2}'
++ echo smtp:smtp_port=25
++ awk -F: '{print $1}'
++ echo smtp:smtp_port=25
++ sed -e 's|smtp:||'
++ echo smtp_port=25
++ awk -F= '{print $1}'
++ awk -F= '{print $2}'
++ echo smtp_port=25
++ awk -F: '{print $1}'
++ echo smtp:smtp_ssl=false
++ sed -e 's|smtp:||'
++ echo smtp:smtp_ssl=false
++ echo smtp_ssl=false
++ awk -F= '{print $1}'
++ echo smtp_ssl=false
++ awk -F= '{print $2}'
++ echo smtp:smtp_starttls=true
++ awk -F: '{print $1}'
++ echo smtp:smtp_starttls=true
++ sed -e 's|smtp:||'
++ echo smtp_starttls=true
++ awk -F= '{print $1}'
++ echo smtp_starttls=true
++ awk -F= '{print $2}'
++ echo smtp:smtp_user=
++ awk -F: '{print $1}'
++ echo smtp:smtp_user=
++ sed -e 's|smtp:||'
++ awk -F= '{print $1}'
++ echo smtp_user=
++ echo smtp_user=
++ awk -F= '{print $2}'
++ date '+%Y-%m-%d %H:%M:%S'
/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/configuration.py:214: FutureWarning: The task_runner setting in [core] has the old default value of u'BashTaskRunner'. This value has been changed to u'StandardTaskRunner' in the running config, but please update your config before Apache Airflow 2.0.
FutureWarning,
Traceback (most recent call last):
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/bin/.airflow", line 32, in
args.func(args)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/utils/cli.py", line 74, in wrapper
return f(*args, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/airflow/bin/cli.py", line 1090, in worker
worker.run(**options)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/bin/worker.py", line 257, in run
**kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 101, in init
self.setup_instance(**self.prepare_args(**kwargs))
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/worker/worker.py", line 122, in setup_instance
self._conninfo = self.app.connection_for_read()
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 743, in connection_for_read
return self._connection(url or self.conf.broker_read_url, **kwargs)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/celery/app/base.py", line 819, in _connection
'broker_connection_timeout', connect_timeout
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/connection.py", line 189, in init
url_params = parse_url(hostname)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 47, in parse_url
scheme, host, port, user, password, path, query = _parse_url(url)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/kombu/utils/url.py", line 81, in url_to_parts
parts.port,
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/backports/urllib/parse.py", line 162, in port
port = int(port, 10)
File "/opt/cloudera/parcels/AIRFLOW-1.10.3-python2.7.15_b2/lib/python2.7/site-packages/future/types/newint.py", line 76, in new
return super(newint, cls).new(cls, val)
ValueError: invalid literal for long() with base 10: ''
The text was updated successfully, but these errors were encountered: