Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

superset db upgrade not working due to removal of FLAGS variable in sqlparse/kewords.py #23742

Closed
LeonardSchuler opened this issue Apr 19, 2023 · 19 comments · Fixed by #24045
Closed
Labels
#bug Bug report

Comments

@LeonardSchuler
Copy link

LeonardSchuler commented Apr 19, 2023

In superset/sql_parse.py sqlparse.keywords.FLAGS is used. However, sqlparse has removed the FLAGS variable from the code sqlparse/kewords.py.
This leads to an AttributeError when running the CLI command
$ superset db upgrade
AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'
I would recommend replacing the FLAGS dependency with the actual value
re.IGNORECASE | re.UNICODE).match
or fix the sqlparse version to "sqlparse==0.4.3" (not recommended as they fixed a security problem recently).

I have experienced this problem under python3.9 and python3.11

@LeonardSchuler LeonardSchuler added the #bug Bug report label Apr 19, 2023
@LeonardSchuler LeonardSchuler changed the title superset db upgrade not working due to removal of FLAGS variable in sqlparse/kewords.yp superset db upgrade not working due to removal of FLAGS variable in sqlparse/kewords.py Apr 19, 2023
@Tafkas
Copy link

Tafkas commented Apr 20, 2023

I have the same issue using Python 3.8.16.

@esevato
Copy link

esevato commented Apr 20, 2023

Clean install Debian 11 fails @ "superset db upgrade" with
AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'

@amilcarnetto
Copy link

downgrading to sqlparse 0.4.3 solve the problem for me

@MoronGamer
Copy link

downgrading to sqlparse 0.4.3 solve the problem for me

Having same error, how do you downgrade sqlparse?

@HoncyLee
Copy link

downgrading to sqlparse 0.4.3 solve the problem for me

Having same error, how do you downgrade sqlparse?

pip install sqlparse=='0.4.3'

@Akshay577
Copy link

Yes, downgrading the sqlparse version to 0.4.3 fixed the issue for me.

@anishone
Copy link

Downgrading with pip install sqlparse=='0.4.3' works. Thanks!

@SUGAM-ARORA
Copy link

downgrading to sqlparse 0.4.3 solve the problem for me

It didn't work for me .

@rohitkumar0613
Copy link

Same didn't work for me

@christianxiaobai
Copy link

Downgrading with pip install sqlparse=='0.4.3' works. Thanks!

@kits-ragh
Copy link

After downgrading sqlparse to 0.4.3 getting following error.

Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

@pierrelafortune
Copy link

+1 to the same FLASK APP error as @kits-ragh

Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

@prb219
Copy link

prb219 commented Jul 13, 2023

@LeonardSchuler Thanks for the suggestions. I tried your suggested changes on the following setup but it fails
Superset version: 2.1.0
Python version: 3.8
SQLparse version: 0.4.4

No issue is observed during the db upgrade process. However, when we try to load the dashboards we see this error
`TypeError: first argument must be string or compiled pattern
2023-07-13 21:03:58,523:ERROR:superset.views.base:first argument must be string or compiled pattern
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1517, in full_dispatch_request
rv = self.dispatch_request()
.
.
.
.
File "/usr/local/lib/python3.8/site-packages/sqlparse/engine/filter_stack.py", line 26, in run
stream = lexer.tokenize(sql, encoding)
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 155, in tokenize
return Lexer.get_default_instance().get_tokens(sql, encoding)
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 52, in get_default_instance
cls._default_intance.default_initialization()
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 59, in default_initialization
self.set_SQL_REGEX(keywords.SQL_REGEX)
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 78, in set_SQL_REGEX
self._SQL_REGEX = [
File "/usr/local/lib/python3.8/site-packages/sqlparse/lexer.py", line 79, in
(re.compile(rx, FLAGS).match, tt)
File "/usr/lib64/python3.8/re.py", line 252, in compile
return _compile(pattern, flags)
File "/usr/lib64/python3.8/re.py", line 303, in _compile
raise TypeError("first argument must be string or compiled pattern")
TypeError: first argument must be string or compiled pattern

`

@enoxos
Copy link

enoxos commented Aug 9, 2023

If you downgraded to sqlparser=0.4.3 and you get the error:

_Error: Could not locate a Flask application. You did not provide the "FLASK_APP" environment variable, and a "wsgi.py" or "app.py" module was not found in the current directory.

then set the FLASK_APP as follows

FLASK_APP=superset

and then rerun the db initialization step.

@mdanishu
Copy link

I'm still getting this flags problem... why is this happening and how can we fix it? downgrading to lower versions didn't fix it

@WaluQ-iu
Copy link

WaluQ-iu commented Sep 7, 2023

I'm still getting this flags problem... why is this happening and how can we fix it? downgrading to lower versions didn't fix it

Try export FLASK_APP=superset and then rerun

@helpwise
Copy link

helpwise commented Sep 14, 2023

/home/ubuntu/.local/bin# sudo -u ubuntu FLASK_APP=superset /home/ubuntu/.local/bin/superset db upgrade

Traceback (most recent call last):
File "/home/ubuntu/.local/bin/superset", line 5, in
from superset.cli.main import superset
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/init.py", line 21, in
from superset.app import create_app
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/app.py", line 23, in
from superset.initialization import SupersetAppInitializer
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/initialization/init.py", line 33, in
from superset.extensions import (
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/extensions/init.py", line 32, in
from superset.utils.async_query_manager import AsyncQueryManager
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/async_query_manager.py", line 26, in
from superset.utils.core import get_user_id
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/core.py", line 106, in
from superset.sql_parse import sanitize_clause
File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/sql_parse.py", line 67, in
re.compile(r"'(''|\\|\|[^'])*'", sqlparse.keywords.FLAGS).match,
AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'

FLASK_APP=superset doesnt work for me either..

I am running Ubuntu 20.04 in Proxmox

@WaluQ-iu
Copy link

/home/ubuntu/.local/bin# sudo -u ubuntu FLASK_APP=superset /home/ubuntu/.local/bin/superset db upgrade

Traceback (most recent call last): File "/home/ubuntu/.local/bin/superset", line 5, in from superset.cli.main import superset File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/init.py", line 21, in from superset.app import create_app File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/app.py", line 23, in from superset.initialization import SupersetAppInitializer File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/initialization/init.py", line 33, in from superset.extensions import ( File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/extensions/init.py", line 32, in from superset.utils.async_query_manager import AsyncQueryManager File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/async_query_manager.py", line 26, in from superset.utils.core import get_user_id File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/utils/core.py", line 106, in from superset.sql_parse import sanitize_clause File "/home/ubuntu/.local/lib/python3.10/site-packages/superset/sql_parse.py", line 67, in re.compile(r"'(''|\||[^'])*'", sqlparse.keywords.FLAGS).match, AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS'

AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS' suggests a compatibility issue or version mismatch with the libraries or modules being used by Superset. Specifically, it mentions an AttributeError related to the FLAGS attribute in the sqlparse.keywords module. Comments above state that you need to downgrade sqlparse to a lower version, check comments above.

@EJ-Humn
Copy link

EJ-Humn commented Nov 8, 2023

In superset/sql_parse.py sqlparse.keywords.FLAGS is used. However, sqlparse has removed the FLAGS variable from the code sqlparse/kewords.py. This leads to an AttributeError when running the CLI command $ superset db upgrade AttributeError: module 'sqlparse.keywords' has no attribute 'FLAGS' I would recommend replacing the FLAGS dependency with the actual value re.IGNORECASE | re.UNICODE).match or fix the sqlparse version to "sqlparse==0.4.3" (not recommended as they fixed a security problem recently).

I have experienced this problem under python3.9 and python3.11

This worked for me:

re.compile(r"'(''|\\\\|\\|[^'])*'", re.IGNORECASE | re.UNICODE).match,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
#bug Bug report
Projects
None yet