Fatal Error when creating DB with hive

[LIN-Yu-Ting]

Hi, I have question on connecting superset with hive server or Apache Spark SQL.

I have initialized a Spark cluster with hive server which I can already connect with beeline as you can see in the following image.

I try to first startup a superset instance located at the same instance of spark master node. However, I got a Fatal Error message without any detail information. In the following image, you can see that connection looks good.

However, when clicking on Connect, you can see that there is a Database Creation Error.

What does it mean ?

I also attach ERROR message obtained from superset instance.

2023-05-11 22:25:08,222:ERROR:flask_appbuilder.api:Object of type bytes is not JSON serializable
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/flask_appbuilder/api/__init__.py", line 110, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/superset/views/base_api.py", line 122, in wraps
    raise ex
  File "/usr/local/lib/python3.8/dist-packages/superset/views/base_api.py", line 113, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/superset/utils/core.py", line 1586, in time_function
    response = func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/superset/utils/log.py", line 266, in wrapper
    value = f(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/superset/views/base_api.py", line 85, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/superset/databases/api.py", line 345, in post
    return self.response(201, id=new_model.id, result=item)
  File "/usr/local/lib/python3.8/dist-packages/flask_appbuilder/api/__init__.py", line 769, in response
    _ret_json = jsonify(kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask/json/__init__.py", line 302, in jsonify
    f"{dumps(data, indent=indent, separators=separators)}\n",
  File "/usr/local/lib/python3.8/dist-packages/flask/json/__init__.py", line 132, in dumps
    return _json.dumps(obj, **kwargs)
  File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/local/lib/python3.8/dist-packages/flask/json/__init__.py", line 51, in default
    return super().default(o)
  File "/usr/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable

It seems that even with Database Creation Error, there is still a Database Connection created.

However, there is error which can not recognize correctly database and table. I try to use dataset function however it only detect database-level item.

[TinTin-DXQ]

i have the same error

[Siham-IT]

Me too, I have the same problem with Apache Hive in version 2.1.0

[Siham-IT]

Finally, we solved this problem by following Usiel's solution:

Usiel/PyHive@fb692d9?diff=split

by changing those two lines in the sqlalchemy_hive.py file:

class HiveDialect(default.DefaultDialect):
name = b'hive'
driver = b'thrift'

to this:

class HiveDialect(default.DefaultDialect):
name = 'hive'
driver = 'thrift'

Alternatively, you can simply use the latest version of PyHive, which already contains these changes.

[vndroid]

Finally, we solved this problem by following Usiel's solution:

Usiel/PyHive@fb692d9?diff=split

by changing those two lines in the sqlalchemy_hive.py file:

class HiveDialect(default.DefaultDialect): name = b'hive' driver = b'thrift'

to this:

class HiveDialect(default.DefaultDialect): name = 'hive' driver = 'thrift'

Alternatively, you can simply use the latest version of PyHive, which already contains these changes.

Amazing! Thanks a lot.

[bkyryliuk]

It seems to be a duplicate of #22316
Give pyhive 0.7.0 a try, as I recall there was a contribution fixing this issue.