Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add extract_errors to Postgres #13997

Merged
merged 4 commits into from
Apr 8, 2021

Conversation

betodealmeida
Copy link
Member

SUMMARY

This PR adds a new error TEST_CONNECTION_INVALID_USERNAME_ERROR, for when the username provided to connect to a given database is invalid.

It also removes the _diagnose method, which tests if the host is up or down, leveraging extract_errors for that instead. I implemented the functionality for Postgres, and will implement in other popular engines in a next PR.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TEST PLAN

Added unit tests.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@betodealmeida
Copy link
Member Author

@etr2460, I did some small changes to extract_errors so it works better with SIP-40, I wanted to make sure you saw this. I refactored it to return List[SupersetError] instead of a list of dicts, and I also modified the method in the base class to make it easier to define new custom errors for specific databases (like we have for Presto currently).

@@ -1133,54 +1133,41 @@ def get_function_names(cls, database: "Database") -> List[str]:
return database.get_df("SHOW FUNCTIONS")["Function"].tolist()

@classmethod
def extract_errors(cls, ex: Exception) -> List[Dict[str, Any]]:
def extract_errors(cls, ex: Exception) -> List[SupersetError]:
raw_message = cls._extract_error_message(ex)

column_match = re.search(COLUMN_NOT_RESOLVED_ERROR_REGEX, raw_message)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can implement this error and TABLE_DOES_NOT_EXIST_ERROR_REGEX in custom_errors, and get rid of the extract_errors method here, like I did for Postgres. I didn't want to change it in this PR to keep it smaller.

@codecov
Copy link

codecov bot commented Apr 7, 2021

Codecov Report

Merging #13997 (eb94584) into master (f06d534) will increase coverage by 0.05%.
The diff coverage is 96.55%.

❗ Current head eb94584 differs from pull request most recent head 0eb17e5. Consider uploading reports for the commit 0eb17e5 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master   #13997      +/-   ##
==========================================
+ Coverage   79.21%   79.26%   +0.05%     
==========================================
  Files         936      936              
  Lines       47408    47405       -3     
  Branches     5940     5938       -2     
==========================================
+ Hits        37554    37577      +23     
+ Misses       9727     9702      -25     
+ Partials      127      126       -1     
Flag Coverage Δ
hive 80.24% <96.29%> (-0.04%) ⬇️
javascript 68.85% <100.00%> (+0.40%) ⬆️
mysql 80.51% <92.59%> (-0.07%) ⬇️
postgres 80.54% <92.59%> (-0.07%) ⬇️
presto 80.26% <96.29%> (-0.04%) ⬇️
python 81.12% <96.29%> (-0.03%) ⬇️
sqlite 80.14% <92.59%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...rset-frontend/src/components/ErrorMessage/types.ts 100.00% <ø> (ø)
...end/src/components/IndeterminateCheckbox/index.tsx 100.00% <ø> (ø)
...ntend/src/dashboard/components/CssEditor/index.jsx 95.83% <ø> (ø)
...nd/src/explore/components/DataTablesPane/index.tsx 88.88% <ø> (ø)
...ols/MetricControl/AdhocMetricEditPopover/index.jsx 84.09% <ø> (ø)
superset/connectors/sqla/models.py 90.77% <50.00%> (+0.01%) ⬆️
...et-frontend/src/components/TableView/TableView.tsx 98.21% <100.00%> (+1.78%) ⬆️
superset/databases/commands/exceptions.py 93.02% <100.00%> (-0.32%) ⬇️
superset/databases/commands/test_connection.py 98.00% <100.00%> (-0.44%) ⬇️
superset/db_engine_specs/base.py 87.44% <100.00%> (+0.36%) ⬆️
... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f06d534...0eb17e5. Read the comment docs.

@betodealmeida betodealmeida force-pushed the ch6434a branch 2 times, most recently from 10090ee to 03f524a Compare April 8, 2021 00:45

self.login("admin")
data = {
"sqlalchemy_uri": "postgres://username:password@localhost:12345/db",
"sqlalchemy_uri": "postgres://username:password@locahost:12345/db",
Copy link
Member

@hughhhh hughhhh Apr 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"sqlalchemy_uri": "postgres://username:password@locahost:12345/db",
"sqlalchemy_uri": "postgres://username:password@badhost:12345/db",

@mock.patch("superset.databases.commands.test_connection.is_hostname_valid")
@mock.patch("superset.databases.commands.test_connection.is_port_open")
@mock.patch("superset.databases.commands.test_connection.is_host_up")
def test_test_connection_failed_closed_port(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we still want to keep this test for the port being closed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't, we don't test this explicitly anymore, but instead rely on the driver to give this information. Because of this we only need one API test now.

@hughhhh hughhhh self-requested a review April 8, 2021 16:51
@betodealmeida betodealmeida merged commit c60a93d into apache:master Apr 8, 2021
allanco91 pushed a commit to allanco91/superset that referenced this pull request May 21, 2021
* feat: add extract_errors to Postgres

* Add unit tests

* Fix lint

* Fix unit tests
QAlexBall pushed a commit to QAlexBall/superset that referenced this pull request Dec 29, 2021
* feat: add extract_errors to Postgres

* Add unit tests

* Fix lint

* Fix unit tests
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.2.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 1.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants