SqlaTable query hangs on list tables API call with large number of tables #24343
Closed
3 tasks done
Labels
#bug
Bug report
An inefficient
SqlaTable
query causes the table list endpoint to hang when you have a large number of tables in the schema.How to reproduce the bug
Expected results
Efficient return of table list from this endpoint
Actual results
API call hangs
Environment
(please complete the following information):
2.1.0
Checklist
Make sure to follow these steps before submitting your issue - thank you!
Additional context
I've gotten the bottom of this issue and will raise a PR. Essentially the call to
SqlaTable
to retrieve associated table info and populate theextra
field is extremely inefficient with a large number of queries, and that's because of the eager loading of thecolumns
andmetrics
relationships: the direct query being called is relatively fine even with thousands of rows, but when I inspected with echo on, sqlalchemy was performing an additional ~41 queries, including several outer joins with these additional tables and several queries likeWHERE table_id IN (...thousands of IDs)
.This can be avoided by using the
load_only
andlazyload
options on this query.The text was updated successfully, but these errors were encountered: