Cache table & database

[mkangia]

A safe change to speed up processing for dataset changes.
We should not need to find the db and then the table for every request.

Deployed on staging to test.
Might need to add tests in the code though.

[Charl1996]

This is great!

I'm curious to know if you've done some performance comparisons at (some) scale? Not necessary, just curious to see the difference in performance.

[mkangia]

Hey @Charl1996

I have not done any performance testing on this.
The inspiration comes from celery logs showing that they are running tasks in seconds (in range 5-60 seconds) at times, which isn't expected.
This will however just reduce one query to find the database but at scale even that would matter.
Just in case if we get celery monitoring on datadog, before this is released, we will able to see the difference if we have a metric for tasks running time.