Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

superset cache warmup not working #10150

Closed
ukm21 opened this issue Jun 24, 2020 · 4 comments
Closed

superset cache warmup not working #10150

ukm21 opened this issue Jun 24, 2020 · 4 comments
Labels
!deprecated-label:bug Deprecated label - Use #bug instead inactive Inactive for >= 30 days

Comments

@ukm21
Copy link

ukm21 commented Jun 24, 2020

cache warmup is triggering cron job but it is not able to open the url and refresh the cache

[2020-06-24 07:58:00,166: INFO/ForkPoolWorker-1] cache-warmup[f1d59564-8816-4c98-a705-32041fa7cffe]: Fetching http://0.0.0.0:30064/superset/explore/?form_data=%7B%22slice_id%22%3A%204%7D

Expected results

cache should refresh based on dummy strategy

Actual results

cache not refreshing based on the strategy configured

Screenshots

image

Celery worker is trying to open the url of a chart but it does not trigger the query and refresh the cache

How to reproduce the bug

  1. configure dummy cache warmup strategy in superset_config.py
  2. wait for the cache warmup to trigger in celery worker node
  3. Then reload any chart, it should fetch chart from cache but instead it querys db to refresh the chart

Environment

(please complete the following information):

  • superset version: 0.36.0
  • python version: 3.6.10
@ukm21 ukm21 added the !deprecated-label:bug Deprecated label - Use #bug instead label Jun 24, 2020
@issue-label-bot
Copy link

Issue-Label Bot is automatically applying the label #bug to this issue, with a confidence of 0.98. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

@stale
Copy link

stale bot commented Aug 23, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label Aug 23, 2020
@stale stale bot closed this as completed Aug 31, 2020
@fgomezotero
Copy link
Contributor

Hello @ukm21 did you solve de problem, I've the same issue.

I'm using superset version 1.1.0

I see how celery beat raise the jobs
[2021-05-08 09:10:00,054: INFO/MainProcess] Scheduler: Sending due task cache-warmup-charts (cache-warmup)

them in celery worker was catched
[2021-05-08 08:59:59,786: INFO/MainProcess] Received task: cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9] [2021-05-08 08:59:59,787: INFO/ForkPoolWorker-1] cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9]: Loading strategy [2021-05-08 08:59:59,787: INFO/ForkPoolWorker-1] cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9]: Loading DummyStrategy [2021-05-08 08:59:59,787: INFO/ForkPoolWorker-1] cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9]: Success! [2021-05-08 08:59:59,817: INFO/ForkPoolWorker-1] cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9]: Fetching http://superset-test.superset-test.svc.cluster.local:8080/superset/explore/?form_da ta=%7B%22slice_id%22%3A%205%7D [2021-05-08 09:00:00,012: INFO/ForkPoolWorker-1] cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9]: Fetching http://superset-test.superset-test.svc.cluster.local:8080/superset/explore/?form_da ta=%7B%22slice_id%22%3A%203%7D [2021-05-08 09:00:00,295: INFO/ForkPoolWorker-1] cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9]: Fetching http://superset-test.superset-test.svc.cluster.local:8080/superset/explore/?form_da ta=%7B%22slice_id%22%3A%204%7D [2021-05-08 09:00:00,490: INFO/ForkPoolWorker-1] Task cache-warmup[6eb71805-f8b8-4d2a-95dd-4d03396268b9] succeeded in 0.7029110584408045s: {'success': ['http://superset-test.superset-test.svc.c luster.local:8080/superset/explore/?form_data=%7B%22slice_id%22%3A%205%7D', 'http://superset-test.superset-test.svc.cluster.local:8080/superset/explore/?form_data=%7B%22slice_id%22%3A%203%7D', 'http://superset-test.superset-test.svc.cluster.local:8080/superset/explore/?form_data=%7B%22slice_id%22%3A%204%7D'], 'errors': []}
But i never saw the celery worker run the queries. I''ve "GLOBAL_ASYNC_QUERIES": True.

Regards

@ukm21
Copy link
Author

ukm21 commented Jun 29, 2021

@fgomezotero cache warmup is using explore url, but explore url will only open the explore window for the chart in the browser and not do warmup. For cache warmup the url should be
http://superset-test.superset-test.svc.cluster.local:8080/superset/warm_up_cache/?slice_id=1

change the get_url() method in superset/tasks/cache.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
!deprecated-label:bug Deprecated label - Use #bug instead inactive Inactive for >= 30 days
Projects
None yet
Development

No branches or pull requests

2 participants