Request cache stopped working?

[amotl]

Problem

It looks like the request cache stopped working? When invoking grafana-wtf on play.grafana.org, the program requests data over and over again. That is especially hard on such a huge Grafana installation, where it takes almost one and a half minutes to acquire all the metadata.

export GRAFANA_URL=https://play.grafana.org
grafana-wtf info --format=yaml

Fetching dashboards in parallel with 5 concurrent request/s]

  4%|█▏                   | 67/1537 [00:03<01:26, 16.99it/s]

grafana:
  version: 11.1.0-68793
  url: https://play.grafana.org
statistics: {}
summary:
  dashboard_panels: 9299
  dashboard_annotations: 1501
  dashboard_templating: 1349

Thoughts

Is it related to the switchover to Niquests, because it does not harmonize with requests-cache well, yet?

• Update to grafana-client 4.0.0 #125

/cc @Ousret

[amotl]

Observations

On the previous version 0.18.0, everything works as expected.

git checkout 0.18.0
export GRAFANA_URL=https://play.grafana.org

Cold-cache situations are expensive, right.

time grafana-wtf info --format=yaml
1m20s

With a warm cache, on subsequent invocations, the program still needs time to load the data from the database into memory, but at least it doesn't have to go to the network again.

time grafana-wtf info --format=yaml
0m9.2s

Evaluation

This issue blocks the upcoming release, but it has no urgency.

[amotl]

Dear @Ousret,

thank you for sharing more insights about how to properly work with requests-cache at #130 (comment). Other than making it more beautiful, do you think it will help to make it work at all? We will be happy to try right away, thanks.

With kind regards,
Andreas.

[amotl]

After applying a few fixes, getting the chaos of session management right, we are back to more reasonable numbers when invoking grafana-wtf subsequently on play.grafana.org, which means the response cache works again. Thanks for your suggestions, @Ousret.

$ time grafana-wtf info --format=yaml
real	0m8.6s

[amotl]

That patch has a corresponding fix.

• Fix caching after migration to Niquests #135