Sherlock wrapper cache needs a process for invalidating old entries

[gpfrancis]

No description provided.

[tms-epcc]

14/JUL/21

• @gpfrancis explained this is currently blocked by a Sherlock issue

[thespacedoctor]

Hi @gpfrancis - I'm not sure what is being asked for here? Is this something I need to deal with?

[gpfrancis]

@thespacedoctor - The issue here is that we should not annotate alerts with cached information that is out of date because a new version of Sherlock/catalogue/crossmatch algorithm is now available. Unfortuntaely, the only information that we have available at present is when Sherlock was run and what version. With this we could use a TTL on the cache, but that is inefficient and/or slow to update, or we could use the Sherlock version and hope that nothing else is ever changed without bumping the version number as well. I spoke to @genghisken about this and he told me that there is a wider issue around this with regard to recording prevenance that is already being looked at and hopefully that will give us a solution to this problem as well. Given this we decided to wait on the outcome of that discussion before doing anything else about this.

As far as I know you don't need to do anything new that you're not already doing.

[thespacedoctor]

As of v2.2.0 Sherlock algorithm now ships alongside the code (and is therefore version controlled)

[tms-epcc]

As discussed with @gpfrancis via email this is not a priority in Lasair cycle 5 and hence is moved into the backlog.

[thespacedoctor]

I think this can be closed. Please reopen if I'm wrong @gpfrancis

[gpfrancis]

We paused work on this because it wasn't high priority, but we neither completed it nor AFAIK made a decision to abandon it. The Sherlock issue referenced - which is now fixed - was something that this depended on, but is not actually the same issue.