[BUG] V13.5.2 Cache issue: unpublished content can be reached

[adavidovic92]

Which Umbraco version are you using? (Please write the exact version, example: 10.1.0)

13.5.2

Bug summary

It was discovered that when attempting to unpublish, there is a delay during which the unpublished content remains accessible.

We have implemented a custom published/unpublished notification handler that sends a request to the client to clear its cache, followed by a new request to Umbraco to retrieve and cache the latest published content. However, due to a delay where unpublished content remains accessible, it re-caches the old content. This issue wasn’t present in the previous 13.3.2 version.

Specifics

No response

Steps to reproduce

• Create a document type.
• Create content using this document type, then click "Save and Publish".
• Implement an API endpoint to retrieve data for this content.
• Update the content data and click "Save and Publish".
• In the custom published notification handler, call the API endpoint.
• Observe that the response still returns the old data.

The video below showcases a clean Umbraco solution with a simple API endpoint and a custom published notification handler that calls this API endpoint. However, the endpoint returns outdated data instead of the latest updates:

TinyTake.by.MangoApps-29-10-2024-04-35-25.mp4

Expected result / actual result

When a custom published/unpublished notification handler is triggered the cache must be refreshed

[github-actions]

Hi there @adavidovic92!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

• We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
• If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
• We'll replicate the issue to ensure that the problem is as described.
• We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot 🤖 🙂

[bielu]

@adavidovic92 you can't rely on unpublished/published notifications, they can happen before nucache operations are finished, to get what you want working you need attach to cache refresher notification, check the whole thread here:
#13339

[adavidovic92]

@bielu @nikolajlauridsen Unfortunately, I need to reopen this issue for several reasons, specifically regarding version 13.5.2:

1. When I tried using ContentCacheRefresherNotification, it triggered before ContentPublishedNotification. This implies that the content should already be updated in the cache by the time ContentPublishedNotification fires.
2. I attempted to relocate my logic — which involves sending a request to the client to clear its cache, followed by another request to Umbraco to retrieve and cache the latest published content — from ContentPublishedNotification to ContentCacheRefresherNotification. However, it still re-caches the old content. This indicates that there is no clear notification to confirm when the Umbraco cache is fully updated, allowing the client to retrieve the latest data.
3. I created a small script to publish and unpublish an item 50 times, with a 4-second interval between each action. During these events, ContentCacheRefresherNotification sends a request to a simple Node.js server to retrieve the new data in Umbraco. However, in all 50 cases, the server received outdated content. Adding a 3-second delay on the Node.js server retrieves the latest data, but this appears to be the only workaround for now.

In contrast, with version 13.3.2, none of the above issues occur:

1. When ContentPublishedNotification fires, it successfully triggers a request to the client to retrieve the latest data, which works flawlessly.
2. Using the same script (publishing and unpublishing an item 50 times at a 4-second interval), ContentPublishedNotification consistently sends requests to the Node.js server, which receives the latest data in all cases.

In summary, after upgrading from 13.3.2 to 13.5.2, the client receives outdated data, and only by introducing a delay can the issue be resolved.

[bielu]

@adavidovic92 ContentCacheRefresherNotification can be randomly trigger before or after ContentPublishedNotification, which is documented, so you actually cant use ContentTreeChangeNotification either, but ContentCacheRefresherNotification (which is confusing as hell, but it what you need to do when using notifications). the only reliable notification is CCRN, not any related to content tree or content publishing.

[adavidovic92]

@bielu Apologies for the confusion in my previous comment. I've corrected it now — I had originally used ContentCacheRefresherNotification

[bielu]

@adavidovic92 can you maybe share simplied version of your notitication handler? as it might be case of reusing previous umbraco context :)

[adavidovic92]

public class ContentCacheRefresherNotificationHandler(ICustomClient customClient) : INotificationAsyncHandler<ContentCacheRefresherNotification>
{
    public async Task HandleAsync(ContentCacheRefresherNotification notification, CancellationToken cancellationToken)
    {
        if (notification.MessageObject is not ContentCacheRefresher.JsonPayload[] payloads)
            return;

        foreach (ContentCacheRefresher.JsonPayload payload in payloads)
        {          
            // Sends a request to a simple Node.js server to clear cache and retrieve the new data in Umbraco 
            await customClient.ClearCache();
        }

        return;
    }
}

[bielu]

@adavidovic92 do you depend on Icontent? as I noticed you use Icontent service, to get fresh IPC you would need use

using (UmbracoContextReference context = _contextFactory.EnsureUmbracoContext())
            {
                IPublishedContent? published = context.UmbracoContext.Content?.GetById(payload.Id);

                // Do stuff
            }

If you want do that in you customClient you still can use factory :) also it might be issue with scopes, you might want create separate subscope in DI to ensure you dont share context with main event context, as than it would use stale version of IPC (There is a lot weird things with scoping and context in umbraco :))

[adavidovic92]

@bielu You see, the content item might not need to be used at all. It is enough to simply send a request to the Node.js server to clear all cache and fetch new data. However, the issue is that it still receives outdated data.

[adavidovic92]

@bielu In our context the content item is only used to construct the correct URL for the item that needs clearing — nothing more. The strange thing is that, within the customClient, the content appears correct and seems updated. However, the server still receives outdated data, and we only get the latest data after setting a timeout. Interestingly, this issue does not occur with version 13.3.2.

[bielu]

@adavidovic92 do you any load balancing? as it might be loadbalacing issue not notification issue 🤔

[adavidovic92]

@bielu No, we don't. This issue reproduces on a local machine with a clean Umbraco 13.5.2 project and does not occur with version 13.3.2. Therefore, load balancing is not a factor.

[adavidovic92]

@bielu #17393 (comment) All of this was done on a local machine.

[adavidovic92]

@bielu This is a simple example of API endpoint for retrieving data which is used by Node.js server. Could the issue be related to publishedContentQuery?

[Route("api/home")]
[ApiController]
[ApiExplorerSettings(GroupName = "Home")]
public class HomeController(IPublishedContentQuery publishedContentQuery) : ControllerBase
{
    [HttpGet]
    [ProducesResponseType(typeof(HomeRecord), StatusCodes.Status200OK)]
    [ProducesResponseType(StatusCodes.Status404NotFound)]
    public Results<NotFound, Ok<HomeRecord>> Get()
    {
        var home = publishedContentQuery
            .ContentAtRoot()
            .OfType<Home>()
            .FirstOrDefault();
        if (home is null)
            return TypedResults.NotFound();

        return TypedResults.Ok(new HomeRecord(home.Title ?? string.Empty));
    }
}

public record HomeRecord(string Title);

[bielu]

Tbh i never used IPublishedContentQuery as I found it hard to work with, so hard to say but I guess, could try wrap it with context factory and creating new context reference? or creating new snapshot wtih snapshot factory(I think it is how it was called)?

[adavidovic92]

@bielu thank you, I will try to check this variant 👍

[adavidovic92]

@bielu Unfortunately, I’ve tried both factories and accessors, but the result is the same—the client only receives old data. The data is only up-to-date if I set a timeout of 2-3 seconds.