Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cache LastAccessed during MemoryCache compaction #61187

Merged
merged 3 commits into from
Dec 2, 2021
Merged

Cache LastAccessed during MemoryCache compaction #61187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2150b5a
Cache LastAccessed during MemoryCache compaction
eerhardt Nov 3, 2021
23c91e4
Merge remote-tracking branch 'upstream/main' into FixMemoryCacheCompa…
eerhardt Dec 1, 2021
78f40eb
Disable parallelization on race condition test
eerhardt Dec 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions src/libraries/Microsoft.Extensions.Caching.Memory/src/MemoryCache.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -391,9 +391,10 @@ public void Compact(double percentage)
private void Compact(long removalSizeTarget, Func<CacheEntry, long> computeEntrySize, CoherentState coherentState)
{
var entriesToRemove = new List<CacheEntry>();
var lowPriEntries = new List<CacheEntry>();
var normalPriEntries = new List<CacheEntry>();
var highPriEntries = new List<CacheEntry>();
// cache LastAccessed outside of the CacheEntry so it is stable during compaction
var lowPriEntries = new List<(CacheEntry entry, DateTimeOffset lastAccessed)>();
var normalPriEntries = new List<(CacheEntry entry, DateTimeOffset lastAccessed)>();
var highPriEntries = new List<(CacheEntry entry, DateTimeOffset lastAccessed)>();
long removedSize = 0;

// Sort items by expired & priority status
Expand All @@ -411,13 +412,13 @@ private void Compact(long removalSizeTarget, Func<CacheEntry, long> computeEntry
switch (entry.Priority)
{
case CacheItemPriority.Low:
lowPriEntries.Add(entry);
lowPriEntries.Add((entry, entry.LastAccessed));
break;
case CacheItemPriority.Normal:
normalPriEntries.Add(entry);
normalPriEntries.Add((entry, entry.LastAccessed));
break;
case CacheItemPriority.High:
highPriEntries.Add(entry);
highPriEntries.Add((entry, entry.LastAccessed));
break;
case CacheItemPriority.NeverRemove:
break;
Expand All @@ -441,7 +442,7 @@ private void Compact(long removalSizeTarget, Func<CacheEntry, long> computeEntry
// ?. Items with the soonest absolute expiration.
// ?. Items with the soonest sliding expiration.
// ?. Larger objects - estimated by object graph size, inaccurate.
static void ExpirePriorityBucket(ref long removedSize, long removalSizeTarget, Func<CacheEntry, long> computeEntrySize, List<CacheEntry> entriesToRemove, List<CacheEntry> priorityEntries)
static void ExpirePriorityBucket(ref long removedSize, long removalSizeTarget, Func<CacheEntry, long> computeEntrySize, List<CacheEntry> entriesToRemove, List<(CacheEntry Entry, DateTimeOffset LastAccessed)> priorityEntries)
{
// Do we meet our quota by just removing expired entries?
if (removalSizeTarget <= removedSize)
Expand All @@ -454,8 +455,8 @@ static void ExpirePriorityBucket(ref long removedSize, long removalSizeTarget, F
// TODO: Refine policy

// LRU
priorityEntries.Sort((e1, e2) => e1.LastAccessed.CompareTo(e2.LastAccessed));
foreach (CacheEntry entry in priorityEntries)
priorityEntries.Sort(static (e1, e2) => e1.LastAccessed.CompareTo(e2.LastAccessed));
foreach ((CacheEntry entry, _) in priorityEntries)
{
entry.SetExpired(EvictionReason.Capacity);
entriesToRemove.Add(entry);
Expand Down
4 changes: 4 additions & 0 deletions ...raries/Microsoft.Extensions.Caching.Memory/src/Microsoft.Extensions.Caching.Memory.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,8 @@
<ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.Primitives\src\Microsoft.Extensions.Primitives.csproj" />
</ItemGroup>

<ItemGroup Condition="'$(TargetFrameworkIdentifier)' == '.NETFramework'">
<PackageReference Include="System.ValueTuple" Version="$(SystemValueTupleVersion)" />
</ItemGroup>
Comment on lines +17 to +19
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backporting a fix that has new dependencies seems problematic.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I create a new private struct for this data instead of using a ValueTuple?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ericstj @davidfowl - any thoughts here? If we decide to backport this to 6.0.x, would this new dependency on the System.ValueTuple package be a problem? Nothing in MS.Ext.Caching.Memory brings in this package today on netfx.

I can easily switch to not use value tuples, but it seems like a pain going forward.

One option could be use value tuples in 7.0 going forward, and just use a custom struct in the 6.0 version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just stick to a small internal struct for now. It's much less intrusive and doesn't require an ifdef. We should then decide when we rip the netstandard 2.0 cord. (.NET 7 or 8?)

Copy link
Member Author

@eerhardt eerhardt Nov 9, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't require an ifdef

The plan I'm proposing doesn't require an ifdef (beyond this PackageReference). The idea is:

For the 7.0.0+ versions of this package (in main) we use ValueTuple like I am here. Then it is available in the future if we want to use it in other places. If servicing 6.0 wasn't a consideration, this is the plan I would take without a 2nd thought.

For the 6.0.1 version of this package (in release/6.0) change the ValueTuple uses in this PR to a small struct and remove the dependency on System.ValueTuple. That way we don't add the new dependency in a service release.

We should then decide when we rip the netstandard 2.0 cord. (.NET 7 or 8?)

The plan above fits great with this. If/when we rip the netstandard2.0 cord, this PackageReference goes away and the C# code doesn't change at all, since ValueTuple is available inbox on NETCOREAPP.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ValueTuple dependency can be a real PITA for users who are using a very old VS that generates invalid binding redirects for .NET Framework projects: dotnet/BenchmarkDotNet#1687 (comment)

I doubt that they use latest MemoryCache version and I think that we should just stop targeting .NET Standard 2.0. But we need to verify that with some data.


</Project>
55 changes: 55 additions & 0 deletions src/libraries/Microsoft.Extensions.Caching.Memory/tests/CompactTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.Internal;
using Xunit;

Expand Down Expand Up @@ -83,4 +85,57 @@ public void CompactPrioritizesLRU()
Assert.Equal("value4", cache.Get("key4"));
}
}

[Collection(nameof(DisableParallelization))]
public class CompactTestsDisableParallelization
{
/// <summary>
/// Tests a race condition in Compact where CacheEntry.LastAccessed is getting updated
/// by a different thread than what is doing the Compact, leading to sorting failing.
///
/// See https://github.com/dotnet/runtime/issues/61032.
/// </summary>
[Fact]
public void CompactLastAccessedRaceCondition()
{
const int numEntries = 100;
MemoryCache cache = new MemoryCache(new MemoryCacheOptions());
Random random = new Random();

void FillCache()
{
for (int i = 0; i < numEntries; i++)
{
cache.Set($"key{i}", $"value{i}");
}
}

// start a few tasks to access entries in the background
Task[] backgroundAccessTasks = new Task[Environment.ProcessorCount];
eerhardt marked this conversation as resolved.
Show resolved Hide resolved
bool done = false;

for (int i = 0; i < backgroundAccessTasks.Length; i++)
{
backgroundAccessTasks[i] = Task.Run(async () =>
{
while (!done)
{
cache.TryGetValue($"key{random.Next(numEntries)}", out _);
await Task.Yield();
}
});
}

for (int i = 0; i < 1000; i++)
{
FillCache();

cache.Compact(1);
}

done = true;

Task.WaitAll(backgroundAccessTasks);
}
}
}
5 changes: 5 additions & 0 deletions ...icrosoft.Extensions.Caching.Memory/tests/Microsoft.Extensions.Caching.Memory.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@
<EnableDefaultItems>true</EnableDefaultItems>
</PropertyGroup>

<ItemGroup>
<Compile Include="$(CommonTestPath)TestUtilities\System\DisableParallelization.cs"
Link="Common\TestUtilities\System\DisableParallelization.cs" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\src\Microsoft.Extensions.Caching.Memory.csproj" SkipUseReferenceAssembly="true" />
<ProjectReference Include="$(LibrariesProjectRoot)Microsoft.Extensions.DependencyInjection\src\Microsoft.Extensions.DependencyInjection.csproj" />
Expand Down