Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

L2ARC with metadata and MFU only #16343

Closed
tkittich opened this issue Jul 12, 2024 · 3 comments · Fixed by #16402
Closed

L2ARC with metadata and MFU only #16343

tkittich opened this issue Jul 12, 2024 · 3 comments · Fixed by #16402
Labels
Component: Memory Management kernel memory management Type: Feature Feature request or new feature

Comments

@tkittich
Copy link
Contributor

Describe the feature would like to see added to OpenZFS

Option to store only metadata (all metadata) and MFU (both metadata and data) in L2ARC

How will this feature improve OpenZFS?

This feature should be the default and would make a perfect L2ARC for home users. Most metadata and most frequently used data would eventually get stored in the L2ARC!! This would be the perfect tiered storage system.

Of course, this is currently possible by using metadata special vdev together with l2arc_mfuonly L2ARC. But a special vdev introduces another risk if damaged without enough redundancy. For RAIDZ2 of home users, it's not easy to have 3 nvme's as special vdev because most mainboards only have 2 nvme slots.

Additional context

Currently primarycache and secondarycache ZFS properties only allow all, none, and metadata. Perhaps this feature could be added as a new setting, i.e. metaandmfu, for primarycache and secondarycache .
@tkittich tkittich added the Type: Feature Feature request or new feature label Jul 12, 2024
@amotin
Copy link
Member

amotin commented Jul 12, 2024

I think we really need to rethink what information are we writing into L2ARC and how. That area got no significant love for decade probably. L2ARC from the beginning was made to write metadata and MFU first. But I think it may got affected by introduction of multilists in ARC. We may also recheck interaction of L2ARC with prefetch -- we may not want to write to L2ARC data that are successfully prefetched from the main pool. We could also redesign write boost there not based on ARC being warm, but on L2ARC being above certain fill percent, that combined with the above preference should result in writes of more usable data.

@rincebrain rincebrain added the Component: Memory Management kernel memory management label Jul 14, 2024
@AllKind
Copy link
Contributor

AllKind commented Jul 15, 2024

Just as a side-note a method to make failures of special vdev non-fatal is in the works: #16185

shodanshok added a commit to shodanshok/zfs that referenced this issue Jul 31, 2024
@shodanshok
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
cf5d3e2 
Fixes openzfs#16343

`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possibile value/behavior is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
shodanshok added a commit to shodanshok/zfs that referenced this issue Jul 31, 2024
@shodanshok
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
2de390c 
Fixes openzfs#16343

`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possible values is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
@shodanshok shodanshok mentioned this issue Jul 31, 2024
Merged
13 tasks
shodanshok added a commit to shodanshok/zfs that referenced this issue Jul 31, 2024
@shodanshok
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
96a6c33 
Fixes openzfs#16343

`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possible values is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
@shodanshok
Copy link
Contributor

I submitted #16402 which should enable what asked in this issue: L2 cache of all metadata but MFU data only. I used the already defined l2arc_mfuonly tunable: while using a dataset property would be great, it seems to me that the L2 ARC feed thread know nothing about properties (it only cares about buffer lists/sublist).

shodanshok added a commit to shodanshok/zfs that referenced this issue Aug 1, 2024
@shodanshok
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
13396a7 
Fixes openzfs#16343

`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possible values is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
@zfsuser zfsuser mentioned this issue Aug 4, 2024
Open
shodanshok added a commit to shodanshok/zfs that referenced this issue Aug 12, 2024
@shodanshok
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
1185485 
Fixes openzfs#16343

`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possible values is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
lundman pushed a commit to openzfsonwindows/openzfs that referenced this issue Sep 4, 2024
@shodanshok @lundman
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
4835c86 
`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possible values is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
Closes openzfs#16343 
Closes openzfs#16402
tonyhutter pushed a commit that referenced this issue Sep 4, 2024
@shodanshok @tonyhutter
Enable L2 cache of all (MRU+MFU) metadata but MFU data only
cd42e99 
`l2arc_mfuonly` was added to avoid wasting L2 ARC on read-once MRU
data and metadata. However it can be useful to cache as much
metadata as possible while, at the same time, restricting data
cache to MFU buffers only.

This patch allow for such behavior by setting `l2arc_mfuonly` to 2
(or higher). The list of possible values is the following:
0: cache both MRU and MFU for both data and metadata;
1: cache only MFU for both data and metadata;
2: cache both MRU and MFU for metadata, but only MFU for data.

Reviewed-by: Alexander Motin <mav@FreeBSD.org>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Gionatan Danti <g.danti@assyoma.it>
Closes #16343 
Closes #16402
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Component: Memory Management kernel memory management Type: Feature Feature request or new feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enable L2 cache of all (MRU+MFU) metadata but MFU data only shodanshok/zfs
5 participants
@rincebrain @AllKind @tkittich @amotin @shodanshok