Auto-configure zram with optimal settings

[mmstick]

Testing

• Play video games that max out the RAM on the system and are sensitive to memory latency. Some form of realtime strategy and a shooter?
• Compiling Firefox, Chrome, or something similarly heavy on RAM while using a web browser for daily stuff.
• Use on systems with very, very little memory and very high memory. Check if the system is performing better or worse.
• Despite most consensus favoring 180 swappiness, we might want to try with a lower value to see if this is really the ideal default for a zram environment.

Note that you can disable and recreate the zram device with

sudo swapoff /dev/zram0
sudo rmmod zram
sudo systemctl start pop-zram

Running zramctl displays the compression algorithm, max size of block device, memory stored inside of it and its compressed size.

Rationale

• Fedora 33 enabled zram by default, with the size of block being half that of total RAM, up to 8GB max
• https://www.reddit.com/r/Fedora/comments/mzun99/new_zram_tuning_benchmarks/
  • Seems optimal settings would be vm.swappiness=180 and vm.page-cluster=0 with zstd comp_alogorithm
• Implemented as a simple bash script that runs on startup, rather than using zram-generator, to avoid a heavier and more complex dependency
  • All the functionality is command-line executions, so bash is a good fit
  • Also gives us more flexibility with tweaking sysctl parameters on demand
  • Since zram recently gained streams support, there isn't a need to have a zram-generator that makes more than one zram device.
  • But people should still be able to use zram-generator instead of the built-in Pop solution if they want to.

Implementation

• Adds a pop-default-settings-zram systemd service that runs /usr/bin/pop-zram-config on system startup
• If a zram device already exists on the system, script exits and no configuration changes are made
• Otherwise, it creates a new zram device and enables it with a priority of 1000. Regenerated each boot in case the amount of RAM changes over time.
• Uses zstd compression with 16 GB max disk size (effectively 4 GB after compression)
• Also applies optimal sysctl values when enabled
• Default values can be overridden with an env file at /etc/pop-zram, which has a default example in /etc/default/pop-zram.
• Makes vm.watermark_scaling_factor = 125, and watermark_boost_factor = 0, a default sysctl value, as recommended here

[lfom]

Please consider keeping things simple and using well known standards. zram-generator is a well known and documented ZRAM manager, it even allows using flexible rules to manage the amount of ZRAM according to the machine RAM:

https://github.com/systemd/zram-generator/blob/main/man/zram-generator.conf.md

Like other systemd services, it is possible to create a simple drop in file like:

/etc/systemd/zram-generator.conf.d/50-pop-zram.conf

with the desired rule (the one suggested in the link above looks very good, for both machine with low RAM and otherwise).

And sysctl values should go to the default place as well, for instance:

/etc/sysctl.conf.d/99-pop-zram.conf

This would use standards, as well as making it very easy to find where the settings are stored, and also allows user customization.

[mmstick]

@lfom We already have a sysctl config file for things that make sense to be applied universally by default (see the PR). In this particular case, it's not a good idea to have conditional parameters stored in a permanent configuration file.

The sysctl parameters are applied in the script because they need to be evaluated at runtime (vm.page-cluster=0 for zstd, 1 for others, 2 for SSD-based swap, 3 for swap on magnetic drives), and only applied when the system is actually using the pop-configured zram block device (vm.swappiness=180 is not ideal for traditional swap on a desktop). Performance would be negatively impacted for those who disable pop-zram.service if settings were generated and written to /etc/sysctl.d/.

As mentioned in the PR's description, I'm really not sure zram-generator is so necessary to add as a dependency when the complexity isn't needed for a system default. If someone wants a more complex setup, they could add zram-generator to their system and use that instead. And this way it's easier to opt out of the Pop config.

I'm also trying to avoid including any config files that are not easily disabled. I only have one optional user-defined env file at /etc/pop-zram to override the default parameters. Distribution default configuration files are placed into /usr/ and overwritten or rewritten on every package update. So a user can't easily disable the config provided by the distribution, unless the service has some way of masking configs.

I'm a fan of simple solutions. This is a pretty basic 20 line bash script to automatically evaluate and apply the recommended system defaults. At it's core, we're talking about a script that's just running

modprobe zram num_devices=1 && sync
zramctl --size "${SIZE}M" --algorithm "${ALGO}" /dev/zram0
mkswap /dev/zram0 && sync
swapon -p "${PRIORITY}" /dev/zram0
sysctl -w "vm.page-cluster=${PAGE_CLUSTERS}" "vm.swappiness=${SWAPPINESS}"

Regardless, we'd still need a system service to apply the sysctl tweaks when using the default configuration provided by Pop.

[lfom]

If the main reason for doing all this is to check the current user configuration for systems already running Pop!_OS, then this can be done during the package installation. This would prevent problems for any user who already changed the default swap configuration IMO. Besides the additional rules in the post-installation script to check the current swap configuration, it would require only enabling zram-generation and placing the two text configuration files.

[benuski]

Enabled this on my HP Dev One with 64gb of RAM. Everything is working as expected with no performance degredations. It currently feels a bit snappier under a basic load, and I'll continue to test it with more intensive things.

[woutheijnen]

Thank you for this great find, it works wonderfully on my PC with 8GB RAM, and I got a solid boost from sometimes 80 fps to 120fps (my refresh rate) and other times from 80fps to 90 fps.
I have various pcs with different amounts of RAM available, currently on 8GB and 16GB it works good, 4GB shows some good things too but want I need to test it a bit more though I think. The 4GB pc doesn't run Pop OS though, it runs Mint with xfce. Also want to try out on a 2GB pc to see how it runs, but will see when I get the time to install it as it currently doesn't run anything.

I have a burning question for some time though and this seems like a good moment to ask it : Pop by default enables zswap and it defaults to lzo/zbud, but wouldn't it be better if it would default to zstd/z3fold ? Or lz4/z3fold ?
I used to play Cities Skylines which is a very memory heavy game when you get all the dlc + mods as it loads all assets to RAM, but even with my pc of 8GB RAM, it managed to be completely playable with lz4/z3fold (haven't tried zstd). I also use the package swapspace to automatically create swap if necessary and this made it so 12-14GB was loaded in swap (on ssd) + 8GB in RAM and the game was perfectly playable at around 50fps and rarely crashed.
The only issue with the package swapspace I had was on my work pc when I dumped a self referenced variable with print_r in PHP, and it started to explode in memory and thus in disk size (easily 40+GB RAM in about a minute), but normally swapspace has configuration options to limit the amount of space it can take.

So maybe those are some other optimizations that can be made by defaulting to zstd/z3fold and by using a tuned swapspace package (instead of running out of memory as is the default case currently) for Pop.

[mmstick]

@woutheijnen This defaults to zstd

[woutheijnen]

@woutheijnen This defaults to zstd

I know but it defaults to zstd for zram only, not for zswap.
When you boot up your pc and execute journalctl | grep zswap it says :
zswap: loaded using pool lzo/zbud

This is for a similar optimization (zswap) to maybe make it default to zstd/z3fold instead of lzo/zbud and maybe swapspace could be handy too for Pop_Os systems in case of systems having low RAM just an optimization suggestion

[n3m0-22]

This looks good. I saw no regressions. I tested on RAM configurations between 4GB and 64GB. On older machines with low RAM (i.e., 4-8GB RAM) the difference was most notable. I had no issues with using zram-generator and the bash script worked as expected. On machines with 16GB daily tasks like browsing, docs and spreadsheets, and streaming (although a bit jittery at times) are still very responsive. This was with a 60GB game downloading and Firefox compiling.

The most extreme case of this making a difference was a 10 year old ThinkPad with 8GB of RAM. I was able to play Control for a while at around 30 FPS. It did still crash, but it was working for a bit. Without this PR it showed around 11 FPS as it loaded the menu and then crashed.

I played around with different swappiness values, but was not able to see a noticeable difference. Though this may vary with different hardware configurations.

It did regenerate itself on the next boot after changes to RAM in the machine (e.g., removing 8GB from a 16GB).

I have also had this running on several machines I use on a regular basis for a few days now doing normal day to day tasks and have seen no problems that I could attribute to this PR.

Note: Tested on many different machines with Intel and AMD hardware with and with out dedicated graphics.

[n3m0-22]

After running through the same testing as before I can confirm I am getting the same results as stated above.

This looks good to me.

[Mirppc]

FYI the settings for Zram will change the defaults for normal swap too. So if Zram is disabled the existing zram swappiness settings are left causing swap to be prioritized over ram.

[ahydronous]

@mmstick it could be a consideration to:

• use zstd for systems <16GB
• use zstd for <24GB systems with an iGPU (since that will 'steal' system memory)
• use lz4 for ≥16GB systems with a discrete GPU
• use lz4 for ≥24GB systems with an iGPU

The reason I say this is because of the charts in the Reddit post you linked: https://imgur.com/a/linux-zram-benchmarks-bUSBlm1. Or was extensive profiling done by the Pop team?

zstd has horrible access times compared to lz4, and generally for system memory access times matter more than throughput.
Tangential, but according to that bench, lz4 does also benefit from vm.page-cluster=0. It pays a pretty big penalty otherwise.