Last Updated on 11/18/2023 by dboth
A New Look at Swap Space
I spend a lot of time playing – I mean working – on my computers and have found a lot of interesting things. One that has most recently come to my attention is the zram0 device. I first noticed it when working on one of my Opensource.com articles several months ago. It showed up in the output from the lsblk command.
[root@voyager ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 931.5G 0 disk
├─sda1 8:1 0 600M 0 part
<SNIP>
zram0 252:0 0 8G 0 disk [SWAP]As you can see, it is identified as swap space which is what first piqued my curiosity so I did some exploration. Zram was originally called “compcache” which stands for “compressed cache.” It turns out that Zram is a tool for creating in-RAM compressed cache – specifically for use as swap space.
But Why?
When I began researching Zram all I found was a couple basic articles about using Zram for swap space. At first this seemed a bit counter-intuitive to me. After all, if you are running out of RAM and you swap pages into a virtual drive in RAM, what is gained? I then found the FedoraProject wiki page that proposed the use “Swap on Zram.” The proposal says, “Swap is useful, except when it’s slow. zram is a RAM drive that uses compression. Create a swap-on-zram during start-up. And no longer use swap partitions by default.” The rest of the page is about details, benefits, side-effects, and feedback.
Using Zram for swap space is intended to do the same thing as regular partition- or file-based swap space. When memory pressure becomes too great, some of the least recently used data is moved to swap space. On average it is compressed to about 50% of its original size and placed in Zram space in RAM. This is much faster than storing those memory pages on a hard drive and frees up the RAM it was using for other use.
How Much Swap?
I tried to find revised recommendations for how much swap and/or Zram swap to configure. This led me back to a reassessment of swap and my previous article, What’s the right amount of swap space for a modern Linux system? As far as I can tell from the most current documentation for RHEL and Fedora, their recommendations for the amount of swap space have not changed. That documentation still ignores the use of Zram. However the tables in that previous article still provide a good starting point for swap space allocation when using older releases of Linux that don’t use Zram or if Zram has been disabled.
The documents I found for the Zram feature are inconsistent in terms of how Zram is allocated with respect to RAM size and the amount of space allocated to Zram swap.
Due to the lack of authoritative documentation I performed some experiments to empirically determine the algorithm used to allocate Zram swap. I used my own physical and virtual systems for this. The results are interesting and do not match any documentation I have so far found.
The default size of Zram is 8Gb on all systems large enough to support that but is typically reduced significantly on hosts with small amounts of RAM. On one VM I use for testing, with 4GB of RAM allocated to the VM, the Zram virtual swap space is allocated 3.8GB. One old Dell I have contains 8GB of RAM and the Zram is set to 7.6GB. When RAM is reduced to 2GB, Zram is reduced to 1.9GB. All of the physical and virtual hosts I have with more than 8GB of RAM show exactly 8GB of Zram; even my primary workstation with 64GB of RAM and other hosts with 16GB or 32GB of RAM.
Based on these few data points I can draw the conclusion that the current default settings are for 8GB of Zram at most and for Zram to be 95% of RAM on hosts with 8GB or less. I have read a number of articles that mention other sizes for Zram swap, even up to 100% of RAM but those all seem to be theoretical rather than reality.
The actual Zram swap allocation for Fedora and other Red Hat related distributions is given in Figure 1. Your distribution may be different.
| RAM | Zram Swap Size |
|---|---|
| <= 8GB | = 0.95*RAM |
| >8GB | = 8GB |
Be aware that the Zram swap size algorithm is not based on any recommendations for the “best” swap size for any given real-world system or application. This Zram swap allocation is a rather probabilistic approach to what should work well on a wide range of Linux hosts. However, the fact that the maximum Zram swap size is configured for 8GB and the fact that I have always recommended 8GB as the maximum amount of traditional swap, I think I can say that Figure 1 documents the optimum sizes for Zram swap.
Managing Zram Swap
The Zram defaults are stored in the /usr/lib/systemd/zram-generator.conf configuration file. The following is from one of my test VMs with 5097GB of RAM allocated.
[root@testvm1 ~]# cat /usr/lib/systemd/zram-generator.conf
# This config file enables a /dev/zram0 device with the default settings:
# — size — same as available RAM or 8GB, whichever is less
# — compression — most likely lzo-rle
#
# To disable, uninstall zram-generator-defaults or create empty
# /etc/systemd/zram-generator.conf file.
[zram0]
zram-size = min(ram, 8192)You can change the default Zram swap size in the last line of the zram-generator.conf configuration file. I recommend against doing that unless you can definitively show a reason for doing so and test your results once you make any changes. Like many other configuration defaults in Linux, the Zram ones have been well tested and are appropriate for most use cases.
The zramctl utility can be used to view the current state of Zram.
[root@testvm1 ~]# zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle 4.8G 4K 80B 12K 4 [SWAP]The traditional swapon command can also be used to view swap including Zram used as swap.
[root@testvm1 ~]# swapon --show
NAME TYPE SIZE USED PRIO
/dev/zram0 partition 4.8G 0B 100One thing to be aware of is that zramctl does not report on Zram if it contains no data so the results would contain null output. Tools like lsblk, swapon, top, free, htop, and so on, do show Zram even if it contains no data.
The swapoff -a command turns off Zram swap as well as traditional HDD or SSD based storage used as swap. The swapon -a command does not show Zram if it is empty. Use zramctl /dev/zram0 instead.
[root@testvm1 ~]# swapon --show
[root@testvm1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 120G 0 disk
├─sda1 8:1 0 1G 0 part /boot/efi
├─sda2 8:2 0 1G 0 part /boot
└─sda3 8:3 0 118G 0 part
├─vg01-root 253:0 0 10G 0 lvm /
├─vg01-swap 253:1 0 3G 0 lvm [SWAP]
├─vg01-usr 253:1 0 30G 0 lvm /usr
├─vg01-home 253:2 0 10G 0 lvm /home
├─vg01-var 253:3 0 30G 0 lvm /var
└─vg01-tmp 253:4 0 10G 0 lvm /tmp
sr0 11:0 1 1024M 0 rom
zram0 252:0 0 0B 0 disk
[root@testvm1 ~]# zramctl
[root@testvm1 ~]#
[root@testvm1 ~]# zramctl /dev/zram0
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lzo-rle 0B 0B 0B 0B 4
[root@testvm1 ~]#Note that /dev/zram0 does not show up in these commands as swap space until it is being used for that purpose. This caused me some confusion until my experiments showed it to be the case.
Creating Zram Swap
Zram itself has been around for about 20 years but has only been in use as swap space on some distributions for the last year or two. The current Linux installation on some or all of your hosts may not have been created with Zram for swap. If that is the case, it can be easily remedied.
For Fedora 32, the last release prior to the default use of Zram for swap, it only takes three easy commands. First, I verified the presence of the zram-swap.service file which is installed as part of the zram RPM package.
# systemctl status zram-swap
● zram-swap.service - Enable compressed swap in memory using zram
Loaded: loaded (/usr/lib/systemd/system/zram-swap.service; disabled; vendor preset: disabled)
Active: inactive (dead)Next I installed the zram-generator-defaults and zram-generator packages.
# dnf install zram-generator-defaults zram-generatorI enabled and started the zram-swap service.
# systemctl enable zram-swap.serviceI then verified that zram0 exists and is being used as swap space.
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 120G 0 disk
├─sda1 8:1 0 2G 0 part /boot
└─sda2 8:2 0 118G 0 part
├─vg01-root 253:0 0 10G 0 lvm /
├─vg01-swap 253:1 0 3G 0 lvm [SWAP]
├─vg01-usr 253:2 0 35G 0 lvm /usr
├─vg01-tmp 253:3 0 15G 0 lvm /tmp
├─vg01-var 253:4 0 35G 0 lvm /var
└─vg01-home 253:5 0 20G 0 lvm /home
sr0 11:0 1 1024M 0 rom
zram0 252:0 0 7.5G 0 disk [SWAP]
That’s all there is to it. It was easy with Fedora. Your distribution will likely be just as easy but somewhat different in the details of the commands.
Augmenting Zram Swap
Zram swap can be augmented with standard secondary storage devices. Adding some more traditional swap space can be especially useful on systems with low amounts of RAM. Such augmentation will not normally be useful on hosts with very large amounts of RAM.
If you do choose to augment swap with some type of storage device, hard drives still work but are significantly slower than using SSD in either SATA or m.2 formats. However, the flash memory in SSD devices has a more limited life than HDD devices, thus systems with large amounts of swap activity will significantly reduce the chronological lifespan of the SSD.
Tuning swap
There is more to tuning swap space than simply allocating a specific amount of swap space. There are other factors that can be used to manage how swap space is used and managed by the system. Swappiness is the primary kernel parameter that can be used to manage swap performance.
I recently wrote an article for Opensource.com, How I troubleshoot swappiness and startup time on Linux, in which I discuss the vm.swappiness kernel setting. The short version is that the default setting for how aggressively the Linux kernel forces swapping to begin and to function is 60. Zero (0) is the least aggressive and 100 – or 200 depending upon what you read – is the most aggressive. At that level I was experiencing delays when working with very large documents in LibreOffice despite having 64GB of RAM in my primary workstation, much of which was unused.
The resolution to this problem is to reduce the vm.swappiness kernel parameter to 13. This works quite well for my use cases but you may need to experiment to get it right for your environment.
Read the linked article for the details. You can also find more general information about tuning the Linux kernel in an article I wrote for Enable Sysadmin, How to tune the Linux kernel with the /proc filesystem.
Swap Size Recommendations
At this time I have found no recommendations from any distribution for swap size when using Zram. Based on my personal experience I have found that having at least a little swap space can be beneficial. Even when you have large amounts of RAM just the single fact that properly tuned swap for your environment is being used and contains data can indicate that more RAM is needed. The default Zram swap size based on Table 1 is more than sufficient for that purpose. It has worked well so far on all of my Linux hosts.
In my opinion the ultimate purpose of swap space is to be a small buffer – a red flag – that lets the system administrator know when more RAM is needed in a system. Of course some very old hardware cannot support more than 4 or 8GB of RAM. In such a case a new motherboard is needed – one which will support enough RAM to perform the task at hand.
My recommendation is to do as I have. I set up Zram swap of the default size on every one of my hosts. I then removed all of the existing swap partitions and my hosts have all been running perfectly with that swap setup. I have never used swap files – as opposed to swap partitions – because of the relatively slow performance.
Removing Traditional Swap Partitions and Files
Since I just mentioned removing all of the old swap partitions I should also mention that the process to do so is not as straightforward as it should be. It is not hard but it took me some research to figure it out because there is a lot of old and incorrect information out there on the Internet. This procedure works for me on Fedora 36.
First turn off swap for the existing swap partitions and files. This can be done using swapoff /dev/nameofswapdevice but it might be easiest to just turn off all swap with the swapoff -a command. This command also turns off any existing Zram swap.
Remove the entries for traditional swap partitions or files in the /etc/fstab file. First I just commented these out in case of unexpected problems. I deleted those entries later. Zram swap does not require an entry in the /etc/fstab file.
You’d think – at least I did at first – that this would be all that is needed and you could remove the swap partitions or the logical volumes designated as swap. But – no. I did remove the logical volume I had designated as swap space and rebooted to test. The reboot failed and hung very early in the boot process.
Fortunately I have set up the kernel so that it displays boot and startup messages rather than the graphical boot designed to hide the “scary stuff” from users. As a result I was able to see the error message indicating that the kernel couldn’t find the swap volume. To recover from this I booted from a Live Fedora USB drive and created a new swap volume. It is not necessary to do anything else. Then I rebooted and removed the swap entries in kernel line of /etc/defaults/grub. The grub boot defaults are set in the /etc/default/grub file. Changes are made to this file and then the grub.cfg file is regenerated.
The default /etc/default/grub configuration file is simple and we only need concern ourselves with one line.
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="resume=/dev/mapper/vg01-swap rd.lvm.lv=vg01/root rd.lvm.lv=vg01/swap rd.lvm.lv=vg01/usr rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=trueI changed the GRUB_CMDLINE_LINUX line to the following.
GRUB_CMDLINE_LINUX="rd.lvm.lv=vg01/root rd.lvm.lv=vg01/usr"Removing “rhgb quiet” causes all of the kernel boot messages and systemd startup messages to be displayed. This can make it easier to quickly locate problems during the boot and startup phases. Removing “resume=/dev/mapper/vg01-swap” and “rd.lvm.lv=vg01/swap” prevents the kernel from looking for the swap volume.
To make these changes take effect it is necessary to rebuild /boot/grub2/grub.cfg. Make a backup of the current grub.cfg file and then run the following command.
# grub2-mkconfig > /boot/grub2/grub.cfgAfter this I removed the swap partition, ran swapon -a and verified with swapon –show and lsblk. Rebooting the system gave me a final check to ensure that the system did boot properly and that the only swap is the Zram swap.
Conclusion
Zram is a tool that is for creating compressed virtual swap space. The ideal swap configuration depends upon your use case and the amount of physical RAM in your host computer. No matter what combination of Zram, swap partitions, and swap files you use for swap, you should always experiment with your own system loads and verify that your swap configuration works for you. However using the default Zram swap without any traditional swap partitions or files works as well for me as any other swap configuration I have ever used and better than many.