BTRFS made easier to setup during installation

[slim-salm]

Good day,
Forgive me if there is already a thread open for this idea.
I know there is a way to install Linux mint on a BTRFS filesystem, but it is a bit advanced for a newcomer.
I have been running BTRFS with zram for my swap now for nearly 2 years on my laptop and the seamless restore is really nice to have, not even to mention the COW functionality.
Although EXT4 is the default, would it not be nice to have a default pre-configured option for a BTRFS that will work out of the box without going in knee deep into the setup doing things like creating subvolumes @ and @home?

Maybe also add a additional feature to support more subvolumes for timeshift, should advanced users desire to have more subvolumes for whatever reason?

I can really see that business would easily be able to make use of linux mint systems as additional workstations with additional toolsets like versioning with BTRFS

[jakjasie1]

I agree, this sinplifies e.g snapshots in Timeshift. Could even enable them by default when installed with BTRFS, since they don't occupy a lot of space in BTRFS. I believe a good way to implement this is to simply add a switch in the installer, like some other programs do.

[Fantu]

This tutorial can help with the use of btrfs on mint in some cases: https://github.com/orgs/linuxmint/discussions/549

[slim-salm]

I know there are many tutorials to set it up. The point I try to make is to make it seamless like it is set up for EXT4 filesystem.

[mtwebster]

We wouldn't recommend btrfs for new users at this point - if someone thinks they need it it's easy to use the 'Something else' option to make and assign the partitions themselves.

My personal experiences have all started great - I love the snapshot capabilities, but have ended with 'where's my disk space and how do I get it back?' - just google 'btrfs lost disk space'. That's not something a typical user should ever have to worry about running into.

[slim-salm]

I do somewhat agree that it should not be recommended for beginners, but I also think it is for the user to decide. The developer should not be gatekeepers.. I didn't say BTRFS should be the default, it can stay at the advanced section, but all I ask is to make the BTRFS setup as seamless as what EXT4 is, thats it, nothing more, nothing less.

On the flip-side I also had problems with EXT4 where it said the disk is full, but it was not full, later after a format I learned the hard way it had to do with inode limit that is reached if I remember the term correctly, as a new user, you also don't want to deal with such problems as well...

[mortim]

We wouldn't recommend btrfs for new users at this point - if someone thinks they need it it's easy to use the 'Something else' option to make and assign the partitions themselves.

My personal experiences have all started great - I love the snapshot capabilities, but have ended with 'where's my disk space and how do I get it back?' - just google 'btrfs lost disk space'. That's not something a typical user should ever have to worry about running into.

You're right but it could be nice if Mint sets up properly the swap memory within the BTRFS filesystem because the special features (especially CoW) of this filesystem does not allow swap by default.

[slim-salm]

If it cannot be done, that is fine, if you don't want to do it that is fine as well.
But just because of issues such as "lost disk space on BTRFS" shouldn't repel you from using it. The probability is when someone does choose to make use of BTRFS, they already made up their mind that they want to try it. But again, it is a idea, I am no developer, but I do like cool technologies like ZFS an BTRFS.

[Fantu]

I have been using btrfs for about ten years now at work on servers and for a few years also on some disks in personal use.
It has both pros and cons and is not optimal for any use (I still use ext4 in most cases).
It seems stable and reliable to me, apart from the well-known problems with raids with parity (where there is a risk of data loss and is better avoid them), in fact I only use raid 1 and in some cases DUP profile (for double copy even on the same disk).
raid 1 or DUP together with data checksums and self-repair guarantee an excellent advantage for integrity, in some cases more than that of hardware raid and software raid with mdadm, with DUP on single hdd also to limit problems such as those given by possible defective sectors.
It requires some occasional maintenance operations more than ext4, the space problems I have seen are mainly due to the management of snapshots and not the fault of btrfs itself (there can easily be similar problems even using zfs and lvm snapshots) even if in part it keeps a little too much space reserved and can require the "balance" to solve.
There is also less performance (especially on hdd), and mainly because of the cow (which can be deactivated, but you would lose most of the features/advantages), so in certain cases I think it is better not to use it.
I think having it better integrated even if not predefined would be very useful.

[slim-salm]

I can imagine the performance loss on normal hdd, but is it that noticeable on ssd/nvme drives? What will a better integration be for you?

[Fantu]

It's not a quick and easy answer, unfortunately I haven't had time yet (and I don't know when I will have) for a good and in-depth analysis for the desktop part what is needed for a good integration.

Performance issues are mainly on HDD and with growth of the number of snapshots can be very significant, as seen for example in several backup disks on virtualization servers (used only for backup disks and history via snapshots, faster and with less space consumption).
The "problem" remains even if you keep old snapshots deleted (leaving a few) and after 1-2 years you notice a latency in many operations, even mounts.
I suppose defragmentation can help, but I avoid it because it loses the space-saving advantage of cow snapshots (which do not replicate the same data), with defragmentation when you move the data of a snapshot that is the same in others (not taking up space) it duplicates taking up more space.
In SSDs, however, defragmentation obviously should not be used, in general it is not necessary and it shortens the life of the disks (due to the numerous additional writings).
I have rarely used the balance, many seem to recommend it as a regular maintenance operation, but it doesn't seem like a good idea to me (I think should be used only when necessary or more rarely "as maintenance"), it takes a long time and with intensive resource usage, in the case of SSDs here too you have to consider that with many more writes the life is shortened.
If you don't use snapshots (or if you only make a temporary one occasionally to use for external backups) and you use it as a storage/fileserver to have control over data integrity and possible self-repair (with dup and raid profiles) as in several of my cases both on servers at work and personal ones then I haven't seen any particular performance problems, not even on hdd.
I've seen a few more performance issues on Windows (I use it with WinBtrfs) but I only use it for backup disks there.
So it is not relevant for me, but if someone wants to use it for other uses on Windows (or even windows, for example in dual boot) it could be very relevant.
!!! IMPORTANT: on Windows there are other limitations and possible problems, use caution and if you use both windows and Linux I recommend you always do any maintenance operations on it only from Linux.

In these weeks I'm evaluating to increase its use on the new virtualization servers, only for the root and not the vm disks, on the old ones based on Debian stable I used it (and still use) without snapshots for many years, "only" for integrity control and self-repair, on the new server based on Proxmox instead I want to use it more with snapshots and any fast restores if necessary.
With Debian stable the problems are very rare and almost only in the initial testing phase while with Proxmox unfortunately several main components (such as kernel and qemu) are near "rolling release" and the problems are more frequent, in particular regressions when updating, which is why I would like to use snapshots for any very fast restores.
I don't know yet if I will have enough time to contribute a small part with the integration, for example for the official grub-btrfs package on Debian, it would be necessary to do/test also the desktop part with timeshift to do it properly and completely.
On the server part, I excluded timeshift from the start because it is not possible to use it only via CLI and without installing the graphical part.

[AtomicRobotMan0101]

As for maintenance, Ive been using BTRFS Assistant for a while.

It makes most tasks fairly obvious.

As for maintenance, its been my considerable experience that most ordinary users/people don't do any maintenance on their systems at all until something Horrible Happens... (Windows users, not so much Linux users, but this will change as adoption increases).

For maintenance, I run a simple cron monthly to do a balance, scrub and trim.

I'd imagine the devs could add these crons in as part of the setup, plus include a simple visual tool such as BTRFS Assistant so people stand a chance with the file system...

.....

I as a vote, wouldn't mind seeing the BTRFS option less buried within the disks part of setup. :)

.....

Edit - as an aside, I just checked my crons and note that these maintenance functions are already present for ZFS from the install.

[Fantu]

Making maintenance easier and automating some things can help, but unfortunately it cannot make up for the shortcomings of the users.

Most of the serious tests on btrfs, even intentionally causing problems or testing the edge situations (obviously only in test environments) I did at the beginning about ten years ago. Once I had the information I needed and having used btrfs on dozens of volumes for many years without particular problems and only in certain uses, it did not allow me to acquire enough knowledge on some things and in fact recently I am getting better informed and doing more tests because I intend to use btrfs more and in new fields, for example on my next workstation I will use btrfs entirely on the root (and not just a data partition and external backups).

I have read about many btrfs problem situations, but many of them are clearly caused by errors and lack of knowledge of users.
For example:

• Consider snapshots as backups: snapshots on the same volume are NOT backups, having snapshots on a btrfs volume (but it could be for something else) does not provide protection from hardware damage but only to recover previous versions of data due to user or software errors. Yet I have seen people who thought they had complete backup-level protection given by snapshots even on a single disk and single profile (some even on metadata).
  Generally speaking, there are too many people who do not do backups at all or neglect them, regardless of the fs used, and I think this is the main cause of data loss.
  I think people should be better informed about what backups are (basically copies of data on other disks and/or devices) and their importance. In my experience I have seen too many people put important data, without making copies, on external disks or even pen drives thinking that they are safer when in fact it is often worse, and I'm not just talking about my early years of experience in computer support (started 26 years ago) but these are unfortunately frequent things even in recent years.
  I think that users should be informed more about the importance of backups, what they are and more help in managing them.
• Unknowingly using defragmentation: firstly defragmentation should not be used on SSD disks, this on any filesystem, it increases wear unnecessarily, but unfortunately I have seen several cases of use on SSD (for example autodefrag in the mount options).
  It is important to know that in btrfs defragmentation, modifying the data, causes the loss of the deduplication that is automatically achieved with snapshots and reflink copies and therefore takes up more space, much more space if thanks to snapshots and reflinks the snapshots and any other copies took up very little space, do not take this into account, use it when it is not needed or use it when you do not have enough free space can easily lead to having full space on the filesystem. If it is necessary, make sure you have enough free space (it could even be 90% or more of the data space of the subvolume that is being defragmented if it has very high deduplication), in many cases it can be useful (or necessary) to delete all the snapshots or almost and also wait for them to be deleted effectively freeing up the space (on large hdds it can take a long time)
• do not consider or underestimate hardware problems: it's a problem on any filesystem, I don't know if on btrfs some users expect that thanks to the checksums it works miracles, but I've seen several times people continue to use and/or try to repair the filesystem on disks with certain hardware problems, in some cases even evidently serious ones.
  When there are certain problems on the disk, unless you are sure that they are only single and few defective sectors you must absolutely stop using it immediately, and this with any filesystem. If you have backups and there is no need for any recovery, create on a new disk and restore from backup. If instead you need to recover the data you must first clone on a good disk (of equal or greater size), I found ddrescue excellent that allows you to better manage the thing, resume the procedure make several attempts etc... only after cloning on the healthy disk then you try to repair the filesystem and/or recover data.
  On ext4 it seems to me that the users have more chances of seeing problems because in many cases of unexpected events/errors it mounts read-only, on btrfs instead thanks to the checksums on both data and metadata it manages to detect and if it has raids or duplicate profiles correct automatically and continue without the user noticing. In this regard, it could be useful to make sure to have more controls and notifications to users in case of problems (is useful also in case of any other filesystem more check and notification on issues).
  Another thing that is useful to know about btrfs is that contrary to what is used on ext4, fsck should be used rarely and with great care, once you have detected problems, and you have excluded that there are serious hardware problems, instead you should first do a scrub to detect the damages thanks to the checksums (unless they are disabled in some parts) and repair thanks to copies from raid or dup. profiles (usually there is dup. metadata even in the case of a single disk).
• poor space consideration, mainly with snapshots: when using snapshots you have to carefully consider the space they can occupy, and often you can risk underestimating the thing arriving at full space errors (this can obviously also happen in other snapshot systems, such as for example lvm and zfs).
  In the case of btrfs I have seen people complaining about having full space errors even though the data occupied by the main working subvolume was much less. Taking an extreme example, the root subvolume occupied only about a third of the actual space but the btrfs volume was full, going to look at the snapshots there are a large quantity, they even kept the first one made after the initial installation and then over time many have been made for system updates (including even a distro version upgrade), in addition to the work on the data. Obviously if you have many snapshots including one made since the beginning you will also have all the changes made since then on the data, even if you delete data it is as if they were not deleted.
  So you must always take into account the space occupied by snapshots, never underestimate it and always manage snapshots by doing any manual or automatic cleaning in order to limit possible problems.
  As for space management even without snapshots it is always good to keep a good margin, it is also useful in other fs even if you could have fewer problems working at the limit with ext4 rather than btrfs.
  Keeping a large margin is also very useful for other reasons, for example on SSD disks to maintain good performance and longer life (this I think would be a good practice to keep with any filesystem on SSD).
  Of the possible useful checks and notifications for a system I think could be for example to bring to the attention of the user if the free space goes below 10%, if there are more than 20 snapshots and/or there are snapshots older than 6 months. Maybe also make the alerts configurable and have more of them, I don't think I've seen any notification software of the kind for desktop, even if I've never looked seriously and for the servers I've made custom scripts (very limited and specific to what we need) for some parts of management and monitoring (with reports and alert by email).

There would be other things to say, but it would be too long and would take too much time. I'm also not good at explaining or in English.

---

About the disk configuration part in the installer, I think it would be a great thing to improve, but I should do more tests before recommending what would be better to add and/or improve. Recently I tried especially the server part, and I noticed for example the proxmox installer much more limited (not only with btrfs) in disk management, before doing the 2 new installations with btrfs on the root on my next workstation if I have enough time I will try to do some tests with the latest versions of different distributions to try to understand what a good disk configuration support could be (for desktop and workstation, for linux mint), and in particular of btrfs, in the installer.