lib/repo: Add min-free-space-percent option, default 3% #987
Conversation
|
OK, this works pretty well in my testing, though I haven't added a unit or integration test yet. Debating a bit which to do. |
| g_mutex_lock (&self->txn_stats_lock); | ||
| self->txn_blocksize = stvfsbuf.f_bsize; | ||
| /* Convert fragment to blocks to compute the total */ | ||
| guint64 total_blocks = (stvfsbuf.f_frsize * stvfsbuf.f_blocks) / stvfsbuf.f_bsize; |
There was a problem hiding this comment.
Doesn't this multiplication yield the total size of the filesystem in bytes already? statvfs(2) says:
fsblkcnt_t f_blocks; /* Size of fs in f_frsize units */
Writing a quick program to check this:
$ cat tmp.c
#include <sys/statvfs.h>
#include <stdio.h>
int main(void)
{
struct statvfs stat;
if (statvfs("/", &stat) < 0)
return 1;
unsigned long long total_bytes = stat.f_blocks * stat.f_frsize;
printf("total size in bytes: %llu\n", total_bytes);
printf("total size in GB: %llu\n", total_bytes >> 30);
return 0;
}
$ ./tmp
total size in bytes: 95133081600
total size in GB: 88
$ df -hT /
Filesystem Type Size Used Avail Use% Mounted on
/dev/mapper/fedora-root ext4 89G 63G 22G 75% /
So then, we need to divide this by f_bsize to be made comparable to bfree, right?
There was a problem hiding this comment.
The first multiplication indeed yields size in bytes. But the reason we divide by the block size is that the intent is to do all calculations in terms of blocks, not bytes. That's how the filesystem works ultimately.
When we're looking at object sizes, we divide it by the block size (and add one to be conservative, though obviously in practice an object could land exactly on a block boundary).
There was a problem hiding this comment.
Oh goodness, I completely missed the / stvfsbuf.f_bsize; chunk of that line! (I really like the split view when looking at diffs, but this chops up lines much more when looking on my laptop).
Hacky, but one way would be to add a test to the |
| { | ||
| g_mutex_lock (&self->txn_stats_lock); | ||
| g_assert_cmpint (self->txn_blocksize, >, 0); | ||
| const fsblkcnt_t object_blocks = (size / self->txn_blocksize) + 1; |
There was a problem hiding this comment.
(Here's where we convert objects into blocks)
This is prep for storage space checks, where we look at free space after parsing the metadata, before we write anything. We did length-limited writes in the fd-based input path, but not for the `GInputStream` path which in practice is used for HTTP pulls.
For ostree-as-host, we're the superuser, so we'll blow past any reserved free space by default. While deltas have size metadata, if one happens to do a loose fetch, we can fill up the disk. Another case is flatpak: the system helper has similar concerns here as ostree-as-host, and for `flatpak --user`, we also want to be nice and avoid filling up the user's quota. Closes: ostreedev#962
cgwalters
force-pushed
the
size-constraints
branch
from
062b54b
to
dcde8ad
Compare
|
🏄♂️ Rebased and now with an installed 🏗️ test ✅ |
For ostree-as-host, we're the superuser, so we'll blow past any reserved free space by default. While deltas have size metadata, if one happens to do a loose fetch, we can fill up the disk. Another case is flatpak: the system helper has similar concerns here as ostree-as-host, and for `flatpak --user`, we also want to be nice and avoid filling up the user's quota. Closes: #962 Closes: #987 Approved by: jlebon
|
☀️ Test successful - status-atomicjenkins |
For ostree-as-host, we're the superuser, so we'll blow past
any reserved free space by default. While deltas have size
metadata, if one happens to do a loose fetch, we can fill
up the disk.
Another case is flatpak: the system helper has similar concerns
here as ostree-as-host, and for
flatpak --user, we alsowant to be nice and avoid filling up the user's quota.
Closes: #962