/boot partition disk usage too high

[dragonraid]

Hi all,

I am running bottlerocket as OS for worker nodes for EKS. I recently updated bottlerocket AMI from ami-0ab0c02538ad82487 to ami-04958b57e72e69fb0 (us-east-1) and noticed that disk usage of boot partition jumped from 70% to 95,9%. I did not notice any issues, just asking if this to expected.

[etungsten]

Hi @dragonraid, the boot fs size did change between v1.8.0 and v1.9.0.

bash-5.1# df -h /boot/
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p3   27M   18M  6.9M  72% /boot
bash-5.1# apiclient get /os
{
  "arch": "x86_64",
  "build_id": "a6233c22",
  "pretty_name": "Bottlerocket OS 1.8.0 (aws-k8s-1.22)",
  "variant_id": "aws-k8s-1.22",
  "version_id": "1.8.0"
}

bash-5.1# df -h /boot/
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p3   14M   13M  539K  96% /boot
bash-5.1# apiclient get os
{
  "os": {
    "arch": "x86_64",
    "build_id": "b8074d44",
    "pretty_name": "Bottlerocket OS 1.9.2 (aws-k8s-1.22)",
    "variant_id": "aws-k8s-1.22",
    "version_id": "1.9.2"
  }
}

This is probably the result of the behavior of mkfs.ext4 with the -d option to create a filesystem from a directory:

bottlerocket/tools/rpm2img

Line 295 in fa341b4

mkfs.ext4 -O ^has_journal -d "${BOOT_MOUNT}" "${BOOT_IMAGE}" "${partsize[BOOT-A]}M"

The boot partition has 40M reserved in total:

nvme0n1p3  259:5    0   40M  0 part /boot

When we're mounting the boot partition and creating a filesystem, mkfs won't allocate the entirety of the reserved space of the boot partition for the fs. We recently trimmed out some unnecessary files from the boot partition in #2223 so mkfs.ext4 ends up creating a smaller filesystem. Which explains what you're seeing here with the smaller fs size and higher utilization.