This repo contains a few bits needed to boot a coreOS VM from an iSCSI root disk. This is still a work in progress
flowchart BT
A(iSCSI Target)
B[FCOS VM] --> |container volume| A
B --> E
E(COSA) --> D
C(COSA) --> |virtioFS| B
D[diskless VM] <--> |netboot| A
R{{HOST RAW image}} --> |container volume| C
Targetcli in a container. It requires root privileges.
# podman build -t targetcli -f Dockerfile .
If you want to run the container on its own (without through the ignition config provided), here is the required arguments:
# podman run -d \
--name targetclid \
-v /lib/modules:/lib/modules \
-v /sys/kernel/config:/sys/kernel/config \
--privileged --cap-add=CAP_SYS_MODULE \
--net host \
quay.io/jbtrystram/targetcli
then podman exec -it targetclid targetcli
, you can then create portals, luns and all iSCSI things.
Note: --net host
is used here because -p 3260:3260
made the LUN to bind to the container network address.
Here we focus on running the iSCSI target container in a virtual machine created through cosa
The ignition
folders contains the necessary butane configuration to launch and configure everything:
target.container
sets up the container explained above as a systemd service.setup-targetcli.service
sets up a systemd service to executetargetcli_script
, only on the first boot and only once.targetcli_script
is simply a bash script that executetargetcli
commands inside the container to configure it.target.bu
ties it all together, essentially copying the above bits in the correct places.
You will need a metal image of fedora coreOS and a regular image :
cosa fetch && cosa build && cosa buildextend-metal
Then, copy the metal raw file in the tmp workdir, expand it and make it writeable:
cp builds/latest/x86_64/fedora-coreos-*.raw tmp/fcos.raw
chmod 664 temp/fcos.raw
Neatly, cosa run
will mount tmp/
at /var/mnt/workdir-tmp
in the VM through virtioFS.
Build the butane config:
butane --pretty --strict target.bu -d . > target.ign
Then cosa run -i /path/to/target.ign
At this point, the iSCSI target setup is complete.
Now, we create a second VM, which should boot from the iSCSI target. We'll need an iPXE boot script:
$ cat /mnt/workdir-tmp/boot.ipxe
#!ipxe
sanboot iscsi:10.0.2.15::::iqn.2023-10.coreos.target.vm:coreos
Then boot !
podman run -ti --privileged --net=host --rm -v /mnt/workdir:/mnt/workdir \
quay.io/coreos-assembler/coreos-assembler shell -- \
kola qemuexec --netboot /mnt/workdir/tmp/boot.ipxe --usernet-addr 10.0.3.0/24
At this point, you should get a GRUB screen, pulled from that initial metal raw
file, booted off iscsi ! Happy hacking.
The setup target in the ignition config is iqn.2023-10.coreos.target.vm:coreos
. No auth is required.
So on the initator, to mount the volume
Discover :
# sudo iscsiadm -m discovery -t st -p 10.0.2.15
10.0.2.15:3260,1 iqn.2023-10.coreos.target.vm:coreos
Mount :
sudo iscsiadm -m node -T iqn.2023-10.coreos.target.vm:coreos -l
Logging in to [iface: default, target: iqn.2023-09.centos.vm:coreos, portal: 192.168.122.2,3260]
Login to [iface: default, target: iqn.2023-09.centos.vm:coreos, portal: 192.168.122.2,3260] successful.
Then find where it was mounted : journalctl --no-pager | grep "Attached SCSI"
The core OS disk image :
sudo blkid /dev/sdc*
/dev/sdc: PTUUID="a1c653e9-8857-4eb1-b69b-6f53952a569d" PTTYPE="gpt"
/dev/sdc1: PARTLABEL="BIOS-BOOT" PARTUUID="6498509e-b163-4660-b067-04ea5db88b1d"
/dev/sdc2: SEC_TYPE="msdos" LABEL_FATBOOT="EFI-SYSTEM" LABEL="EFI-SYSTEM" UUID="E45-2746" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI-SYSTEM" PARTUUID="60f508bc-0dee-4355-b052-aae90650b550"
/dev/sdc3: LABEL="boot" UUID="78ec57df-7912-4cf1-b9c9-2a782bf6ea83" BLOCK_SIZE="1024" TYPE="ext4" PARTLABEL="boot" PARTUUID="3f1a5f26-36e1-4411-807c-b579d20d855d"
/dev/sdc4: LABEL="root" UUID="0f296b65-e139-4729-9890-c9312eb91bbd" BLOCK_SIZE="512" TYPE="xfs" PARTLABEL="root" PARTUUID="0a29aecb-1b5b-450b-9964-1760b3b72acf"
https://medium.com/oracledevs/kvm-iscsi-part-i-iscsi-boot-with-ipxe-f533f2666075
https://cleech.github.io/blog/2013/11/05/booting-virtual-machines-using-iscsi-part-2/
https://developers.redhat.com/blog/2019/04/24/how-to-run-systemd-in-a-container?source=sso