Home

Welcome to the iomemory-vsl4 wiki!

The iomemory-vsl4 driver is meant for v4.x devices. For iomemory v3/x device drivers please go here iomemory-vsl.. The changes to the iomemory v4 driver are somewhat interchangeable with the iomemory v3 driver sometimes....

Releases

Releases are tagged, and should be checked out by their tag. The release tags follow Linux Kernel versions. E.g. v5.6.0 (Danger Noodle) will work on all 5.x kernels that are 5.6 and lower, but is not guaranteed to work on 5.7.

Tag	Codename
v5.10.0	Spicy Nope Rope

Ceph deployment

One good use of a Fusion-IO device would be to put the Block.db on a partition to use it as NVME-like storage. Here's the basic steps to get a Ceph node with a Fusion-IO card running:

If you had 4 slow spinning-rust devices (HDD) and a Fusion-IO card, you could partition the card with ceph-volume to act as a dedicated WAL/DB device, or you could have ceph consume all of it as fast SSD storage.

Proxmox deployment

Note that in Proxmox, their tooling does not support /dev/fio* devices, so if you wanted to use fio as a DB/WAL device but had no OSDs created, you would need to create a single temporary OSD via the GUI so to it generates a new keyring pair, then you should be able to run ceph-volume. After successfully running ceph-volume to create all of your OSDs, you can delete the original OSD from the GUI and re-create it using the CLI using ceph-volume again.

Format the card with 4K sectors:

fio-detach /dev/fct0 fio-format /dev/fct0 -s 4K fio-attach /dev/fct0

Using as a block device in ceph:

ceph-volume lvm create --bluestore --data /dev/fioa

Using as a dedicated DB/WAL device (note this will use the whole drive and is not recommended):

ceph-volume lvm create --bluestore --data /dev/sdX --block.db /dev/fioa

Using FIO partitions as dedicated DB/WAL device

For the example below, I am using 5x 1.2TB SAS HDDs, and creating 300GB partitions on my faster FIO flash card for the DB. Let us assume five rotational (sda, sdb, sdc, sdd and sde) and one (fast) solid state drive (fioa).

First create the volume groups:

$ vgcreate ceph-block-0 /dev/sda
$ vgcreate ceph-block-1 /dev/sdb
$ vgcreate ceph-block-2 /dev/sdc
$ vgcreate ceph-block-3 /dev/sdd
$ vgcreate ceph-block-4 /dev/sde

Now create the logical volumes for block:

$ lvcreate -l 100%FREE -n block-0 ceph-block-0
$ lvcreate -l 100%FREE -n block-1 ceph-block-1
$ lvcreate -l 100%FREE -n block-2 ceph-block-2
$ lvcreate -l 100%FREE -n block-3 ceph-block-3
$ lvcreate -l 100%FREE -n block-4 ceph-block-4

We are creating 5 OSDs for the five slow spinning devices, so assuming a 3TB FIO device in /dev/fioa we will create 5 logical volumes, each of 300GB:

$ vgcreate ceph-db-0 /dev/fioa
$ lvcreate -L 300GB -n db-0 ceph-db-0
$ lvcreate -L 300GB -n db-1 ceph-db-0
$ lvcreate -L 300GB -n db-2 ceph-db-0
$ lvcreate -L 300GB -n db-3 ceph-db-0
$ lvcreate -L 300GB -n db-4 ceph-db-0

Finally, create the 5 OSDs with ceph-volume:

$ ceph-volume lvm create --bluestore --data ceph-block-0/block-0 --block.db ceph-db-0/db-0
$ ceph-volume lvm create --bluestore --data ceph-block-1/block-1 --block.db ceph-db-0/db-1
$ ceph-volume lvm create --bluestore --data ceph-block-2/block-2 --block.db ceph-db-0/db-2
$ ceph-volume lvm create --bluestore --data ceph-block-3/block-3 --block.db ceph-db-0/db-3
$ ceph-volume lvm create --bluestore --data ceph-block-4/block-4 --block.db ceph-db-0/db-4

Flashing

Download the official firmware files from Western Digital Support and download the firmware files released by the OEM (e.g. IBM or HP).

Run the following on the host where the card lives (assuming you've already followed the steps to download and build the driver): sudo fio-update-iodrive path_to_official_firmware

It will complain that "part [partnumber]" is not in the file - in my case that part number was PA005831109

root@pve1:~# fio-update-iodrive fusion_4.3.7-20200113.fff 
WARNING: DO NOT TURN OFF POWER OR RUN ANY IODRIVE UTILITIES WHILE THE FIRMWARE UPDATE IS IN PROGRESS
  Please wait...this could take a while

Updating: [                    ] (  0%) /
Error: Firmware file does not contain firmware for device 'fct0', part 'PA005831109'.
root@pve1:~# 

Edit the OEM INFO file with 7zip (the .fff file is just a archive), search for that PN (e.g. PA005831109) inside the INFO file and copy the whole section and save it in notepad or somewhere else for now. You should have copied something that looks like this (part number, file names and version will likely be different for you, this is just an example):

[PA005831109]
version = 8.9.9.118194
file = gen3_tangerine_fusion_8.9.9.118194.bin
format = bin
ecc = 11b
cntrpdiversion = 0.0.15
cntrpdifile = avogadro_0.0.15.pdi
cntrpdiformat = pdi
nce = 1f80000
nceversion = 1.0.9.100764
ncefile = nce_1.0.9.100764.bin
errfw = 1f80000
errfwversion = 0.0.3
errfwfile = errfw_0.0.3.bin
optrom = 3c00000
optromversion = 4.3.7.1205
optromfile = uefi-noboot-3002_4.3.7.1205.rom

At this point there's only a three sections that change between different hardware revisions:

file = gen3_xxx_fusion_8.9.x.xxxxxxx.bin
cntrpdiversion = 0.0.xx
cntrpdifile = [filename].pdi

Open the official firmware file with 7zip, edit the INFO file inside and find a section that matches both the file (e.g. gen3_tangerine_fusion), and the cntrpdifile name that you copied from the OEM firmware file (e.g. cntrpdifile = avogadro). Copy the whole section, and paste it at the top of the file. Now rename the part number in the brackets - e.g. replace [5491-72320] with [PA005831109], then save and and flash again with the new firmware file

If you have any further questions or get stuck, feel free to drop by Discord (link in Support section below).

Support

Please join Discord iomemory-vsl4 or create an Issue.