Support devices with multiple firmware slots

[fhunleth]

Currently NervesHub assumes one firmware image per device. There are a few reasons why it would be useful to have multiple firmware images for a device:

1. Reduce size of firmware updates. For example, one firmware update slot could contain the bulk of the code and a second one has data tables that are small, but update frequently. Users could update the second one independent of the first one.
2. Support pre-built kiosk images. Kiosk images using chromium are quite large. It might be nice to have a setup where you could have a main kiosk image that contains chromium and then a second slot that has the Elixir/HTML/JS to run the kiosk.
3. Support devices that are composed from multiple smaller devices. The idea would be that NervesHub could send down updates for the internal devices independent of the main one. Currently, the main device would need to embed the firmware for the internal devices and update them on its own.

In addition to updating the NervesHub database schema and APIs to understand device slots, it will also be necessary to modify how deployments are handled. I think there are two options:

1. A deployment has a firmware image associated with each slot. NervesHub's goal would be to make a device that matches a deployment have the same firmware in each of the slots.
2. Require the user to make a deployment for each device slot and add a match condition to allow slots to depend on each other.

My leaning is to the first, since it seems easier to understand the final state of the device.

Additionally, the order of firmware updates to a device should be deterministic. Do slots get updated in order? I.e., slot 0 is first, then slot 1, etc. Or do slots get updated in reverse order? Or do we let the user assign a priority to each slot so that a custom ordering can be defined in a deployment. I assume that it will be difficult for users to remove the coupling between images in different slots, so some accommodation will be needed.

[fhunleth]

@danielspofford Anything else that you'd add or change here?

[danielspofford]

1. Is the number of slots dictated at the product or deployment level?
2. Given a physical device currently operating with 3 slots, can deploys take it to 4 slots? 1?
3. Are we assuming that one individual slot will be the "boot" slot?
4. It would be nice to allow all slots to download their update, and only then reboot. Likely the most flexible thing here is to make NervesHub only reboot upon firmware download if no client (NervesHub client, the callback handler) is defined such that it is the default behavior. But if one is defined, then whether a reboot occurs or not is in the client's hands. Docs would need to make this clear.
5. Does each slot have its own A/B partition?
6. I would like to talk through how/if this would facilitate updating x OTP applications within an OTP release that has x + pos_integer() OTP applications in it. Perhaps this could be achieved by having Shoehorn run some non-Nerves / project-specific logic that purges and repopulates the release's ebin/some_app dir with the relevant .app and .beam files. Or it could happen post-download but pre-reboot in app code, but what if it fails 🤔. Maybe, hopefully, I'm overthinking this or there is an easier way.

[fhunleth]

I view the scope of this issue is limited to adding a slot index to firmware update notifications sent to device and the ramifications of adding an index to NervesHub. I don't see any reason why NervesHub would need to limit the number of slots per device or assign meaning to slots. The number of slots and the semantics behind the slots seems very application-specific. NervesHub does need knowledge about how slots depend on each other so that it doesn't upgrade one slot in an incompatible way.

All of the other questions that you asked are application-specific - A/B partition, rebooting, boot slots, switching slot count between updates, and OTP-knowledge. For example, maybe you don't care about power-safe updates for a slot, then there's no need to waste space on A/B partitions. If you don't need to reboot to use an update to a slot, then don't reboot. I personally feel that NervesHub should stay out of these decisions so as to not force users to pick a strategy that doesn't work with their platform. It seems really hard to pick something that works for everyone.

[danielspofford]

I agree we don't want any of this implementation to be application-specific. I was asking questions through the lens of a specific application as a means of exploring what flexibility we have and how you saw some of this playing out. Your response is what I was hoping for :).

Some of these things are impacted by the current behavior of NervesHub and I am curious if it will be updated or if some of this pushes you "off the ranch". Your comment kind of makes me think the latter: NervesHub is useful for the traditional case of today, but if you want multiple slots, you may have to fork/write your own NervesHub to consume this info and act accordingly. For example when NervesHub downloads an update today I believe it streams it directly into the inactive partition, yes?

I think reverse order makes sense WRT deterministic slot updating, e.g. 3, 2, 1, 0.

1. A deployment has a firmware image associated with each slot. NervesHub's goal would be to make a device that matches a deployment have the same firmware in each of the slots.

☝️ makes sense to me as well, at least for a good amount of time.

[joshk]

With the advent of Archives, is this issue still relevant?

[fhunleth]

I think we can close it.