Feature Request: Automatic device join to project or easy setup scritpt for mass deploys

[Macrofarad]

Description

For a system admin or engineer creating an image for manufacturing to deploy, it would be ideal to have a method to join new devices to a given project upon first boot such as a parameterized automatic setup script. This avoids the scenario where said SA/engineer has to spend all of their time configuring new devices into a project for large scale installs. Bonus points if it adds in new devices into a group for "pending approval" or similar.

[ZJvandeWeg]

Bonus points if it adds in new devices into a group for "pending approval" or similar.

I think this should be a second iteration feature, and the first iteration should allow a shared secret to assign a device to a team and project with a cookie cutter approach.

[knolleary]

For 1.2 we should do the design work on this and then see if it can be delivered in 1.2/1.3 timeframe.

[knolleary]

For a device to connect to the platform it needs a device.yml file that contains all of the various configuration options that are specific to the platform/device:

This is currently obtained by manually registering the device and downloading the information provided.

There are two possible models for supporting bulk device registration that we see in other platforms:

1. Support a CSV import of device details to register multiple in one go.
   • this is still a manual task and requires knowledge of all devices up front. The ux for then downloading all of the different sets of credentials is awkward.
2. Support an auto-provisioning mode where a device is given a single URL and token which can be used to self-register and obtain the full set of credentials.

This issue, as described, falls into the latter model, so that's what I'll focus on in the design. I wanted to acknowledge the other mode here because it is a valid alternative approach albeit for a slightly different use case.

The main challenge here is how to achieve this securely.

The high-level flow would be:

1. In Project Settings, a user can enable 'Automatic Device Provisioning'. When they do so, an AccessToken is generated with limit scope that only allows calls to certain device APIs. From here on, we'll call that the ProvisioningToken in this design.
2. The POST /api/v1/devices (create device) is updated so it can be called with a ProvisioningToken
   • it currently validates the session user is in the team. That logic will have to be updated to validate the ProvisioningToken is related to a project in that team.
   • When called with a ProvisioningToken, it should also auto-assign the device to the token's associated project.
3. The Device Agent can be configured with 'auto-provisioning' enabled - it will have to be provided the FlowForge base_url and the ProvisioningToken.
4. When the Agent starts, if auto-provisioning is enabled and it doesn't find a device.yml file, then call the /api/v1/devices endpoint using the ProvisioningToken to get its credentials and write to that file.
   • At this point, I think it will be business as normal for the Device Agent.

Considerations:

• Should the ProvisioningToken expire at all? Wary of long-lived tokens that could get out into the open. But equally, the goal here is to be able to minimise the need to touch devices in the field.
• If a user decides to turn off auto-provisioning on a project, the ProvisioningToken will get deleted. If they turn it back on, a new token will be generated - so any devices in the field that have the old token and have not yet provisioned themselves will be stranded.

[knolleary]

The CSV import option is covered by #1369

[Steve-Mcl]

Following 1:1 with Nick, it was explained there are considerations around billing that needs to be understood before this feature can be fully completed. This will be discussed in a product meeting and fed back.

[knolleary]

We've agreed to defer any questions about billing for devices for the future. So nothing is blocking making progress on this item.

[Steve-Mcl]

Early demo of Device Provisioning.

I would greatly appreciate quick feedback guys (@joepavitt @robmarcer @knolleary @ZJvandeWeg in particular - but all comments and suggestions are of course welcome)

device-provisioning-alpha.mp4

Questions to mull:

1. Should we bother with the Platform enable/disable (perhaps we could just leave it to the team owner to add (or not add) provisioning tokens)
2. Better wording and descriptions?
3. Should the "Team - Settings - Devices" just be hidden altogether if the platform has disabled Provisioning?
4. Should we permit setting expiry?
   1. Currently, it is shown in the tokens table but I have not yet implemented an entry field (see future proposal around MAC/IP filtering which may make more sense than an expiry)
5. Lastly, there is the ability to Set/Change/Unset the auto assigned project on the token.
   1. The logic here was: existing devices (as yet rolled out) could be auto assigned to a newer version project (or not) without having to generate a new token (and subsequently have to fix up the creds on 100+ new devices the team has previously prepared sitting on a shelf)

Future Thoughts:

• Permitting Enable/Disable option per token would mean provisioning could be delayed/paused/suspended without deleting the token (should there be a need to investigate something / fix something external / add licences etc)
• MAC/IP/Hostname filter for limiting device joins (e.g. should someone from another team duplicate the device as a base image and forgets to update the token - a nightmare waiting to happen).

[knolleary]

Should we bother with the Platform enable/disable

No - it should just be enabled. If a requirement comes up in the future for an admin to be able to disable the feature we can revisit, but for now, this should just be available.

Should the "Team - Settings - Devices" just be hidden altogether if the platform has disabled Provisioning?

Given me reply above, this is N/A...

Better wording and descriptions?

Need to step through it in more detail, but 'auto deploy project' I think should change to use 'assign' rather than 'deploy', because that is the action the user has to take if they don't select this option.

Should we permit setting expiry?

I think we discussed that should be a future feature. It isn't in scope for what is required in the first iteration. Please raise a story.

Lastly, there is the ability to Set/Change/Unset the auto assigned project on the token.

Not clear what the question is. Although the video doesn't show it being done, I noticed an 'edit token' option in the context menu. So I presume you have already done the work to support changing the project? That wasn't in the original scope, but if you've already implemented it then are you asking if that should be removed?

[Steve-Mcl]

Lastly, there is the ability to Set/Change/Unset the auto assigned project on the token.

Not clear what the question is. Although the video doesn't show it being done, I noticed an 'edit token' option in the context menu. So I presume you have already done the work to support changing the project? That wasn't in the original scope, but if you've already implemented it then are you asking if that should be removed?

I guess so yes. Since I took an existing dialog (which had edit capability) it kinda just "fell into place". I did consider removing the edit ability but it grew on me as a very useful feature.

Should we bother with the Platform enable/disable

No - it should just be enabled.

Thanks - will remove.

Should we permit setting expiry?

I think we discussed that should be a future feature.

Agreed - will remove column from table.

[Steve-Mcl]

Status Post

As of 2023/02/06)

outdated - click to show

PRs

• Forge API: Device provisioning API #1659
• Forge UI: Device Provisioning UI #1664
• Agent: WIP
• Docs: TODO

As of 2023/02/09)

PRs

• Forge API: Device provisioning API #1659
• Forge UI: Device Provisioning UI #1664
• Agent: Auto Device Provisioning device-agent#56
• Docs:
  • FF: docs: info on Device Provisioning and Auto Registration #1687
  • Agent: Part of Auto Device Provisioning device-agent#56 in commit 97b18bb

[Steve-Mcl]

Tested locally and on staging.
test1: a provisioning token without a project set - device created in forge, device.yml updated on device
test2: a provisioning token updated to set project - device auto updated
test3: a new device was configured with the provisioning token - device created in forge, device.yml updated on device, device switched out of provisioning mode and downloaded the snapshot

(Used an npmrc to permit the project nodes and file nodes to be downloaded to the device from staging)