Q. re. plugins

[dshaw619]

I see that a custom plugin script is run during phase 0, phase 1, and post-install of the --customize command just like a custom phase script. If one wanted a plugin to run during a --burn command, would it be the same script (perhaps with different parameters) or should you use a separate script?

What would the phase parameter indicate if running during a burn operation? Maybe 'post-install'?

What's the fundamental/practical difference between running a --burn command with a plugin vs. b0script & b1script?

p.s. Maybe I just am not familiar enough with navigating and searching Github, but it took some hunting to find sdm-plugin-template referenced in sdm Plugins. A link right when it was mentioned would be very helpful.

[gitbls]

Great questions! I just looked at the docs, and indeed...the questions you are asking are not answered there, so will do some wordsmithing there once the extent of the gap is clear.

Here's what I think is the missing piece: How Plugins work

When a --plugin plug-name switch is given, all 3 phases of the plugin plug-name are called, and in a context that is correct for each phase (Phase 0, Phase 1, post-install). This means that

• Phase 0 code can make references to the network, host storage, etc.
• Phase 1 and post-install code runs in a container and can only see local storage in the container (IMG, SSD/SD, etc)

In the case of --customize, the 3 phases of the plugin are called by sdm at the appropriate point:

• At the end of Phase 0
• At the end of Phase 1, after apt update but before sdm does the apps/xapps installs (so you can affect the apps/xapps installs if needed)
• At the end of the post-install phase before dropping to a command prompt, or exiting if --batch

By the way, this is exactly how custom phase scripts work, except that the custom phase script (invoked via --cscript) is called rather than a plugin (invoked via --plugin). You can think of plugins as a modular generalization of the custom phase script.

Yes, custom phase scripts AND plugins can be used together.

However, only Plugins can be used with --burn . In this case, you would specify on the --burn command line the plugins to run on the burned SSD/SD. After dd completes, the disk gets mounted and the requested plugins are run. First Phase 0 for all requested plugins, then Phase 1 and lastly, post-install.

This means you can build modular plugins for just about anything you want, and have a lot more control over how you build your deployed disks. I build all my server images from a single IMG, and add in various services when I burn one. This means I can easily put (for instance) the Samba server on my file server, but no Samba on all the rest.

I found that having a common configuration, with all my various bits and pieces (.bashrc, .ssh files, other tools/scripts, etc) already there makes me SO much more productive. It took me a while to get everything into my custom phase script, but I tried to improve it every time I ran into something else that I needed in the image.

[dshaw619]

Thanks for that comprehensive explanation. Very helpful. I didn't realize there was a phase 0, phase 1, and post-install also in the --burn process. I will focus on plugins.

[gitbls]

Thanks for that comprehensive explanation. Very helpful. I didn't realize there was a phase 0, phase 1, and post-install also in the --burn process. I will focus on plugins.

Phase 0 and Phase 1 names were coined very early in building sdm. A better way to think about these might be:

• Mounted disk environment: The disk being operated on (customize, when done standalone or as part of a --burn command) is mounted and available in phase 0 of custom phase scripts, plugins, and b0script. Since sdm is running in the context of your host OS, the disk is mounted in the host. sdm defines the env var $SDMPT to point to the mounted disk. IOW all bash references to the target must be done using $SDMPT/path/to/whatever.
• Container environment: sdm is running in a container (systemd-nspawn) with the disk mounted as the container's system disk. So, /path/to/whatever works. But, so does $SDMPT/path/to/whatever. If you have code you want to be able to run in both mounted disk mode and container mode, you can use $SDMPT liberally, since it's not defined in this environment. Note to self: code cleanup and documentation improvement opportunity

[gitbls]

Also, you asked about the difference between plugins and b0script vs b1script. b0script and b1script are a result of not spending enough time thinking about the problem when they were done. I was solving a specific problem and they were a great solution at the time.

Now that I've built up more capability in sdm (generalized spawning being the important aspect), plugins are a natural next step with much more capability than b0script/b1script.

But, there are still uses for b0script/b1script, such as used in sdm-gburn. While what it wants to do could all be done by building plugins, b0script/b1script are very convenient in this use case.

[gitbls]

I've added the wiki page Programming plugins and custom phase scripts, which captures my notes above on Phase 0/1/post-install.

Please have a read and let me know your thoughts. Thanks!

[dshaw619]

Some comments/suggestions:

• In 'Phase 0' section, paragraph 1: change "... $SDMPT that always points to the top of the directory tree" to "$SDMPT that always points to the top of the directory tree on the image being customized".

• In 'Phase 0' section, paragraph 2: change "... a correct reference to a file in the IMG in the mounted disk environment is $SDMPT/etc/rc.local" to "a correct reference to a file in the IMG in the mounted disk environment is, for example, $SDMPT/etc/rc.local".

• In 'Phase 0' section, paragraph 3: delete "and pretty much every other command". This seems confusing since it is followed by things you CAN do.

• In 'Phase 1' section, paragraph 1: change "... code running in the container cannot access the host OS file system" to "... code running in the container cannot access the host OS file system, only files on the image being customized".

• In same paragraph: change "... $SDMPT/etc/rc.local and /etc/rc.local are equivalent" to "... $SDMPT/etc/rc.local and /etc/rc.local are equivalent and both refer to the file on the image being customized".

• In 'post-install' section, paragraph 2: change "At the conclusion of Phase 1, sdm performs an apt update" to "At the conclusion of Phase 1, sdm performs an apt update and apt upgrade". [should this actually be apt full-upgrade?]

• In 'post-install' section: move paragraphs 2 & 3 to Phase 1 section. I think they would make more sense there and then lead into post-install.

• In 'when burning' section, paragraph 1: I find this confusing. You have to think in phases as what's done in the plugin script depends on which phase is running. Would "When burning, sdm runs the plugin script 3 times (Phase 0, Phase 1, post-install) with the same mounted disk or container environments as described above" be a correct statement and possible replacement for paragraphs 1 & 2 ?

• Would one typically have a separate plugin script for customizing and burning? Seems like it would be pretty easy to reference the appropriate one in the --customize or --burn command. If so, this might be worth mentioning.

• From where in the Wiki will this be linked? It can be difficult to find info in the Wiki (especially if one doesn't know to look for it) if it's not linked from a page where it might be first introduced or suggested. I don't think there is a way to search for all references to 'plugin'. It seems that searching with 'find a page' at the top of the truncated 'Pages' list on the top, right side of Wiki pages only finds pages with the search term in the title (e.g. a search for 'container' doesn't find anything. Searching for 'plugin' will find 'Programming plugins and custom phase scripts' and 'sdm Plugins', but not the mention on the 'Command Details' page.

• That was a long-winded and round-about way to get to my suggestions re. linking the page:

• On README > 'What else can sdm do?' > 'Other customizations' add a mention of plugins as the preferred method and link from there to 'sdm Plugins'.

• Rename 'sdm Plugins' to 'Plugins'. This would make it much easier to find when scanning through the list of Wiki pages. (would require changing the link on 'Command Details'.

• In the 'Writing your own plugin' section of 'sdm Plugins' add a link 'Programming plugins and custom phase scripts'; maybe even move the whole 'Writing your own plugin' section contents and just leave a brief mention and link.

Hope some of this is useful.

[gitbls]

WOW! Thanks for these!! I'll have a detailed look at them and update the docs. Tied up for a bit, so probably won't get to it for a couple of weeks.

[gitbls]

I'm in the process of updating the Wiki and will be adding the Wiki sources to the github.

I've added most of your comments above, still a few left to go through. I wanted to respond to one question you asked:

"Would one typically have a separate plugin script for customizing and burning? Seems like it would be pretty easy to reference the appropriate one in the --customize or --burn command. If so, this might be worth mentioning."

The great thing about plugins is that the exact same plugin code can be used during customization or burning. There's no need for two different versions. I can't envision a situation where this would be useful, but if plugin code needs to know whether it's part of a customize or a burn it can test something like [[ "$SDMNSPAWN" =~ "Burn" ]] which will be true if burning, false if otherwise. This can be used in all 3 phases, since the burn code carefully sets it to either Burn0 or Burn1.

[dshaw619]

I'm still trying to wrap my head around this. Why would one want to run "the exact same plugin code" during both customization and burning? Seems duplicative at best. I must be missing something.

[gitbls]

Well, you wouldn't, unless you like to do everything twice. BUT, there ARE reasons you might want to run the plugin at burn time INSTEAD of during customization.

For instance, I will be adding a plugin shortly for pistrong, which is an IPSEC VPN installer/tool. You probably don't want that on EVERY system you build, so you wouldn't use it during customize, but if you want to have it installed on your VPN server Pi, you can run the plugin when you burn that system's disk.

[1stcall]

When configuring multiple hosts, during the customization you should run the plugins you wish to apply to all hosts. during the burn stage, you should run the plugins wish to apply to the host that you are burning.

I hope this helps.

[dshaw619]

Both of those answers helped. Thanks!