Add initial support for the OpenBSD operating system #712

reyk · 2017-05-12T20:29:12Z

There are some differences between FreeBSD, OpenBSD, and Linux.

Notes:

OpenBSD ships LibreSSL, but WALinuxAgent needs OpenSSL for CMS, so
use the openssl port and the "eopenssl" binary instead.
Don't run the custom DHCP client but parse /var/db/dhclient.leases.hvn0.
OpenBSD's lease file uses a modified syntax.
OpenBSD does not have /proc. WALinuxAgent should never assume that
the /proc filesystem is available.
OpenBSD does not have sudo, but its replacement doas. The OpenBSD
class implements support for modifying the doas.conf file.
Unlike FreeBSD, OpenBSD supports and mounts UDF DVDs just fine.
Create a swap partition instead of a swap file on the resource disk.
Many other minor changes for OpenBSD

TODO:

The /proc checks needs to be replaced with pgrep/ps etc. checks for
OpenBSD. For now it just checks if /proc is available or returns
without error.
Make sure to install waagent with --register-service as setuptools
will not set the permissions of /etc/rc.d/waagent correctly (the file
has to be executable to be used by OpenBSD's rc system).

msftclas · 2017-05-12T20:29:16Z

@reyk,
Thanks for your contribution.
To ensure that the project team has proper rights to use your work, please complete the Contribution License Agreement at https://cla.microsoft.com.

It will cover your contributions to all Microsoft-managed open source projects.
Thanks,
Microsoft Pull Request Bot

reyk · 2017-05-12T20:37:47Z

azurelinuxagent/common/osutil/openbsd.py

+                return
+            if retry < max_retry - 1:
+                mountlist = shellutil.run_get_output("/sbin/mount")[1]
+                logger.info("mountlist: '{0}'", mountlist)


line 254 is a debug message that should be removed.

msftclas · 2017-05-12T21:13:10Z

@reyk, thanks for signing the contribution license agreement. We will now validate the agreement and then the pull request.

Thanks, Microsoft Pull Request Bot

lordgordon · 2017-05-17T12:28:04Z

Hi,

thank you for your work, really appreciated. Do you think your changes are able to make WALinuxAgent working fine with LibreSSL on a FreeBSD?

Thanks

reyk · 2017-05-17T12:51:08Z

@lordgordon: As LibreSSL comes from OpenBSD, I looked into using it directly. But we have to use the OpenSSL port instead because LibreSSL is not compatible with WALinuxAgent (pkg_add openssl; binary is installed as "eopenssl" on OpenBSD) .

Details: the Azure wire protocol uses CMS to encrypt the certificates payload that includes the SSH public keys. I don't know why public keys have to be encrypted this way, but the agent needs "openssl cms" to decrypt them. But CMS has been removed from LibreSSL because it was poor and complex code and not even enabled in OpenSSL for a long time (http://marc.info/?l=openbsd-cvs&m=147301176512716&w=2).

@mbelop and me managed to port CMS as a stand-alone binary that links with LibreSSL, but it is a big and ugly port that needs to provide its own versions for many internal functions within libcrypto. I can try to release it separately, but the best solution for OpenBSD and maybe FreeBSD is to use upstream OpenSSL for WALinuxAgent. I know that kind of hurts when you try to move away from using it.

I also wonder why Azure doesn't just include the SSH public keys as Values in the ovf-env.xml file; this way we wouldn't need CMS.

lordgordon · 2017-05-17T13:01:51Z

@reyk thank you very much for your detailed explanation. I agree, the real issue here is Azure's forcing to use CMS.

As I mainly need LibreSSL for nginx I think the easiest workaround is to install LibreSSL with nginx in a jail.

KylieLiang · 2017-05-22T07:16:12Z

@hglkrijger Could you please review the patch from Reyk? They develop the patch to enable waagent for OpenBSD.

hglkrijger

generally lgtm. @brendandixon any other comments?

hglkrijger · 2017-05-22T18:38:50Z

config/openbsd/waagent.conf

+# OS.EnableRDMA=y
+
+# Enable or disable goal state processing auto-update, default is enabled
+AutoUpdate.Enabled=n


I recommend leaving this enabled

Sure, I can enable it again. I turned it off because it wouldn't work before the merge.

hglkrijger · 2017-05-22T18:39:10Z

azurelinuxagent/pa/deprovision/default.py

@@ -95,6 +104,9 @@ def del_dhcp_lease(self, warnings, actions):
        actions.append(DeprovisionAction(fileutil.rm_files, ["/var/db/dhclient.leases.hn0",
                                                             "/var/lib/NetworkManager/dhclient-*.lease"]))

+        # For OpenBSD
+        actions.append(DeprovisionAction(fileutil.rm_files, ["/var/db/dhclient.leases.hvn0"]))


Should this have a wildcard?

I modeled it after the FreeBSD one above that only deletes hn0 (I don't know what NetworkManager is). But, yes, that should be OK to remove /var/db/dhclient.leases.hvn* in case that there was any additional configured NIC.

hglkrijger · 2017-05-22T18:41:41Z

azurelinuxagent/common/version.py

+    if os.path.isdir("/proc"):
+        pids = [pid for pid in os.listdir('/proc') if pid.isdigit()]
+    else:
+        pids = []


What is the behavior of waagent --version here?

WALinuxAgent-2.2.11 running on openbsd 6.1 Python: 2.7.13 Goal state agent: 2.2.11

Could you introduce a new osutil callback that allows OS without /proc to implement their own method to get a running process? On OpenBSD, we could implement a callback with the pgrep system utility. There are two places that could use it instead of accessing /proc directly: set_goal_state_agent() in common/version.py and validate_cloud_init() in pa/provision/default.py.

@hglkrijger I rebased and force-pushed an update based your recommendations, see the individual commits for the changes. Can we get it merged now?

@reyk, sure, that sounds reasonable. Would you please open an issue to track?

There are some differences between FreeBSD, OpenBSD, and Linux. Notes: - OpenBSD ships LibreSSL, but WALinuxAgent needs OpenSSL for CMS, so use the openssl port and the "eopenssl" binary instead. - Don't run the custom DHCP client but parse /var/db/dhclient.leases.hvn0. OpenBSD's lease file uses a modified syntax. - OpenBSD does not have /proc. WALinuxAgent should never assume that the /proc filesystem is available. - OpenBSD does not have sudo, but its replacement doas. The OpenBSD class implements support for modifying the doas.conf file. - Unlike FreeBSD, OpenBSD supports and mounts UDF DVDs just fine. - Create a swap partition instead of a swap file on the resource disk. - Many other minor changes for OpenBSD TODO: - The /proc checks needs to be replaced with pgrep/ps etc. checks for OpenBSD. For now it just checks if /proc is available or returns without error. Make sure to install waagent with --register-service as setuptools will not set the permissions of /etc/rc.d/waagent correctly (the file has to be executable to be used by OpenBSD's rc system).

The OpenBSD disk handler is an addition, not a replacement. Found by jonathangray

pylint found one bug in setting the password; other changes are just for the style.

Issue reported by Lili Deng.

@hglkrijger

Pointed out by @hglkrijger

@hglkrijger

Requested by @hglkrijger

hglkrijger

lgtm. @brendandixon any comments?

msftclas added the cla-required label May 12, 2017

reyk commented May 12, 2017

View reviewed changes

msftclas added cla-signed and removed cla-required labels May 12, 2017

hglkrijger reviewed May 22, 2017

View reviewed changes

reyk added 7 commits May 23, 2017 13:57

Remove debug statement that was use to fix mount_dvd on OpenBSD.

041d44a

I accidentally replaced the FreeBSD resource disk handler.

f1dcf4f

The OpenBSD disk handler is an addition, not a replacement. Found by jonathangray

pylint

0936198

pylint found one bug in setting the password; other changes are just for the style.

strip newline from generated password hash.

8c117af

Issue reported by Lili Deng.

Keep AutoUpdate enabled, it was disabled for initial porting.

2f758e2

Pointed out by @hglkrijger

Use a wildcard and delete all DHCP lease files on *BSD.

169bd2d

Requested by @hglkrijger

reyk force-pushed the waagent-openbsd branch from 213d53c to 169bd2d Compare May 23, 2017 12:01

hglkrijger approved these changes May 23, 2017

View reviewed changes

hglkrijger merged commit aad6267 into Azure:master May 30, 2017

hglkrijger added this to the v2.2.13 milestone May 30, 2017

reyk deleted the waagent-openbsd branch June 22, 2017 08:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add initial support for the OpenBSD operating system #712

Add initial support for the OpenBSD operating system #712

reyk commented May 12, 2017

msftclas commented May 12, 2017

reyk May 12, 2017 •

edited

Loading

msftclas commented May 12, 2017

lordgordon commented May 17, 2017

reyk commented May 17, 2017

lordgordon commented May 17, 2017

KylieLiang commented May 22, 2017

hglkrijger left a comment

hglkrijger May 22, 2017

reyk May 23, 2017

hglkrijger May 22, 2017

reyk May 23, 2017

hglkrijger May 22, 2017

reyk May 23, 2017

reyk May 23, 2017

hglkrijger May 23, 2017

hglkrijger left a comment

Add initial support for the OpenBSD operating system #712

Add initial support for the OpenBSD operating system #712

Conversation

reyk commented May 12, 2017

msftclas commented May 12, 2017

reyk May 12, 2017 • edited Loading

Choose a reason for hiding this comment

msftclas commented May 12, 2017

lordgordon commented May 17, 2017

reyk commented May 17, 2017

lordgordon commented May 17, 2017

KylieLiang commented May 22, 2017

hglkrijger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hglkrijger left a comment

Choose a reason for hiding this comment

reyk May 12, 2017 •

edited

Loading