Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use init() to handle plugin invocation #5726

Merged
merged 2 commits into from
Jun 19, 2019
Merged

Use init() to handle plugin invocation #5726

merged 2 commits into from
Jun 19, 2019

Conversation

notnoop
Copy link
Contributor

@notnoop notnoop commented May 19, 2019

Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI
commands to be handled by command CLI framework. Plugin launchers use
discover.NomadBinary() to identify the binary and start it.

This has few downsides: The trivial one is that when running tests, one
must re-compile the nomad binary as the tests need to invoke the nomad
executable to start plugin. This is frequently overlooked, resulting in
puzzlement.

The more significant issue with executor in particular is in relation
to external driver:

  • Plugin must identify the path of invoking nomad binary, which is not
    trivial; discvoer.NomadBinary() now returns the path to the plugin
    rather than to nomad, preventing external drivers from launching
    executors.

  • The external driver may get a different version of executor than it
    expects (specially if we make a binary incompatible change in future).

This commit addresses both downside by having the plugin invocation
handling through an init() call, similar to how libcontainer init
handler is done in [1] and recommened by libcontainer [2]. init()
will be invoked and handled properly in tests and external drivers.

For external drivers, this change will cause external drivers to launch
the executor that's compiled against.

There a are a couple of downsides to this approach:

  • These specific packages (i.e executor, logmon, and dockerlog) need to
    be careful in use of init(), package initializers. Must avoid having
    command execution rely on any other init in the package. I prefixed
    files with z_ (golang processes files in lexical order), but ensured
    we don't depend on order.
  • The command handling is spread in multiple packages making it a bit
    less obvious how plugin starts are handled.

[1] drivers/shared/executor/libcontainer_nsenter_linux.go
[2] https://github.com/opencontainers/runc/tree/eb4aeed24ffbf8e2d740fafea39d91faa0ee84d0/libcontainer#using-libcontainer

@notnoop notnoop changed the title Use init to handle plugin invocation Use init() to handle plugin invocation May 19, 2019
@notnoop notnoop requested a review from nickethier May 19, 2019 18:08
Currently, nomad "plugin" processes (e.g. executor, logmon, docker_logger) are started as CLI
commands to be handled by command CLI framework.  Plugin launchers use
`discover.NomadBinary()` to identify the binary and start it.

This has few downsides: The trivial one is that when running tests, one
must re-compile the nomad binary as the tests need to invoke the nomad
executable to start plugin.  This is frequently overlooked, resulting in
puzzlement.

The more significant issue with `executor` in particular is in relation
to external driver:

* Plugin must identify the path of invoking nomad binary, which is not
trivial; `discvoer.NomadBinary()` now returns the path to the plugin
rather than to nomad, preventing external drivers from launching
executors.

* The external driver may get a different version of executor than it
expects (specially if we make a binary incompatible change in future).

This commit addresses both downside by having the plugin invocation
handling through an `init()` call, similar to how libcontainer init
handler is done in [1] and recommened by libcontainer [2].  `init()`
will be invoked and handled properly in tests and external drivers.

For external drivers, this change will cause external drivers to launch
the executor that's compiled against.

There a are a couple of downsides to this approach:
* These specific packages (i.e executor, logmon, and dockerlog) need to
be careful in use of `init()`, package initializers.  Must avoid having
command execution rely on any other init in the package.  I prefixed
files with `z_` (golang processes files in lexical order), but ensured
we don't depend on order.
* The command handling is spread in multiple packages making it a bit
less obvious how plugin starts are handled.

[1] drivers/shared/executor/libcontainer_nsenter_linux.go
[2] https://github.com/opencontainers/runc/tree/eb4aeed24ffbf8e2d740fafea39d91faa0ee84d0/libcontainer#using-libcontainer
Copy link
Member

@nickethier nickethier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM! Can you add comments above the inits about what they are and why they are there. The PR description explains this well, it would be good to have it in the source for future devs.

@notnoop notnoop merged commit 6708a0c into master Jun 19, 2019
@notnoop notnoop deleted the b-plugins-via-init branch June 19, 2019 01:09
notnoop pushed a commit that referenced this pull request Jun 19, 2019
@github-actions
Copy link

github-actions bot commented Feb 8, 2023

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants