Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

tooling for a customized kernel #77

Closed
egernst opened this issue Apr 3, 2018 · 1 comment
Closed

tooling for a customized kernel #77

egernst opened this issue Apr 3, 2018 · 1 comment
Labels
enhancement Improvement to an existing feature feature New functionality

Comments

@egernst
Copy link
Member

egernst commented Apr 3, 2018

Background

Over last week I've gone through thought-process on a couple of different e2e use-cases that involved device-passthrough for 'interesting' container workloads. Examples include:

  • remote write to NVMe over fabric via RDMA (mlx NIC)
  • openCL application: run hash cat on NVIDIA GPU

In both cases, the setup includes loading a software stack on the system which will ultimately make use of out-of-tree kernel drivers, or at least using DKMS to apply the latest drivers they require. This works well if you "own" the host or virtual machine, but isn't very straight forward in our statically configured Kata kernel/image. I think Kata Containers provide a great benefit here (being able to have a guest kernel that has these features enabled, taking the burden away from the host system and or IaaS/VM provisioning). We need to support this well, though.

Feature request

I think we should have a way to be able to streamline, or at least document, the workflow of pulling particular DKMS / out of tree drivers and build them against the Kata kernel, and include and automatically load this from our kata-containers image.

Getting graphics card device pass-through and making use of an nvidia.ko should be an easy first example. I think?
@egernst
Copy link
Member Author

egernst commented Apr 3, 2018

/cc @jcvenegas @bergwolf @gnawux @devimc

@devimc devimc added enhancement Improvement to an existing feature feature New functionality labels Jul 17, 2019
devimc pushed a commit to devimc/kata-agent that referenced this issue Jul 30, 2019
agent: add support for loading kernel modules
2e26843 
Some containers may require HW devices to improve its performance. In Kata
containers these devices can be passed though VFIO and used in the container
only if the guest kernel has the drivers built-in for them. When kernel
fragment support landed (kata-containers/packaging#646), kata lost support
for several HW devices like Intel GPUs, this was expected since one of the
goals of this change was to have a minimal kernel in order to improve boot
time, memory footprint and reduce the attack surface by enabling just the
features required to run kata containers.
Currently if the users want to use a HW device in kata container, He/She has
to build a custom kernel and create a custom image that includes the modules
that will be loaded by systemd. From user's perspective this is a pain, since
not all users know how to create a custom image or configure/build a custom
kernel.
Same as the `debug_console` feature (kata-containers#547), the goal of
this change is to facilitate the things to the final user and *do not* depend
on systemd.

fixes kata-containers#615

see kata-containers/osbuilder#77

Signed-off-by: Julio Montes <julio.montes@intel.com>
@devimc devimc mentioned this issue Jul 30, 2019
Merged
devimc pushed a commit to devimc/kata-agent that referenced this issue Jul 31, 2019
agent: add support for loading kernel modules
75dbf82 
Some containers may require HW devices to improve its performance. In Kata
containers these devices can be passed though VFIO and used in the container
only if the guest kernel has the drivers built-in for them. When kernel
fragment support landed (kata-containers/packaging#646), kata lost support
for several HW devices like Intel GPUs, this was expected since one of the
goals of this change was to have a minimal kernel in order to improve boot
time, memory footprint and reduce the attack surface by enabling just the
features required to run kata containers.
Currently if the users want to use a HW device in kata container, He/She has
to build a custom kernel and create a custom image that includes the modules
that will be loaded by systemd. From user's perspective this is a pain, since
not all users know how to create a custom image or configure/build a custom
kernel.
Same as the `debug_console` feature (kata-containers#547), the goal of
this change is to facilitate the things to the final user and *do not* depend
on systemd.

fixes kata-containers#615

see kata-containers/osbuilder#77

Signed-off-by: Julio Montes <julio.montes@intel.com>
devimc pushed a commit to devimc/kata-agent that referenced this issue Jul 31, 2019
agent: add support for loading kernel modules
fe8a7e0 
Some containers may require HW devices to improve its performance. In Kata
containers these devices can be passed though VFIO and used in the container
only if the guest kernel has the drivers built-in for them. When kernel
fragment support landed (kata-containers/packaging#646), kata lost support
for several HW devices like Intel GPUs, this was expected since one of the
goals of this change was to have a minimal kernel in order to improve boot
time, memory footprint and reduce the attack surface by enabling just the
features required to run kata containers.
Currently if the users want to use a HW device in kata container, He/She has
to build a custom kernel and create a custom image that includes the modules
that will be loaded by systemd. From user's perspective this is a pain, since
not all users know how to create a custom image or configure/build a custom
kernel.
Same as the `debug_console` feature (kata-containers#547), the goal of
this change is to facilitate the things to the final user and *do not* depend
on systemd.
The list of kernel modules and their options are set through gRPC when the
sandbox is created.

fixes kata-containers#615

see kata-containers/osbuilder#77

Signed-off-by: Julio Montes <julio.montes@intel.com>
devimc pushed a commit to devimc/kata-agent that referenced this issue Aug 1, 2019
agent: add support for loading kernel modules
ad72fe8 
Some containers may require HW devices to improve its performance. In Kata
containers these devices can be passed though VFIO and used in the container
only if the guest kernel has the drivers built-in for them. When kernel
fragment support landed (kata-containers/packaging#646), kata lost support
for several HW devices like Intel GPUs, this was expected since one of the
goals of this change was to have a minimal kernel in order to improve boot
time, memory footprint and reduce the attack surface by enabling just the
features required to run kata containers.
Currently if the users want to use a HW device in kata container, He/She has
to build a custom kernel and create a custom image that includes the modules
that will be loaded by systemd. From user's perspective this is a pain, since
not all users know how to create a custom image or configure/build a custom
kernel.
Same as the `debug_console` feature (kata-containers#547), the goal of
this change is to facilitate the things to the final user and *do not* depend
on systemd.
The list of kernel modules and their parameters are set through gRPC when the
sandbox is created.

fixes kata-containers#615

see kata-containers/osbuilder#77

Signed-off-by: Julio Montes <julio.montes@intel.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement Improvement to an existing feature feature New functionality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@devimc @egernst @jodh-intel