From 455509b45b8663b46cdbc9194058cfc02c77cd0f Mon Sep 17 00:00:00 2001 From: Stanislav Kosorin Date: Thu, 22 Aug 2024 12:38:32 +0200 Subject: [PATCH] Add/update installation scripts, update README with versions --- README.md | 30 +- requirements.txt | 2 +- scripts/README.md | 5 + scripts/containers.conf | 896 ++++++++++++++++++++++++++++++++++++++ scripts/install_criu.sh | 61 ++- scripts/install_p4c.sh | 44 ++ scripts/install_pi.sh | 41 ++ scripts/install_podman.sh | 49 +++ 8 files changed, 1094 insertions(+), 34 deletions(-) create mode 100644 scripts/README.md create mode 100644 scripts/containers.conf create mode 100755 scripts/install_p4c.sh create mode 100755 scripts/install_pi.sh create mode 100755 scripts/install_podman.sh diff --git a/README.md b/README.md index eef679a..16fdbf3 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,6 @@ This is the repository for the Google Summer of Code project [P4-Enabled Contain - [Installation](#installation) - [Prerequisites](#prerequisites) - [Install Python Dependencies](#install-python-dependencies) - - [Installing PI](#installing-pi) - [Build Custom Podman Images](#build-custom-podman-images) - [Running examples](#running-examples) @@ -16,10 +15,15 @@ This is the repository for the Google Summer of Code project [P4-Enabled Contain ## Installation ### Prerequisites -- [Podman](https://podman.io/docs/installation) -- [Python 3](https://www.python.org/downloads/) -- [pip](https://pip.pypa.io/en/stable/installation/) -- [P4 Compiler (p4c)](https://github.com/p4lang/p4c) +- [Python 3](https://www.python.org/downloads/) (3.10+) +- [pip3](https://pip.pypa.io/en/stable/installation/) +- [CRIU](https://criu.org/Main_Page) (v3.19) +- [crit](https://criu.org/CRIT) (v3.19) +- [P4 Compiler](https://github.com/p4lang/p4c) (v1.2.4.14) +- [PI](https://github.com/p4lang/PI) +- [Podman](https://podman.io/docs/installation) (v5.2.1) + +We have provided [scripts](scripts) to install CRIU (with crit), the P4 compiler, PI, and Podman. The scripts have been tested on Ubuntu 22.04 and 24.04 and are not guaranteed to work on all machines. If you encounter any issues, please refer to the official documentation of the respective projects. ### Install Python Dependencies ```bash @@ -28,22 +32,6 @@ source .venv/bin/activate pip install -r requirements.txt ``` -### Installing PI -This project uses the P4Runtime API to communicate with the switch. The P4Runtime API is implemented in the [P4Runtime Interface (PI)](https://github.com/p4lang/PI). - -When installing PI, make sure to configure with the `--with-proto` flag to compile proto files and `--with-python_prefix=/path/to/this/repo/.venv` to install the p4 library in your virtual environment. For example: -```bash -./configure --with-proto --with-python_prefix=/absolute/path/to/p4containerflow/.venv -``` -After running: -```bash -sudo make install -``` -The p4 library files in your `.venv` will be owned by the root user. Make sure to change owner by running: -```bash -sudo chown -R $USER .venv -``` - ### Build Custom Podman Images ```bash make images diff --git a/requirements.txt b/requirements.txt index a042e43..957c3c9 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,6 +5,6 @@ grpcio==1.64.1 itsdangerous==2.2.0 Jinja2==3.1.4 MarkupSafe==2.1.5 -protobuf==5.27.2 +protobuf==3.20.0 scapy==2.5.0 Werkzeug==3.0.3 diff --git a/scripts/README.md b/scripts/README.md new file mode 100644 index 0000000..f8aa8ac --- /dev/null +++ b/scripts/README.md @@ -0,0 +1,5 @@ +# Scripts + +This directory contains scripts to install CRIU, the P4 compiler, PI and Podman. The scripts have been tested on Ubuntu 22.04 and 24.04 and are not guaranteed to work on all machines. If you encounter any issues, please refer to the official documentation of the respective projects. + +The `edit_files_img.py` script is used to edit the `files.img` file in a container checkpoint by rewriting the socket IP address. This is used by some of the examples to successfully restore containers with a changing IP address. \ No newline at end of file diff --git a/scripts/containers.conf b/scripts/containers.conf new file mode 100644 index 0000000..ce63ade --- /dev/null +++ b/scripts/containers.conf @@ -0,0 +1,896 @@ +# The containers configuration file specifies all of the available configuration +# command-line options/flags for container engine tools like Podman & Buildah, +# but in a TOML format that can be easily modified and versioned. + +# Please refer to containers.conf(5) for details of all configuration options. +# Not all container engines implement all of the options. +# All of the options have hard coded defaults and these options will override +# the built in defaults. Users can then override these options via the command +# line. Container engines will read containers.conf files in up to three +# locations in the following order: +# 1. /usr/share/containers/containers.conf +# 2. /etc/containers/containers.conf +# 3. $XDG_CONFIG_HOME/containers/containers.conf or +# $HOME/.config/containers/containers.conf if $XDG_CONFIG_HOME is not set +# Items specified in the latter containers.conf, if they exist, override the +# previous containers.conf settings, or the default settings. + +[containers] + +# List of annotation. Specified as +# "key = value" +# If it is empty or commented out, no annotations will be added +# +#annotations = [] + +# Used to change the name of the default AppArmor profile of container engine. +# +#apparmor_profile = "container-default" + +# The hosts entries from the base hosts file are added to the containers hosts +# file. This must be either an absolute path or as special values "image" which +# uses the hosts file from the container image or "none" which means +# no base hosts file is used. The default is "" which will use /etc/hosts. +# +#base_hosts_file = "" + +# List of cgroup_conf entries specifying a list of cgroup files to write to and +# their values. For example `memory.high=1073741824` sets the +# memory.high limit to 1GB. +# cgroup_conf = [] + +# Default way to to create a cgroup namespace for the container +# Options are: +# `private` Create private Cgroup Namespace for the container. +# `host` Share host Cgroup Namespace with the container. +# +#cgroupns = "private" + +# Control container cgroup configuration +# Determines whether the container will create CGroups. +# Options are: +# `enabled` Enable cgroup support within container +# `disabled` Disable cgroup support, will inherit cgroups from parent +# `no-conmon` Do not create a cgroup dedicated to conmon. +# +#cgroups = "enabled" + +# List of default capabilities for containers. If it is empty or commented out, +# the default capabilities defined in the container engine will be added. +# +#default_capabilities = [ +# "CHOWN", +# "DAC_OVERRIDE", +# "FOWNER", +# "FSETID", +# "KILL", +# "NET_BIND_SERVICE", +# "SETFCAP", +# "SETGID", +# "SETPCAP", +# "SETUID", +# "SYS_CHROOT", +#] + +# A list of sysctls to be set in containers by default, +# specified as "name=value", +# for example:"net.ipv4.ping_group_range=0 0". +# +default_sysctls = [ +"net.ipv4.ping_group_range=0 0", +] + +# A list of ulimits to be set in containers by default, specified as +# "=:", for example: +# "nofile=1024:2048" +# See setrlimit(2) for a list of resource names. +# Any limit not specified here will be inherited from the process launching the +# container engine. +# Ulimits has limits for non privileged container engines. +# +#default_ulimits = [ +# "nofile=1280:2560", +#] + +# List of devices. Specified as +# "::", for example: +# "/dev/sdc:/dev/xvdc:rwm". +# If it is empty or commented out, only the default devices will be used +# +#devices = [] + +# List of default DNS options to be added to /etc/resolv.conf inside of the container. +# +#dns_options = [] + +# List of default DNS search domains to be added to /etc/resolv.conf inside of the container. +# +#dns_searches = [] + +# Set default DNS servers. +# This option can be used to override the DNS configuration passed to the +# container. The special value "none" can be specified to disable creation of +# /etc/resolv.conf in the container. +# The /etc/resolv.conf file in the image will be used without changes. +# +#dns_servers = [] + +# Environment variable list for the conmon process; used for passing necessary +# environment variables to conmon or the runtime. +# +#env = [ +# "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", +#] + +# Pass all host environment variables into the container. +# +#env_host = false + +# Set the ip for the host.containers.internal entry in the containers /etc/hosts +# file. This can be set to "none" to disable adding this entry. By default it +# will automatically choose the host ip. +# +# NOTE: When using podman machine this entry will never be added to the containers +# hosts file instead the gvproxy dns resolver will resolve this hostname. Therefore +# it is not possible to disable the entry in this case. +# +#host_containers_internal_ip = "" + +# Default proxy environment variables passed into the container. +# The environment variables passed in include: +# http_proxy, https_proxy, ftp_proxy, no_proxy, and the upper case versions of +# these. This option is needed when host system uses a proxy but container +# should not use proxy. Proxy environment variables specified for the container +# in any other way will override the values passed from the host. +# +#http_proxy = true + +# Run an init inside the container that forwards signals and reaps processes. +# +#init = false + +# Container init binary, if init=true, this is the init binary to be used for containers. +# If this option is not set catatonit is searched in the directories listed under +# the helper_binaries_dir option. It is recommended to just install catatonit +# there instead of configuring this option here. +# +#init_path = "/usr/libexec/podman/catatonit" + +# Default way to to create an IPC namespace (POSIX SysV IPC) for the container +# Options are: +# "host" Share host IPC Namespace with the container. +# "none" Create shareable IPC Namespace for the container without a private /dev/shm. +# "private" Create private IPC Namespace for the container, other containers are not allowed to share it. +# "shareable" Create shareable IPC Namespace for the container. +# +#ipcns = "shareable" + +# Default way to set an interface name inside container. Defaults to legacy +# pattern of ethX, where X is a integer, when left undefined. +# Options are: +# "device" Uses the network_interface name from the network config as interface name. +# Falls back to the ethX pattern if the network_interface is not set. +#interface_name = "" + +# keyring tells the container engine whether to create +# a kernel keyring for use within the container. +# +#keyring = true + +# label tells the container engine whether to use container separation using +# MAC(SELinux) labeling or not. +# The label flag is ignored on label disabled systems. +# +#label = true + +# label_users indicates whether to enforce confined users in containers on +# SELinux systems. This option causes containers to maintain the current user +# and role field of the calling process. By default SELinux containers run with +# the user system_u, and the role system_r. +#label_users = false + +# Logging driver for the container. Available options: k8s-file and journald. +# +#log_driver = "k8s-file" + +# Maximum size allowed for the container log file. Negative numbers indicate +# that no size limit is imposed. If positive, it must be >= 8192 to match or +# exceed conmon's read buffer. The file is truncated and re-opened so the +# limit is never exceeded. +# +#log_size_max = -1 + +# Specifies default format tag for container log messages. +# This is useful for creating a specific tag for container log messages. +# Containers logs default to truncated container ID as a tag. +# +#log_tag = "" + +# List of mounts. Specified as +# "type=TYPE,source=,destination=,", for example: +# "type=bind,source=/var/lib/foobar,destination=/var/lib/foobar,ro". +# If it is empty or commented out, no mounts will be added +# +#mounts = [] + +# Default way to to create a Network namespace for the container +# Options are: +# `private` Create private Network Namespace for the container. +# `host` Share host Network Namespace with the container. +# `none` Containers do not use the network +# +#netns = "private" + +# Create /etc/hosts for the container. By default, container engine manage +# /etc/hosts, automatically adding the container's own IP address. +# +#no_hosts = false + +# Tune the host's OOM preferences for containers +# (accepts values from -1000 to 1000). +#oom_score_adj = 0 + +# Default way to to create a PID namespace for the container +# Options are: +# `private` Create private PID Namespace for the container. +# `host` Share host PID Namespace with the container. +# +#pidns = "private" + +# Maximum number of processes allowed in a container. +# +#pids_limit = 2048 + +# Copy the content from the underlying image into the newly created volume +# when the container is created instead of when it is started. If false, +# the container engine will not copy the content until the container is started. +# Setting it to true may have negative performance implications. +# +#prepare_volume_on_create = false + +# Give extended privileges to all containers. A privileged container turns off +# the security features that isolate the container from the host. Dropped +# Capabilities, limited devices, read-only mount points, Apparmor/SELinux +# separation, and Seccomp filters are all disabled. Due to the disabled +# security features the privileged field should almost never be set as +# containers can easily break out of confinment. +# +# Containers running in a user namespace (e.g., rootless containers) cannot +# have more privileges than the user that launched them. +# +#privileged = false + +# Run all containers with root file system mounted read-only +# +# read_only = false + +# Path to the seccomp.json profile which is used as the default seccomp profile +# for the runtime. +# +#seccomp_profile = "/usr/share/containers/seccomp.json" + +# Size of /dev/shm. Specified as . +# Unit is optional, values: +# b (bytes), k (kilobytes), m (megabytes), or g (gigabytes). +# If the unit is omitted, the system uses bytes. +# +#shm_size = "65536k" + +# Set timezone in container. Takes IANA timezones as well as "local", +# which sets the timezone in the container to match the host machine. +# +#tz = "" + +# Set umask inside the container +# +#umask = "0022" + +# Default way to to create a User namespace for the container +# Options are: +# `auto` Create unique User Namespace for the container. +# `host` Share host User Namespace with the container. +# +#userns = "host" + +# Default way to to create a UTS namespace for the container +# Options are: +# `private` Create private UTS Namespace for the container. +# `host` Share host UTS Namespace with the container. +# +#utsns = "private" + +# List of volumes. Specified as +# "::", for example: +# "/db:/var/lib/db:ro". +# If it is empty or commented out, no volumes will be added +# +#volumes = [] + +#[engine.platform_to_oci_runtime] +#"wasi/wasm" = ["crun-wasm"] +#"wasi/wasm32" = ["crun-wasm"] +#"wasi/wasm64" = ["crun-wasm"] + +[secrets] +#driver = "file" + +[secrets.opts] +#root = "/example/directory" + +[network] + +# Network backend determines what network driver will be used to set up and tear down container networks. +# Valid values are "cni" and "netavark". +# The default value is empty which means that it will automatically choose CNI or netavark. If there are +# already containers/images or CNI networks preset it will choose CNI. +# +# Before changing this value all containers must be stopped otherwise it is likely that +# iptables rules and network interfaces might leak on the host. A reboot will fix this. +# +#network_backend = "" + +# Path to directory where CNI plugin binaries are located. +# +#cni_plugin_dirs = [ +# "/usr/local/libexec/cni", +# "/usr/libexec/cni", +# "/usr/local/lib/cni", +# "/usr/lib/cni", +# "/opt/cni/bin", +#] + +# List of directories that will be searched for netavark plugins. +# +#netavark_plugin_dirs = [ +# "/usr/local/libexec/netavark", +# "/usr/libexec/netavark", +# "/usr/local/lib/netavark", +# "/usr/lib/netavark", +#] + +# The firewall driver to be used by netavark. +# The default is empty which means netavark will pick one accordingly. Current supported +# drivers are "iptables", "nftables", "none" (no firewall rules will be created) and "firewalld" (firewalld is +# experimental at the moment and not recommend outside of testing). +# +#firewall_driver = "" + +# The network name of the default network to attach pods to. +# +#default_network = "podman" + +# The default subnet for the default network given in default_network. +# If a network with that name does not exist, a new network using that name and +# this subnet will be created. +# Must be a valid IPv4 CIDR prefix. +# +#default_subnet = "10.88.0.0/16" + +# DefaultSubnetPools is a list of subnets and size which are used to +# allocate subnets automatically for podman network create. +# It will iterate through the list and will pick the first free subnet +# with the given size. This is only used for ipv4 subnets, ipv6 subnets +# are always assigned randomly. +# +#default_subnet_pools = [ +# {"base" = "10.89.0.0/16", "size" = 24}, +# {"base" = "10.90.0.0/15", "size" = 24}, +# {"base" = "10.92.0.0/14", "size" = 24}, +# {"base" = "10.96.0.0/11", "size" = 24}, +# {"base" = "10.128.0.0/9", "size" = 24}, +#] + +# Configure which rootless network program to use by default. Valid options are +# `slirp4netns` and `pasta` (default). +# +#default_rootless_network_cmd = "pasta" + +# Path to the directory where network configuration files are located. +# For the CNI backend the default is "/etc/cni/net.d" as root +# and "$HOME/.config/cni/net.d" as rootless. +# For the netavark backend "/etc/containers/networks" is used as root +# and "$graphroot/networks" as rootless. +# +#network_config_dir = "/etc/cni/net.d/" + +# Port to use for dns forwarding daemon with netavark in rootful bridge +# mode and dns enabled. +# Using an alternate port might be useful if other dns services should +# run on the machine. +# +#dns_bind_port = 53 + +# A list of default pasta options that should be used running pasta. +# It accepts the pasta cli options, see pasta(1) for the full list of options. +# +#pasta_options = [] + +[engine] +# Index to the active service +# +#active_service = "production" + +#List of compression algorithms. If set makes sure that requested compression variant +#for each platform is added to the manifest list keeping original instance intact in +#the same manifest list on every `manifest push`. Supported values are (`gzip`, `zstd` and `zstd:chunked`). +# +#add_compression = ["gzip", "zstd", "zstd:chunked"] + +# Enforces using docker.io for completing short names in Podman's compatibility +# REST API. Note that this will ignore unqualified-search-registries and +# short-name aliases defined in containers-registries.conf(5). +#compat_api_enforce_docker_hub = true + +# Specify one or more external providers for the compose command. The first +# found provider is used for execution. Can be an absolute and relative path +# or a (file) name. +#compose_providers=[] + +# Emit logs on each invocation of the compose command indicating that an +# external compose provider is being executed. +#compose_warning_logs = true + +# The compression format to use when pushing an image. +# Valid options are: `gzip`, `zstd` and `zstd:chunked`. +# This field is ignored when pushing images to the docker-daemon and +# docker-archive formats. It is also ignored when the manifest format is set +# to v2s2. +# +#compression_format = "gzip" + +# The compression level to use when pushing an image. +# Valid options depend on the compression format used. +# For gzip, valid options are 1-9, with a default of 5. +# For zstd, valid options are 1-20, with a default of 3. +# +#compression_level = 5 + +# Cgroup management implementation used for the runtime. +# Valid options "systemd" or "cgroupfs" +# +#cgroup_manager = "systemd" + +# Environment variables to pass into conmon +# +#conmon_env_vars = [ +# "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" +#] + +# Paths to look for the conmon container manager binary +# +#conmon_path = [ +# "/usr/libexec/podman/conmon", +# "/usr/local/libexec/podman/conmon", +# "/usr/local/lib/podman/conmon", +# "/usr/bin/conmon", +# "/usr/sbin/conmon", +# "/usr/local/bin/conmon", +# "/usr/local/sbin/conmon" +#] + +# Enforces using docker.io for completing short names in Podman's compatibility +# REST API. Note that this will ignore unqualified-search-registries and +# short-name aliases defined in containers-registries.conf(5). +#compat_api_enforce_docker_hub = true + +# The database backend of Podman. Supported values are "" (default), "boltdb" +# and "sqlite". An empty value means it will check whenever a boltdb already +# exists and use it when it does, otherwise it will use sqlite as default +# (e.g. new installs). This allows for backwards compatibility with older versions. +# Please run `podman-system-reset` prior to changing the database +# backend of an existing deployment, to make sure Podman can operate correctly. +# +#database_backend = "" + +# Specify the keys sequence used to detach a container. +# Format is a single character [a-Z] or a comma separated sequence of +# `ctrl-`, where `` is one of: +# `a-z`, `@`, `^`, `[`, `\`, `]`, `^` or `_` +# Specifying "" disables this feature. +#detach_keys = "ctrl-p,ctrl-q" + +# Determines whether engine will reserve ports on the host when they are +# forwarded to containers. When enabled, when ports are forwarded to containers, +# ports are held open by as long as the container is running, ensuring that +# they cannot be reused by other programs on the host. However, this can cause +# significant memory usage if a container has many ports forwarded to it. +# Disabling this can save memory. +# +#enable_port_reservation = true + +# Environment variables to be used when running the container engine (e.g., Podman, Buildah). +# For example "http_proxy=internal.proxy.company.com". +# Note these environment variables will not be used within the container. +# Set the env section under [containers] table, if you want to set environment variables for the container. +# +#env = [] + +# Define where event logs will be stored, when events_logger is "file". +#events_logfile_path="" + +# Sets the maximum size for events_logfile_path. +# The size can be b (bytes), k (kilobytes), m (megabytes), or g (gigabytes). +# The format for the size is ``, e.g., `1b` or `3g`. +# If no unit is included then the size will be read in bytes. +# When the limit is exceeded, the logfile will be rotated and the old one will be deleted. +# If the maximum size is set to 0, then no limit will be applied, +# and the logfile will not be rotated. +#events_logfile_max_size = "1m" + +# Selects which logging mechanism to use for container engine events. +# Valid values are `journald`, `file` and `none`. +# +#events_logger = "journald" + +# Creates a more verbose container-create event which includes a JSON payload +# with detailed information about the container. +#events_container_create_inspect_data = false + +# Whenever Podman should log healthcheck events. +# With many running healthcheck on short interval Podman will spam the event +# log a lot as it generates a event for each single healthcheck run. Because +# this event is optional and only useful to external consumers that may want +# to know when a healthcheck is run or failed allow users to turn it off by +# setting it to false. The default is true. +# +#healthcheck_events = true + +# A is a list of directories which are used to search for helper binaries. +# +#helper_binaries_dir = [ +# "/usr/local/libexec/podman", +# "/usr/local/lib/podman", +# "/usr/libexec/podman", +# "/usr/lib/podman", +#] + +# Path to OCI hooks directories for automatically executed hooks. +# +#hooks_dir = [ +# "/usr/share/containers/oci/hooks.d", +#] + +# Directories to scan for CDI Spec files. +# +#cdi_spec_dirs = [ +# "/etc/cdi", +#] + +# Manifest Type (oci, v2s2, or v2s1) to use when pulling, pushing, building +# container images. By default image pulled and pushed match the format of the +# source image. Building/committing defaults to OCI. +# +#image_default_format = "" + +# Default transport method for pulling and pushing for images +# +#image_default_transport = "docker://" + +# Maximum number of image layers to be copied (pulled/pushed) simultaneously. +# Not setting this field, or setting it to zero, will fall back to containers/image defaults. +# +#image_parallel_copies = 0 + +# Tells container engines how to handle the built-in image volumes. +# * anonymous: An anonymous named volume will be created and mounted +# into the container. +# * tmpfs: The volume is mounted onto the container as a tmpfs, +# which allows users to create content that disappears when +# the container is stopped. +# * ignore: All volumes are just ignored and no action is taken. +# +#image_volume_mode = "" + +# Default command to run the infra container +# +infra_command = "/pause" + +# Infra (pause) container image name for pod infra containers. When running a +# pod, we start a `pause` process in a container to hold open the namespaces +# associated with the pod. This container does nothing other than sleep, +# reserving the pod's resources for the lifetime of the pod. By default container +# engines run a built-in container using the pause executable. If you want override +# specify an image to pull. +# +infra_image = "k8s.gcr.io/pause:3.8" + +# Default Kubernetes kind/specification of the kubernetes yaml generated with the `podman kube generate` command. +# The possible options are `pod` and `deployment`. +#kube_generate_type = "pod" + +# Specify the locking mechanism to use; valid values are "shm" and "file". +# Change the default only if you are sure of what you are doing, in general +# "file" is useful only on platforms where cgo is not available for using the +# faster "shm" lock type. You may need to run "podman system renumber" after +# you change the lock type. +# +#lock_type = "shm" + +# MultiImageArchive - if true, the container engine allows for storing archives +# (e.g., of the docker-archive transport) with multiple images. By default, +# Podman creates single-image archives. +# +#multi_image_archive = false + +# Default engine namespace +# If engine is joined to a namespace, it will see only containers and pods +# that were created in the same namespace, and will create new containers and +# pods in that namespace. +# The default namespace is "", which corresponds to no namespace. When no +# namespace is set, all containers and pods are visible. +# +#namespace = "" + +# Path to the slirp4netns binary +# +#network_cmd_path = "" + +# Default options to pass to the slirp4netns binary. +# Valid options values are: +# +# - allow_host_loopback=true|false: Allow the slirp4netns to reach the host loopback IP (`10.0.2.2`). +# Default is false. +# - mtu=MTU: Specify the MTU to use for this network. (Default is `65520`). +# - cidr=CIDR: Specify ip range to use for this network. (Default is `10.0.2.0/24`). +# - enable_ipv6=true|false: Enable IPv6. Default is true. (Required for `outbound_addr6`). +# - outbound_addr=INTERFACE: Specify the outbound interface slirp should bind to (ipv4 traffic only). +# - outbound_addr=IPv4: Specify the outbound ipv4 address slirp should bind to. +# - outbound_addr6=INTERFACE: Specify the outbound interface slirp should bind to (ipv6 traffic only). +# - outbound_addr6=IPv6: Specify the outbound ipv6 address slirp should bind to. +# - port_handler=rootlesskit: Use rootlesskit for port forwarding. Default. +# Note: Rootlesskit changes the source IP address of incoming packets to a IP address in the container +# network namespace, usually `10.0.2.100`. If your application requires the real source IP address, +# e.g. web server logs, use the slirp4netns port handler. The rootlesskit port handler is also used for +# rootless containers when connected to user-defined networks. +# - port_handler=slirp4netns: Use the slirp4netns port forwarding, it is slower than rootlesskit but +# preserves the correct source IP address. This port handler cannot be used for user-defined networks. +# +#network_cmd_options = [] + +# Whether to use chroot instead of pivot_root in the runtime +# +#no_pivot_root = false + +# Number of locks available for containers, pods, and volumes. Each container, +# pod, and volume consumes 1 lock for as long as it exists. +# If this is changed, a lock renumber must be performed (e.g. with the +# 'podman system renumber' command). +# +#num_locks = 2048 + +# Set the exit policy of the pod when the last container exits. +#pod_exit_policy = "continue" + +# Whether to pull new image before running a container +# +#pull_policy = "missing" + +# Indicates whether the application should be running in remote mode. This flag modifies the +# --remote option on container engines. Setting the flag to true will default +# `podman --remote=true` for access to the remote Podman service. +# +#remote = false + +# Number of times to retry pulling/pushing images in case of failure +# +#retry = 3 + +# Delay between retries in case pulling/pushing image fails. +# If set, container engines will retry at the set interval, +# otherwise they delay 2 seconds and then exponentially back off. +# +#retry_delay = "2s" + +# Default OCI runtime +# +runtime = "runc" + +# List of the OCI runtimes that support --format=json. When json is supported +# engine will use it for reporting nicer errors. +# +#runtime_supports_json = ["crun", "runc", "kata", "runsc", "youki", "krun"] + +# List of the OCI runtimes that supports running containers with KVM Separation. +# +#runtime_supports_kvm = ["kata", "krun"] + +# List of the OCI runtimes that supports running containers without cgroups. +# +#runtime_supports_nocgroups = ["crun", "krun"] + +# Default location for storing temporary container image content. Can be overridden with the TMPDIR environment +# variable. If you specify "storage", then the location of the +# container/storage tmp directory will be used. +# image_copy_tmp_dir="/var/tmp" + +# Number of seconds to wait without a connection +# before the `podman system service` times out and exits +# +#service_timeout = 5 + +# Directory for persistent engine files (database, etc) +# By default, this will be configured relative to where the containers/storage +# stores containers +# Uncomment to change location from this default +# +#static_dir = "/var/lib/containers/storage/libpod" + +# Number of seconds to wait for container to exit before sending kill signal. +# +#stop_timeout = 10 + +# Number of seconds to wait before exit command in API process is given to. +# This mimics Docker's exec cleanup behaviour, where the default is 5 minutes (value is in seconds). +# +#exit_command_delay = 300 + +# map of service destinations +# +# [engine.service_destinations] +# [engine.service_destinations.production] +# URI to access the Podman service +# Examples: +# rootless "unix:///run/user/$UID/podman/podman.sock" (Default) +# rootful "unix:///run/podman/podman.sock (Default) +# remote rootless ssh://engineering.lab.company.com/run/user/1000/podman/podman.sock +# remote rootful ssh://root@10.10.1.136:22/run/podman/podman.sock +# +# uri = "ssh://user@production.example.com/run/user/1001/podman/podman.sock" +# Path to file containing ssh identity key +# identity = "~/.ssh/id_rsa" + +# Directory for temporary files. Must be tmpfs (wiped after reboot) +# +#tmp_dir = "/run/libpod" + +# Directory for libpod named volumes. +# By default, this will be configured relative to where containers/storage +# stores containers. +# Uncomment to change location from this default. +# +#volume_path = "/var/lib/containers/storage/volumes" + +# Default timeout (in seconds) for volume plugin operations. +# Plugins are external programs accessed via a REST API; this sets a timeout +# for requests to that API. +# A value of 0 is treated as no timeout. +#volume_plugin_timeout = 5 + +# Paths to look for a valid OCI runtime (crun, runc, kata, runsc, krun, etc) +[engine.runtimes] +#crun = [ +# "/usr/bin/crun", +# "/usr/sbin/crun", +# "/usr/local/bin/crun", +# "/usr/local/sbin/crun", +# "/sbin/crun", +# "/bin/crun", +# "/run/current-system/sw/bin/crun", +#] + +#crun-vm = [ +# "/usr/bin/crun-vm", +# "/usr/local/bin/crun-vm", +# "/usr/local/sbin/crun-vm", +# "/sbin/crun-vm", +# "/bin/crun-vm", +# "/run/current-system/sw/bin/crun-vm", +#] + +#kata = [ +# "/usr/bin/kata-runtime", +# "/usr/sbin/kata-runtime", +# "/usr/local/bin/kata-runtime", +# "/usr/local/sbin/kata-runtime", +# "/sbin/kata-runtime", +# "/bin/kata-runtime", +# "/usr/bin/kata-qemu", +# "/usr/bin/kata-fc", +#] + +#runc = [ +# "/usr/bin/runc", +# "/usr/sbin/runc", +# "/usr/local/bin/runc", +# "/usr/local/sbin/runc", +# "/sbin/runc", +# "/bin/runc", +# "/usr/lib/cri-o-runc/sbin/runc", +#] + +#runsc = [ +# "/usr/bin/runsc", +# "/usr/sbin/runsc", +# "/usr/local/bin/runsc", +# "/usr/local/sbin/runsc", +# "/bin/runsc", +# "/sbin/runsc", +# "/run/current-system/sw/bin/runsc", +#] + +#youki = [ +# "/usr/local/bin/youki", +# "/usr/bin/youki", +# "/bin/youki", +# "/run/current-system/sw/bin/youki", +#] + +#krun = [ +# "/usr/bin/krun", +# "/usr/local/bin/krun", +#] + +[engine.volume_plugins] +#testplugin = "/run/podman/plugins/test.sock" + +[machine] +# Number of CPU's a machine is created with. +# +#cpus=1 + +# The size of the disk in GB created when init-ing a podman-machine VM. +# +#disk_size=10 + +# Default Image used when creating a new VM using `podman machine init`. +# Can be specified as registry with a bootable OCI artifact, download URL, or a local path. +# Registry target must be in the form of `docker://registry/repo/image:version`. +# Container engines translate URIs $OS and $ARCH to the native OS and ARCH. +# URI "https://example.com/$OS/$ARCH/foobar.ami" would become +# "https://example.com/linux/amd64/foobar.ami" on a Linux AMD machine. +# If unspecified, the default Podman machine image will be used. +# +#image = "" + +# Memory in MB a machine is created with. +# +#memory=2048 + +# The username to use and create on the podman machine OS for rootless +# container access. +# +#user = "core" + +# Host directories to be mounted as volumes into the VM by default. +# Environment variables like $HOME as well as complete paths are supported for +# the source and destination. An optional third field `:ro` can be used to +# tell the container engines to mount the volume readonly. +# +#volumes = [ +# "$HOME:$HOME", +#] + +# Virtualization provider used to run Podman machine. +# If it is empty or commented out, the default provider will be used. +# +#provider = "" + +# Rosetta supports running x86_64 Linux binaries on a Podman machine on Apple silicon. +# The default value is `true`. Supported on AppleHV(arm64) machines only. +# +#rosetta=true + +# The [machine] table MUST be the last entry in this file. +# (Unless another table is added) +# TOML does not provide a way to end a table other than a further table being +# defined, so every key hereafter will be part of [machine] and not the +# main config. + +[farms] +# +# the default farm to use when farming out builds +# default = "" +# +# map of existing farms +#[farms.list] + +[podmansh] +# Shell to spawn in container. Default: /bin/sh. +#shell = "/bin/sh" +# +# Name of the container the podmansh user should join. +#container = "podmansh" +# +# Default timeout in seconds for podmansh logins. +# Favored over the deprecated "podmansh_timeout" field. +#timeout = 30 diff --git a/scripts/install_criu.sh b/scripts/install_criu.sh index 27a1666..f58e6bd 100755 --- a/scripts/install_criu.sh +++ b/scripts/install_criu.sh @@ -1,18 +1,55 @@ -sudo apt install -y build-essential +#!/bin/bash +# Install CRIU (Checkpoint/Restore In Userspace) on Ubuntu 22.04+ -sudo apt install -y libprotobuf-dev libprotobuf-c-dev protobuf-c-compiler protobuf-compiler python3-protobuf +# Print script commands and exit on errors. +set -xe -sudo apt install -y libbsd-dev pkg-config libbsd-dev iproute2 libnftables-dev libcap-dev libnl-3-dev libnl-3-200 libnet1-dev libnet1 libnl-3-dev libnet-dev libaio-dev libgnutls28-dev python3-future libdrm-dev +# Install dependencies +sudo apt-get install -y \ + build-essential \ + libprotobuf-dev \ + libprotobuf-c-dev \ + protobuf-c-compiler \ + protobuf-compiler \ + python3-protobuf \ + libbsd-dev \ + pkg-config \ + libbsd-dev \ + iproute2 \ + libnftables-dev \ + libcap-dev \ + libnl-3-dev \ + libnl-3-200 \ + libnet1-dev \ + libnet1 \ + libnl-3-dev \ + libnet-dev \ + libaio-dev \ + libgnutls28-dev \ + python3-future \ + libdrm-dev \ + asciidoc \ + xmlto -sudo apt install -y asciidoc xmlto - -# CRIU -printf "\nInstalling CRIU...\n" -git config --global advice.detachedHead false -git clone --depth 1 --branch master https://github.com/checkpoint-restore/criu.git +git clone --depth 1 --branch v3.19 https://github.com/checkpoint-restore/criu.git cd criu -make criu -j$(nproc) -cp ./criu/criu /sbin -criu --version || exit 1 + +# NOTE: On Ubuntu 24+, you might experience an error similar to: +# error: ‘net/unix/’ directive output truncated writing 9 bytes into a region of size 0 [-Werror=format-truncation=] +# The following issue suggests a fix: +# https://github.com/checkpoint-restore/criu/issues/2398#issuecomment-2079198028 +make -j$(nproc) +# Install criu globally +sudo make install-criu +# Install crit in a virtual environment, +# make sure to activate it before running the script. +# make BINDIR=... crit never worked for me, +# see https://github.com/checkpoint-restore/criu/blob/criu-dev/INSTALL.md +pip3 install ./lib +pip3 install ./crit + +criu --version +crit --version + cd .. # rm -rf criu diff --git a/scripts/install_p4c.sh b/scripts/install_p4c.sh new file mode 100755 index 0000000..abddb2a --- /dev/null +++ b/scripts/install_p4c.sh @@ -0,0 +1,44 @@ +#!/bin/bash +# Install the p4 compiler on Ubuntu 22.04+ + +# Print script commands and exit on errors. +set -xe + +sudo apt-get install -y \ + cmake \ + g++ \ + git \ + automake \ + libtool \ + libgc-dev \ + bison \ + flex \ + libfl-dev \ + libboost-dev \ + libboost-iostreams-dev \ + libboost-graph-dev \ + llvm \ + pkg-config \ + python3 \ + python3-pip \ + tcpdump + +git clone --depth 1 --branch v1.2.4.14 https://github.com/p4lang/p4c.git +cd p4c +git submodule update --init --recursive + +pip3 install --user -r requirements.txt +mkdir -p build +cd build +cmake .. -DENABLE_TEST_TOOLS=ON +# Note: This might take minutes to hours depending on your machine. +# If you are on a distro which provides the p4lang-p4c package, +# it's recommended to install it instead of building from source. +make -j$(nproc) +sudo make install/strip +sudo ldconfig + +p4c --version + +cd ../.. +# rm -rf p4c diff --git a/scripts/install_pi.sh b/scripts/install_pi.sh new file mode 100755 index 0000000..77ad833 --- /dev/null +++ b/scripts/install_pi.sh @@ -0,0 +1,41 @@ +#!/bin/bash +# Install p4lang/PI on Ubuntu 22.04+ + +# Print script commands and exit on errors. +set -xe + +# If you want to use a specific venv, +# change this variable to its absolute path +PYTHON_VENV="$VIRTUAL_ENV" +source "${PYTHON_VENV}/bin/activate" + +sudo apt-get install -y \ + libreadline-dev \ + valgrind \ + libtool-bin \ + libboost-dev \ + libboost-system-dev \ + libboost-thread-dev \ + libprotobuf-dev \ + protobuf-compiler \ + protobuf-compiler-grpc \ + libgrpc-dev \ + libgrpc++-dev + +pip3 install protobuf==3.20.0 + +git clone --depth 1 --branch main https://github.com/p4lang/PI.git +cd PI +git checkout 05cb92564af77ae4826565cbde84e3fd4960c6bd +git submodule update --init --recursive + +./autogen.sh +configure_python_prefix="--with-python_prefix=${PYTHON_VENV}" +./configure --with-proto --without-internal-rpc --without-cli --without-bmv2 ${configure_python_prefix} +make -j$(nproc) +sudo make install +sudo ldconfig +sudo chown -R $USER $PYTHON_VENV + +cd .. +# rm -rf PI diff --git a/scripts/install_podman.sh b/scripts/install_podman.sh new file mode 100755 index 0000000..32b76b7 --- /dev/null +++ b/scripts/install_podman.sh @@ -0,0 +1,49 @@ +#!/bin/bash +# Install podman on Ubuntu 22.04+ + +# Print script commands and exit on errors. +set -xe + +sudo apt-get install -y \ + btrfs-progs \ + crun \ + git \ + go-md2man \ + iptables \ + libassuan-dev \ + libbtrfs-dev \ + libc6-dev \ + libdevmapper-dev \ + libglib2.0-dev \ + libgpgme-dev \ + libgpg-error-dev \ + libprotobuf-dev \ + libprotobuf-c-dev \ + libseccomp-dev \ + libselinux1-dev \ + libsystemd-dev \ + make \ + pkg-config \ + uidmap \ + conmon \ + runc + +# Create default configuration files +sudo mkdir -p /etc/containers +sudo curl -L -o /etc/containers/registries.conf https://raw.githubusercontent.com/containers/image/main/registries.conf +sudo curl -L -o /etc/containers/policy.json https://raw.githubusercontent.com/containers/image/main/default-policy.json + +sudo mkdir -p /usr/share/containers +# runtime = "runc", infra_image="k8s.gcr.io/pause:3.8", infra_command = "/pause", otherwise default +sudo cp "$(dirname "$(realpath "$0")")/containers.conf" /usr/share/containers/containers.conf + +# Requires go v1.21.0 (strict) +git clone --depth 1 --branch v5.2.1 https://github.com/containers/podman/ +cd podman +make -j$(nproc) BUILDTAGS="selinux seccomp" PREFIX=/usr +sudo GOPATH=/usr/bin/go PATH=$GOPATH/bin:$PATH make install PREFIX=/usr + +podman --version + +cd .. +# rm -rf podman