Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Settings Models to the Settings SDK #41

Merged
merged 466 commits into from
Jun 17, 2024

Conversation

cbgbt
Copy link
Contributor

@cbgbt cbgbt commented Jun 14, 2024

Description of changes:
This moves the following packages from https://github.com/bottlerocket-os/bottlerocket to this repository:

  • All current settings extension packages
  • Any settings models not yet ported to settings extensions
  • modeled-types, useful Rust types that can be composed into new settings models
  • model-derive macro for auto-implementing useful string traits in settings models
  • scalar and scalar-derive macro to assist modeling scalar values in settings models

This change was constructed by copying content and git history from the Bottlerocket repository directly, and then making discrete changes to those contents as a series of commits -- the commit history is very useful in reviewing this change.

From a user perspective, the bottlerocket-settings-models package is meant to expose the public API of our existing models, as well as tools used to build new models.

Pending

  • Build Bottlerocket repository with dependencies on this code
  • Fix comments/documentation in the contents of these files

This is the script that was used to import git history from bottlerocket-os/bottlerocket:

#!/usr/bin/env bash

set -e

cd /tmp
DIR="${1:=/tmp/sdk-models}"
BR=$DIR/bottlerocket
SDK=$DIR/bottlerocket-settings-sdk

BOTTLEROCKET_BRANCH=deleteme-models-export
SDK_IMPORT_BRANCH=deleteme-models-import
SDK_FINAL_BRANCH=vend-plugins-with-history

rm -rf $DIR

git clone git@github.com:bottlerocket-os/bottlerocket.git $BR

git clone git@github.com:bottlerocket-os/bottlerocket-settings-sdk.git $SDK
cd $SDK
git remote remove origin
git remote add origin git@github.com:cbgbt/bottlerocket-settings-sdk.git
git remote add BECAREFUL_UPSTREAM git@github.com:bottlerocket-os/twoliter.git

cd $BR
git checkout -b $BOTTLEROCKET_BRANCH
git fast-export $BOTTLEROCKET_BRANCH -- \
  sources/models \
  sources/settings-extensions > $DIR/export.fi

cd $SDK
git switch --orphan $BOTTLEROCKET_BRANCH
git fast-import < $DIR/export.fi
git checkout develop
git checkout -b $SDK_IMPORT_BRANCH
git merge --signoff --allow-unrelated-histories -m'chore: merge models history' $BOTTLEROCKET_BRANCH

mkdir -p bottlerocket-settings-models

mv sources/models/* bottlerocket-settings-models/
mv sources/settings-extensions/ bottlerocket-settings-models/

git add --all
git rm sources/models/.gitignore
git commit -m'chore: move models into desired directories'
rm -rf sources/

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

zmrow and others added 30 commits September 1, 2022 15:59
This commit adds the ability to configure DNS name servers and search
suffixes via a new settings prefix `settings.dns`.  These DNS settings
populate a configuration file which gets used by `netdog` to write the
`resolv.conf`.  `netdog write-resolv-conf` is triggered via restart
command.  If DNS settings exist, they are used to write the
`resolv.conf`, otherwise the DHCP lease for the primary interface is
used.
modeled_types: Use FromStr instead of TryFrom
This reverts commit cf7a49a705883bf5377f3a6de746aa8642a0a2ca.
Add a new boolean settings.boot.reboot-to-reconcile to govern whether
Bottlerocket should automatically reboot if kernel or systemd command
line parameters are reconfigured during boot. This could happen either
via user-data or via a bootstrap container. In either case, command line
changes for the kernel or systemd will not take effect until the next
reboot.

This change only introduces the new setting to the model and threads it
through in all places touching BootSettings. The flag is not yet armed,
i.e. no reboot action is taken.

Signed-off-by: Markus Boehme <markubo@amazon.com>
This adds a new `kubernetes.log-level` setting to allow
configuration of the kubelet logging verbosity. With this, an end user
can turn up verbosity if they are trying to troubleshoot something, or
turn down verbosity if they don't need as many log messages.

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
In b696d6f, the new `kubernetes.log-level` setting was implemented but
in the actual model the name of the new setting was `kubelet_log_level`.
This renames it to `log_level` since that's what is used in the
templated configuration files and in the documentation.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Migrations: migrate to admin container v0.9.2
Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
The symlink points to the newer aws-k8s-1.24 variant

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
The symlink points to the newer vmware-k8s-1.24 variant

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
The symlink points to the newer metal-k8s-1.24 variant

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
This adds `settings.container-runtime.max-container-log-line-size` and
`settings.container-runtime.max-concurrent-downloads` settings. The
affect how containerd is configured.

`max-container-log-line-size` controls how long a log line can be from a
container before containerd breaks it into multiple separate lines.

`max-concurrent-downloads` controls how many concurrent downloads will
be done in parallel to download an image.

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
- Remove aws-k8s-1.20 variant
- Remove vmware-k8s-1.20 variant

Signed-off-by: John McBride <jpmmcb@amazon.com>
I noticed while doing some other work that a couple imports ended up
being inserted between a comment and the import it was commenting on.
This moves that comment back to where it was supposed to be and runs
format on the file to organize those imports.

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
Signed-off-by: Ben Cressey <bcressey@amazon.com>
Adds configuration settings for controlling AWS credentials and
configuration. This enables setting the `~/.aws/config` and
`~/.aws/credentials` file contents so anything using an aws client can
be configured to use something other than the default instance role.

This adds the AwsSettings to the non-AWS k8s variants in preparation of
their use for configuring credential providers.

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
prepare foundations for Secure Boot and image resigning
Add kubelet config option `credential-providers` to allow configuring
image credential provider settings.

Mention of the new settings have been added to the README, but more
detailed documentation on how to use the feature will be necessary.
Those docs will be added in a future commit.

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
This enables the ability to provide a TLS public and private key to be
used by the kubelet process for HTTPS communication.

This corresponds to the `--tls-cert-file` and `--tls-key-file`
arguments (or the `tlsCertFile` and `tlsPrivateKeyFile` config
settings).

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
This adds the capability to use Kubernetes image credential providers to
retrieve credentials to use when pulling images for container creation.

Initially we will only support the ecr-credential-provider, but things
are set up so we may add more providers in future updates.

Signed-off-by: Sean McGinnis <stmcg@amazon.com>
With this, 4 additional configurations for the ECS agent are supported
though the API.

There are two configuration files used to set up the ECS agent:

- /etc/ecs/ecs.config.json
- /etc/ecs/ecs.config

We favor the former to add new configurations, and we only use the latter
on special cases, i.e. when the configurations to be added aren't
modeled as part of the struct that represents the agent's configuration,
or when special deserialization is used to parse the configurations.

The configurations added in this change are as follows:

ECS_CONTAINER_STOP_TIMEOUT: supported through the container-stop-timeout
API; this configuration is rendered in the /etc/ecs/ecs.config file since
this configuration is of type Duration (1m, 1s, 1h). This type must be
parsed by calling the time.ParseDuration function which isn't called
under the hood by the serialization libraries used in the ECS agent.

ECS_ENGINE_TASK_CLEANUP_WAIT_DURATION: supported through the
task-cleanup-wait API; this configuration is of the same type as the
previous configuration and was rendered following the same reasoning.

ECS_RESERVED_MEMORY: supported through the reserved-memory API; this
configuration is rendered in /etc/ecs/config.ecs.json since the
configuration's type can be deserialized without additional helper
functions.

ECS_TASK_METADATA_RPS_LIMIT: this configuration represents a
comma-separated string with two values used to set the throttling rates
in the metadata service exposed by the ECS agent. These values don't
have to be set together, since the ECS agent will use default values if
either is missing. Thus, this configuration is supported through the
metadata-service-rps and metadata-service-burst APIs. Both
configurations are rendered in the /etc/ecs/confing.ecs.json file, since
the configurations' type can be deserialized without additional helper
functions.

Signed-off-by: Arnaldo Garcia Rincon <agarrcia@amazon.com>
Add additional ECS API configurations
@cbgbt
Copy link
Contributor Author

cbgbt commented Jun 14, 2024

  • Force pushed to add some docstrings missing on new modules
  • Added commits for fixes pointed out by @sam-berning

@cbgbt cbgbt marked this pull request as ready for review June 14, 2024 22:07
Cargo.toml Outdated Show resolved Hide resolved
@cbgbt
Copy link
Contributor Author

cbgbt commented Jun 17, 2024

I'm re-structuring this PR To bring the git history from the bottlerocket-os/bottlerocket with the contents that are copied.

@cbgbt cbgbt marked this pull request as draft June 17, 2024 15:37
@cbgbt
Copy link
Contributor Author

cbgbt commented Jun 17, 2024

  • Adds bottlerocket-os/bottlerocket history for sources/models and sources/settings-extensions
  • Renames string_impls_for to string_impls_for -- @bcressey

@cbgbt cbgbt marked this pull request as ready for review June 17, 2024 16:45
@cbgbt cbgbt merged commit c5dfef4 into bottlerocket-os:develop Jun 17, 2024
1 check passed
@cbgbt cbgbt deleted the vend-plugins branch June 17, 2024 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.