Add "shibaken" generator to populate user-data with public keys from IMDS #1331

jpculp · 2021-02-18T04:20:21Z

Issue number:
N/A

Description of changes:

Add "shibaken" generator to populate user-data with public keys from IMDS

This populates the host container's user-data setting with public keys
available from IMDS in the event that user-data has not been populated
by the user.

Adds a new migration helper AddMetadataMigration to remove setting
metadata (like setting-generators).

Adds a new migration for shibaken setting-generator.

Testing done:

Build aws-ecs-1 image and launched instance.

Verified that host-containers.admin.user-data contained a base64-encoded block.

Enabled and launched admin container, which started fine.

Verified that /.bottlerocket/host-containers/admin/user-data contained JSON with the following structure:

{
   "ssh":{
      "authorized_keys":[
         "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClKsfkNkuSevGj3eYhCe53pcjqP3maAhDFcvBS7O6V
          hz2ItxCih+PnDSUaw+WNQn/mZphTk/a/gU8jEzoOWbkM4yxyb/wB96xbiFveSFJuOp/d6RJhJOI0iBXr
          lsLnBItntckiJ7FbtxJMXLvvwJryDUilBMTjYtwB+QhYXUMOzce5Pjz5/i8SeJtjnV3iAoG/cQk+0FzZ
          qaeJAAHco+CY/5WrUBkrHmFJr6HcXkvJdWPkYQS3xqC0+FmUZofz221CBt5IMucxXPkX4rWi+z7wB3Rb
          BQoQzd8v7yeb7OzlPnWOyN0qFU0XA246RA8QFYiCNYwI3f05p6KLxEXAMPLE my-key-pair"
      ]
   }
}

Ran sudo sheltie to verify root shell was still available.

@etungsten verified the migration between (1.0.5, 1.0.6)

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

packages/os/os.spec

sources/api/shibaken/src/main.rs

GLOSSARY.md

packages/os/os.spec

sources/api/migration/migration-helpers/src/common_migrations.rs

sources/api/shibaken/src/main.rs

zmrow

Also - could we update the testing of this PR to reflect that the migration up/down was tested and the settings appeared as we expected them to and the admin host container works in both versions?

zmrow

Looks good!

🪂

sources/api/shibaken/src/main.rs

sources/api/migration/migration-helpers/src/common_migrations.rs

GLOSSARY.md

sources/api/migration/migrations/v1.0.6/add-shibaken/src/main.rs

tjkirch · 2021-02-22T22:53:58Z

sources/api/shibaken/src/main.rs

+}
+
+/// Helper to fetch an IMDSv2 session token that is valid for 60 seconds.
+fn fetch_imds_session_token(client: &Client) -> Result<String> {


tangent: It is a bit unfortunate to have three implementations of this now, each slightly different. We should consider making an imds crate as a follow-up.

I definitely agree. I can take that on in another PR if folks are interested.

sources/api/shibaken/src/main.rs

jpculp · 2021-02-23T22:13:12Z

For some context on the recent pushes that squashed together...

Rebased and addressed comments/concerns with the shibaken crate itself:
https://github.com/bottlerocket-os/bottlerocket/compare/e617e9ec..0d9fb467

Replaced instances of host container with admin container and enhanced the AddMetadataMigration to accept multiple settings (+ unit test):
https://github.com/bottlerocket-os/bottlerocket/compare/0d9fb467..30d61d14

Attempt to manually fix merge conflict:
https://github.com/bottlerocket-os/bottlerocket/compare/30d61d14..925e9f1c

Went back and just rebased instead:
https://github.com/bottlerocket-os/bottlerocket/compare/925e9f1c..11e5885fc2463e2495bcdff0ca7a61d6dd730df0

sources/api/shibaken/src/main.rs

sources/api/migration/migration-helpers/src/common_migrations.rs

packages/os/os.spec

sources/models/shared-defaults/aws-host-containers.toml

jpculp · 2021-02-24T04:37:04Z

Changed SettingsMetadata to SettingMetadata
Simplified parse_args function logic
Replaced 'a lifetimes with static in AddMetadataMigration code
Additional clean-up

sources/api/shibaken/src/main.rs

…IMDS This populates the admin container's user-data setting with public keys available from IMDS in the event that user-data has not been populated by the user.

Adds new migration helper `AddMetadataMigration` to remove setting metadata (like setting-generators).

jpculp · 2021-02-24T19:57:20Z

Changed SettingMetadata from a vec to a slice in AddMetadataMigration
Replaced instances of url with uri in shibaken
Removed comma from the last element of the (1.0.5, 1.0.6) migration list for consistency

(CI failure)

zmrow

🦺

Adds a new migration for shibaken setting-generator

jpculp · 2021-02-24T21:45:44Z

Changed SettingMetadata from a vec to a slice in add-shibaken
Added additional commas to migration lists in Release.toml. The hope is we can be consistent in adding them to make it easier to append additional migrations between PRs.

jpculp requested review from tjkirch, bcressey, jahkeup and zmrow February 18, 2021 04:20

bcressey reviewed Feb 18, 2021

View reviewed changes

jpculp force-pushed the shibaken branch 2 times, most recently from 21b15d4 to 6894374 Compare February 19, 2021 04:36

jpculp requested a review from bcressey February 19, 2021 04:42

zmrow reviewed Feb 19, 2021

View reviewed changes

jahkeup mentioned this pull request Feb 19, 2021

Change admin container logic to use public keys from user-data bottlerocket-os/bottlerocket-admin-container#19

Merged

tjkirch mentioned this pull request Feb 19, 2021

AWS-independent ssh mechanism or documentation #1326

Closed

jpculp force-pushed the shibaken branch from 6894374 to e617e9e Compare February 19, 2021 23:09

jpculp changed the title ~~Add "shibaken" generator to build user-data with public keys from IMDS~~ Add "shibaken" generator to populate user-data with public keys from IMDS Feb 19, 2021

zmrow approved these changes Feb 22, 2021

View reviewed changes

bcressey reviewed Feb 22, 2021

View reviewed changes

sources/api/shibaken/src/main.rs Outdated Show resolved Hide resolved

sources/api/migration/migration-helpers/src/common_migrations.rs Outdated Show resolved Hide resolved

tjkirch reviewed Feb 22, 2021

View reviewed changes

samuelkarp reviewed Feb 22, 2021

View reviewed changes

sources/api/shibaken/src/main.rs Outdated Show resolved Hide resolved

jpculp force-pushed the shibaken branch 4 times, most recently from 925e9f1 to 11e5885 Compare February 23, 2021 21:00

tjkirch reviewed Feb 23, 2021

View reviewed changes

bcressey reviewed Feb 24, 2021

View reviewed changes

sources/models/shared-defaults/aws-host-containers.toml Outdated Show resolved Hide resolved

jpculp force-pushed the shibaken branch from 11e5885 to 332cf65 Compare February 24, 2021 04:23

jpculp requested review from tjkirch, samuelkarp and bcressey February 24, 2021 04:37

jpculp requested a review from zmrow February 24, 2021 04:37

bcressey approved these changes Feb 24, 2021

View reviewed changes

sources/api/shibaken/src/main.rs Outdated Show resolved Hide resolved

jpculp added 2 commits February 24, 2021 19:50

Add "shibaken" generator to populate user-data with public keys from …

5f47c8c

…IMDS This populates the admin container's user-data setting with public keys available from IMDS in the event that user-data has not been populated by the user.

migration-helpers: AddMetadataMigration for setting metadata

a7d71ab

Adds new migration helper `AddMetadataMigration` to remove setting metadata (like setting-generators).

jpculp force-pushed the shibaken branch from 332cf65 to 511f7c4 Compare February 24, 2021 19:53

tjkirch previously approved these changes Feb 24, 2021

View reviewed changes

zmrow approved these changes Feb 24, 2021

View reviewed changes

migrations: add add-shibaken migration

8095c02

Adds a new migration for shibaken setting-generator

jpculp force-pushed the shibaken branch from 511f7c4 to 8095c02 Compare February 24, 2021 21:25

tjkirch approved these changes Feb 24, 2021

View reviewed changes

bcressey approved these changes Feb 24, 2021

View reviewed changes

jpculp merged commit ba9afa4 into bottlerocket-os:develop Feb 24, 2021

jpculp mentioned this pull request Feb 25, 2021

README: Add note to user-data section of custom host containers #1355

Merged

jpculp deleted the shibaken branch February 26, 2021 02:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add "shibaken" generator to populate user-data with public keys from IMDS #1331

Add "shibaken" generator to populate user-data with public keys from IMDS #1331

jpculp commented Feb 18, 2021 •

edited

Loading

zmrow left a comment

zmrow left a comment

tjkirch Feb 22, 2021

jpculp Feb 23, 2021

jpculp commented Feb 23, 2021 •

edited

Loading

jpculp commented Feb 24, 2021

jpculp commented Feb 24, 2021

zmrow left a comment

jpculp commented Feb 24, 2021

Add "shibaken" generator to populate user-data with public keys from IMDS #1331

Add "shibaken" generator to populate user-data with public keys from IMDS #1331

Conversation

jpculp commented Feb 18, 2021 • edited Loading

zmrow left a comment

Choose a reason for hiding this comment

zmrow left a comment

Choose a reason for hiding this comment

tjkirch Feb 22, 2021

Choose a reason for hiding this comment

jpculp Feb 23, 2021

Choose a reason for hiding this comment

jpculp commented Feb 23, 2021 • edited Loading

jpculp commented Feb 24, 2021

jpculp commented Feb 24, 2021

zmrow left a comment

Choose a reason for hiding this comment

jpculp commented Feb 24, 2021

jpculp commented Feb 18, 2021 •

edited

Loading

jpculp commented Feb 23, 2021 •

edited

Loading