Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrity-protect stateful partition on CS image #251

Merged
merged 1 commit into from
Oct 12, 2022

Conversation

alexmwu
Copy link
Contributor

@alexmwu alexmwu commented Oct 12, 2022

This commit enables a dm-crypt layer of encryption with integrity (using authenticated encryption with additional data) on the partition backing /dev/stateful.
This mechanism slows down boot by ~1min.
The key is stored in memory, and, as a result, the writable, "stateful" partition is wiped on subsequent boots.

Testing

Ran ./run_cloudbuild.sh debug in test project.
Created vm from output image.

Examined serial log of the VM:


[�[0;32m  OK  �[0m] Finished �[0;1;39mWait for Network to be Configured�[0m.
[    8.024822] protect-stateful-partition.sh[157]: WARNING: Device /dev/sda1 already contains a 'ext4' superblock signature.
[    8.029283] protect-stateful-partition.sh[157]: Existing 'ext4' superblock signature (offset: 1080 bytes) on device /dev/sda1 will be wiped.
[    8.034254] protect-stateful-partition.sh[157]: WARNING: Locking directory /run/cryptsetup is missing!
[    8.038296] protect-stateful-partition.sh[157]: Key slot 0 created.
[    8.042261] protect-stateful-partition.sh[157]: Wiping device to initialize integrity checksum.
[    8.046252] protect-stateful-partition.sh[157]: You can interrupt this by pressing CTRL+c (rest of not wiped device will contain invalid checksum).
[   78.583430] protect-stateful-partition.sh[157]: �[2K
Progress:   0.6%, ETA 01:22,   48 MiB written, speed  95.4 MiB/s�[2K
Progress:   1.5%, ETA 01:06,  118 MiB written, speed 116.8 MiB/s�[2K
Progress:   2.2%, ETA 01:06,  177 MiB written, speed 117.1 MiB/s�[2K
Progress:   3.0%, ETA 01:05,  235 MiB written, speed 116.8 MiB/s�[2K
Progress:   3.7%, ETA 01:05,  295 MiB written, speed 117.0 MiB/s�[2K
Progress:   4.5%, ETA 01:04,  353 MiB written, speed 116.6 MiB/s�[2K
Progress:   5.2%, ETA 01:04,  412 MiB written, speed 116.8 MiB/s�[2K
Progress:   6.0%, ETA 01:03,  472 MiB written, speed 117.0 MiB/s�[2K
Progress:   6.7%, ETA 01:03,  531 MiB written, speed 117.1 MiB/s�[2K
Progress:   7.5%, ETA 01:02,  591 MiB written, speed 117.2 MiB/s�[2K
Progress:   8.2%, ETA 01:02,  651 MiB written, speed 117.3 MiB/s�[2K
Progress:   9.0%, ETA 01:01,  711 MiB written, speed 117.4 MiB/s�[2K
Progress:   9.7%, ETA 01:00,  771 MiB written, speed 117.4 MiB/s�[2K
Progress:  10.4%, ETA 01:00,  825 MiB written, speed 116.7 MiB/s�[2K
Progress:  11.2%, ETA 01:00,  885 MiB written, speed 116.8 MiB/s�[2K
Progress:  11.9%, ETA 00:59,  944 MiB written, speed 116.9 MiB/s�[2K
Progress:  12.7%, ETA 00:59, 1004 MiB written, speed 116.9 MiB/s�[2K
Progress:  13.4%, ETA 00:58, 1064 MiB written, speed 117.0 MiB/s�[2K
Progress:  14.2%, ETA 00:58, 1123 MiB written, speed 117.1 MiB/s�[2K
Progress:  14.9%, ETA 00:57, 1178 MiB written, speed 116.7 MiB/s�[2K
Progress:  15.6%, ETA 00:57, 1237 MiB written, speed 116.7 MiB/s�[2K
Progress:  16.3%, ETA 00:56, 1296 MiB written, speed 116.7 MiB/s�[2K
Progress:  17.1%, ETA 00:56, 1354 MiB written, speed 116.6 MiB/s�[2K
Progress:  17.8%, ETA 00:55, 1411 MiB written, speed 116.4 MiB/s�[2K
Progress:  18.5%, ETA 00:55, 1468 MiB written, speed 116.3 MiB/s�[2K
Progress:  19.2%, ETA 00:55, 1526 MiB written, speed 116.2 MiB/s�[2K
Progress:  20.0%, ETA 00:54, 1583 MiB written, speed 116.1 MiB/s�[2K
Progress:  20.7%, ETA 00:54, 1641 MiB written, speed 116.1 MiB/s�[2K
Progress:  21.4%, ETA 00:53, 1698 MiB written, speed 115.9 MiB/s�[2K
Progress:  22.1%, ETA 00:53, 1755 MiB written, speed 115.8 MiB/s�[2K
Progress:  22.9%, ETA 00:52, 1813 MiB written, speed 115.8 MiB/s�[2K
Progress:  23.6%, ETA 00:52, 1870 MiB written, speed 115.7 MiB/s�[2K
Progress:  24.3%, ETA 00:51, 1928 MiB written, speed 115.7 MiB/s�[2K
Progress:  25.0%, ETA 00:51, 1985 MiB written, speed 115.6 MiB/s�[2K
Progress:  25.8%, ETA 00:50, 2043 MiB written, speed 115.5 MiB/s�[2K
Progress:  26.5%, ETA 00:50, 2100 MiB written, speed 115.5 MiB/s�[2K
Progress:  27.2%, ETA 00:49, 2158 MiB written, speed 115.4 MiB/s�[2K
Progress:  28.0%, ETA 00:49, 2216 MiB written, speed 115.4 MiB/s�[2K
Progress:  28.7%, ETA 00:49, 2273 MiB written, speed 115.2 MiB/s�[2K
Progress:  29.4%, ETA 00:48, 2332 MiB written, speed 115.3 MiB/s�[2K
Progress:  29.7%, ETA 00:49, 2351 MiB written, speed 113.3 MiB/s�[2K
Progress:  30.4%, ETA 00:48, 2411 MiB written, speed 113.5 MiB/s�[2K
Progress:  31.1%, ETA 00:48, 2469 MiB written, speed 113.5 MiB/s�[2K
Progress:  31.9%, ETA 00:47, 2526 MiB written, speed 113.5 MiB/s�[2K
Progress:  32.6%, ETA 00:47, 2584 MiB written, speed 113.5 MiB/s�[2K
Progress:  33.3%, ETA 00:46, 2641 MiB written, speed 113.5 MiB/s�[2K
Progress:  34.0%, ETA 00:46, 2699 MiB written, speed 113.5 MiB/s�[2K
Progress:  34.8%, ETA 00:45, 2757 MiB written, speed 113.5 MiB/s�[2K
Progress:  35.5%, ETA 00:45, 2814 MiB written, speed 113.5 MiB/s�[2K
Progress:  36.2%, ETA 00:44, 2872 MiB written, speed 113.5 MiB/s�[2K
Progress:  36.9%, ETA 00:44, 2929 MiB written, speed 113.5 MiB/s�[2K
Progress:  37.7%, ETA 00:43, 2987 MiB written, speed 113.5 MiB/s�[2K
Progress:  38.4%, ETA 00:43, 3045 MiB written, speed 113.5 MiB/s�[2K
Progress:  39.1%, ETA 00:42, 3102 MiB written, speed 113.5 MiB/s�[2K
Progress:  39.9%, ETA 00:41, 3160 MiB written, speed 113.5 MiB/s�[2K
Progress:  40.6%, ETA 00:41, 3217 MiB written, speed 113.5 MiB/s�[2K
Progress:  41.3%, ETA 00:40, 3275 MiB written, speed 113.5 MiB/s�[2K
Progress:  42.0%, ETA 00:40, 3333 MiB written, speed 113.5 MiB/s�[2K
Progress:  42.8%, ETA 00:39, 3390 MiB written, speed 113.5 MiB/s�[2K
Progress:  43.5%, ETA 00:39, 3448 MiB written, speed 113.5 MiB/s�[2K
Progress:  43.8%, ETA 00:39, 3470 MiB written, speed 112.4 MiB/s�[2K
Progress:  44.5%, ETA 00:39, 3528 MiB written, speed 112.4 MiB/s�[2K
Progress:  45.2%, ETA 00:38, 3585 MiB written, speed 112.4 MiB/s�[2K
Progress:  46.0%, ETA 00:38, 3643 MiB written, speed 112.4 MiB/s�[2K
Progress:  46.7%, ETA 00:37, 3701 MiB written, speed 112.5 MiB/s�[2K
Progress:  47.4%, ETA 00:37, 3758 MiB written, speed 112.5 MiB/s�[2K
Progress:  48.1%, ETA 00:36, 3816 MiB written, speed 112.5 MiB/s�[2K
Progress:  48.9%, ETA 00:36, 3873 MiB written, speed 112.5 MiB/s�[2K
Progress:  49.6%, ETA 00:35, 3931 MiB written, speed 112.5 MiB/s�[2K
Progress:  50.3%, ETA 00:34, 3989 MiB written, speed 112.6 MiB/s�[2K
Progress:  51.0%, ETA 00:34, 4046 MiB written, speed 112.6 MiB/s�[2K
Progress:  51.8%, ETA 00:33, 4104 MiB written, speed 112.6 MiB/s�[2K
Progress:  52.5%, ETA 00:33, 4161 MiB written, speed 112.6 MiB/s�[2K
Progress:  53.2%, ETA 00:32, 4219 MiB written, speed 112.6 MiB/s�[2K
Progress:  53.9%, ETA 00:32, 4277 MiB written, speed 112.6 MiB/s�[2K
Progress:  54.7%, ETA 00:31, 4334 MiB written, speed 112.6 MiB/s�[2K
Progress:  55.4%, ETA 00:31, 4392 MiB written, speed 112.7 MiB/s�[2K
Progress:  56.1%, ETA 00:30, 4449 MiB written, speed 112.7 MiB/s�[2K
Progress:  56.9%, ETA 00:30, 4507 MiB written, speed 112.7 MiB/s�[2K
Progress:  57.6%, ETA 00:29, 4565 MiB written, speed 112.7 MiB/s�[2K
Progress:  58.0%, ETA 00:29, 4595 MiB written, speed 112.0 MiB/s�[2K
Progress:  58.6%, ETA 00:29, 4644 MiB written, speed 111.8 MiB/s�[2K
Progress:  59.3%, ETA 00:28, 4701 MiB written, speed 111.8 MiB/s�[2K
Progress:  60.0%, ETA 00:28, 4759 MiB written, speed 111.9 MiB/s�[2K
Progress:  60.7%, ETA 00:27, 4816 MiB written, speed 111.9 MiB/s�[2K
Progress:  61.5%, ETA 00:27, 4874 MiB written, speed 111.9 MiB/s�[2K
Progress:  62.2%, ETA 00:26, 4931 MiB written, speed 111.9 MiB/s�[2K
Progress:  62.9%, ETA 00:26, 4988 MiB written, speed 111.9 MiB/s�[2K
Progress:  63.6%, ETA 00:25, 5046 MiB written, speed 112.0 MiB/s�[2K
Progress:  64.4%, ETA 00:25, 5103 MiB written, speed 112.0 MiB/s�[2K
Progress:  65.1%, ETA 00:24, 5161 MiB written, speed 112.0 MiB/s�[2K
Progress:  65.8%, ETA 00:24, 5218 MiB written, speed 112.0 MiB/s�[2K
Progress:  66.5%, ETA 00:23, 5275 MiB written, speed 112.0 MiB/s�[2K
Progress:  67.3%, ETA 00:23, 5333 MiB written, speed 112.1 MiB/s�[2K
Progress:  68.0%, ETA 00:22, 5390 MiB written, speed 112.1 MiB/s�[2K
Progress:  68.7%, ETA 00:22, 5448 MiB written, speed 112.1 MiB/s�[2K
Progress:  69.4%, ETA 00:21, 5505 MiB written, speed 112.1 MiB/s�[2K
Progress:  70.2%, ETA 00:21, 5563 MiB written, speed 112.1 MiB/s�[2K
Progress:  70.9%, ETA 00:20, 5621 MiB written, speed 112.1 MiB/s�[2K
Progress:  71.6%, ETA 00:20, 5678 MiB written, speed 112.1 MiB/s�[2K
Progress:  72.2%, ETA 00:19, 5720 MiB written, speed 111.7 MiB/s�[2K
Progress:  72.7%, ETA 00:19, 5766 MiB written, speed 111.5 MiB/s�[2K
Progress:  73.5%, ETA 00:18, 5823 MiB written, speed 111.5 MiB/s�[2K
Progress:  74.2%, ETA 00:18, 5881 MiB written, speed 111.6 MiB/s�[2K
Progress:  74.9%, ETA 00:17, 5938 MiB written, speed 111.6 MiB/s�[2K
Progress:  75.6%, ETA 00:17, 5995 MiB written, speed 111.6 MiB/s�[2K
Progress:  76.4%, ETA 00:16, 6053 MiB written, speed 111.6 MiB/s�[2K
Progress:  77.1%, ETA 00:16, 6110 MiB written, speed 111.6 MiB/s�[2K
Progress:  77.8%, ETA 00:15, 6168 MiB written, speed 111.6 MiB/s�[2K
Progress:  78.5%, ETA 00:15, 6225 MiB written, speed 111.7 MiB/s�[2K
Progress:  79.3%, ETA 00:14, 6283 MiB written, speed 111.7 MiB/s�[2K
Progress:  80.0%, ETA 00:14, 6341 MiB written, speed 111.7 MiB/s�[2K
Progress:  80.7%, ETA 00:13, 6398 MiB written, speed 111.7 MiB/s�[2K
Progress:  81.4%, ETA 00:13, 6456 MiB written, speed 111.7 MiB/s�[2K
Progress:  82.2%, ETA 00:12, 6513 MiB written, speed 111.7 MiB/s�[2K
Progress:  82.9%, ETA 00:12, 6571 MiB written, speed 111.8 MiB/s�[2K
Progress:  83.6%, ETA 00:11, 6629 MiB written, speed 111.8 MiB/s�[2K
Progress:  84.3%, ETA 00:11, 6686 MiB written, speed 111.8 MiB/s�[2K
Progress:  85.1%, ETA 00:10, 6744 MiB written, speed 111.8 MiB/s�[2K
Progress:  85.8%, ETA 00:10, 6801 MiB written, speed 111.8 MiB/s�[2K
Progress:  86.4%, ETA 00:09, 6849 MiB written, speed 111.5 MiB/s�[2K
Progress:  86.9%, ETA 00:09, 6893 MiB written, speed 111.3 MiB/s�[2K
Progress:  87.7%, ETA 00:08, 6951 MiB written, speed 111.3 MiB/s�[2K
Progress:  88.4%, ETA 00:08, 7008 MiB written, speed 111.3 MiB/s�[2K
Progress:  89.1%, ETA 00:07, 7066 MiB written, speed 111.3 MiB/s�[2K
Progress:  89.8%, ETA 00:07, 7123 MiB written, speed 111.3 MiB/s�[2K
Progress:  90.6%, ETA 00:06, 7180 MiB written, speed 111.3 MiB/s�[2K
Progress:  91.3%, ETA 00:06, 7238 MiB written, speed 111.4 MiB/s�[2K
Progress:  92.0%, ETA 00:05, 7295 MiB written, speed 111.4 MiB/s�[2K
Progress:  92.7%, ETA 00:05, 7353 MiB written, speed 111.4 MiB/s�[2K
Progress:  93.5%, ETA 00:04, 7410 MiB written, speed 111.4 MiB/s�[2K
Progress:  94.2%, ETA 00:04, 7467 MiB written, speed 111.4 MiB/s�[2K
Progress:  94.9%, ETA 00:03, 7525 MiB written, speed 111.5 MiB/s�[2K
Progress:  95.6%, ETA 00:03, 7582 MiB written, speed 111.5 MiB/s�[2K
Progress:  96.4%, ETA 00:02, 7640 MiB written, speed 111.5 MiB/s�[2K
Progress:  97.1%, ETA 00:02, 7697 MiB written, speed 111.5 MiB/s�[2K
Progress:  97.8%, ETA 00:01, 7755 MiB written, speed 111.5 MiB/s�[2K
Progress:  98.6%, ETA 00:01, 7813 MiB written, speed 111.5 MiB/s�[2K
Progress:  99.3%, ETA 00:00, 7870 MiB written, speed 111.5 MiB/s�[2K
Finished, time 01:11.061, 7927 MiB written, speed 111.6 MiB/s
[   79.312494] protect-stateful-partition.sh[157]: Command successful.
         Starting �[0;1;39mCreates the /dev/…ink to the stateful device.�[0m...
[   79.569398] protect-stateful-partition.sh[269]: Key slot 0 unlocked.
[   79.580270] protect-stateful-partition.sh[269]: Command successful.
[�[0;32m  OK  �[0m] Finished �[0;1;39mCreates the /dev/…mlink to the stateful device.�[0m.
[   79.683822] protect-stateful-partition.sh[286]: mke2fs 1.46.5 (30-Dec-2021)
[   79.692302] protect-stateful-partition.sh[286]: Creating filesystem with 2029522 4k blocks and 507904 inodes
[   79.704266] protect-stateful-partition.sh[286]: Filesystem UUID: f96d2a3e-3697-41fc-9004-438304a1222b
[   79.715259] protect-stateful-partition.sh[286]: Superblock backups stored on blocks:
[   79.725261] protect-stateful-partition.sh[286]: 	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
[   79.737274] protect-stateful-partition.sh[286]: Allocating group tables:  0/62�����     �����done
[   79.748256] protect-stateful-partition.sh[286]: Writing inode tables:  0/62�����     �����done
[   79.759277] protect-stateful-partition.sh[286]: Creating journal (16384 blocks): done
[   79.953652] protect-stateful-partition.sh[286]: Writing superblocks and filesystem accounting information:  0/62�����10/62�����     �����done
[�[0;32m  OK  �[0m] Finished �[0;1;39mProtect stateful partition�[0m.
[�[0;32m  OK  �[0m] Found device �[0;1;39m/dev/stateful�[0m.
         Starting �[0;1;39mFile System Check on /dev/stateful�[0m...
[   80.047086] systemd-fsck[298]: /dev/mapper/protected_stateful_partition: clean, 11/507904 files, 57186/2029522 blocks
[�[0;32m  OK  �[0m] Finished �[0;1;39mFile System Check on /dev/stateful�[0m.
         Mounting �[0;1;39m/mnt/stateful_partition�[0m...
[   80.227167] EXT4-fs (dm-3): mounted filesystem with ordered data mode. Opts: commit=30. Quota mode: none.
[�[0;32m  OK  �[0m] Mounted �[0;1;39m/mnt/stateful_partition�[0m.
         Starting �[0;1;39mMake /mnt/stateful_partition private�[0m...
         Starting �[0;1;39mResize stateful partition�[0m...
[�[0;32m  OK  �[0m] Finished �[0;1;39mMake /mnt/stateful_partition private�[0m.

This commit enables a dm-crypt layer of encryption with integrity
(using authenticated encryption with additional data) on the
partition backing /dev/stateful.
This mechanism slows down boot by ~1min.
The key is stored in memory, and, as a result, the writable,
"stateful" partition is wiped on subsequent boots.
Copy link
Contributor

@michael-pregman michael-pregman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alexmwu alexmwu merged commit 87b2a5e into google:master Oct 12, 2022
@alexmwu alexmwu deleted the protected-stateful branch October 12, 2022 00:44
alexmwu added a commit to alexmwu/go-tpm-tools that referenced this pull request Dec 16, 2022
Breaking Changes:

New Features:
Add IsHardened in launch spec: google#244
Add container logging redirect policy: google#249
Add SEV-SNP attestation support: google#240
Integrity-protect stateful partition on CS image: google#251
Retry launcher OIDC token refresh with backoff: google#261
Change restart policy behavior to reboot: google#260
Add ability to GetGCEInstanceInfo from a certificate: google#267

Bug Fixes:
COS event log: require CEL events to use PCR13, add a launch separator, and don't skip unknown events: google#246
Measure LaunchSeparator event: google#247
Skip unallocated PCR selections when reading all PCRs: google#258
Remove gRPC client and use of insecure credentials: google#262
Fix server.VerifyAttestation proto merging(google#263) and defer of os.Exit(google#264): google#265

Other Changes:
Add fake verifier client: google#234
Update CI Go Version to 1.19: google#241
Add launcher integration testing support: google#255
Test multi-writer PD creation disabled: google#256
Update go-sev-guest dependency to v0.2.6: google#259
Change OIDC retry policy to hourly and add jitter to refresh time: google#266
Add wrapper cloudbuild workflow to trigger image build and testing: google#269
@alexmwu alexmwu mentioned this pull request Dec 16, 2022
alexmwu added a commit that referenced this pull request Dec 16, 2022
Breaking Changes:

New Features:
Add IsHardened in launch spec: #244
Add container logging redirect policy: #249
Add SEV-SNP attestation support: #240
Integrity-protect stateful partition on CS image: #251
Retry launcher OIDC token refresh with backoff: #261
Change restart policy behavior to reboot: #260
Add ability to GetGCEInstanceInfo from a certificate: #267

Bug Fixes:
COS event log: require CEL events to use PCR13, add a launch separator, and don't skip unknown events: #246
Measure LaunchSeparator event: #247
Skip unallocated PCR selections when reading all PCRs: #258
Remove gRPC client and use of insecure credentials: #262
Fix server.VerifyAttestation proto merging(#263) and defer of os.Exit(#264): #265

Other Changes:
Add fake verifier client: #234
Update CI Go Version to 1.19: #241
Add launcher integration testing support: #255
Test multi-writer PD creation disabled: #256
Update go-sev-guest dependency to v0.2.6: #259
Change OIDC retry policy to hourly and add jitter to refresh time: #266
Add wrapper cloudbuild workflow to trigger image build and testing: #269
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants