Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Addition of kvm_capabilities and vcpu_features to custom CPU templates #3967

Merged
merged 12 commits into from
Aug 21, 2023
Merged

Addition of kvm_capabilities and vcpu_features to custom CPU templates #3967

merged 12 commits into from
Aug 21, 2023

Conversation

ShadowCurse
Copy link
Contributor

@ShadowCurse ShadowCurse commented Jul 13, 2023 •
edited by zulinx86
Loading

Changes

Main changes are additions of couple fields to the cpu templates:

  • Added ability to modify KVM capabilities checks performed by Firecracker with kvm_capabilities field.
  • (only for aarch64) Added ability to specify additional vcpu features with vcpu_features field

Reason

KVM capabilities:

  • It is useful to be able to add/remove kvm capabilities checks. Currently it is mostly used only on aarch64 in combination with vcpu_features. For x86_64 there was an issue that FC did not boot on old cpu, because some KVM feature was not available there, even though it was only used for snapshots.

VCPU features(aarch64 only):

  • With ability to specify additional vcpu features users now can enable some VCPU features that can only be enabled during vcpu initialization such as PAC or SVE.

License Acceptance

By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache 2.0 license. For more information on following
Developer Certificate of Origin and signing off your commits, please check
CONTRIBUTING.md.

PR Checklist

  • If a specific issue led to this PR, this PR closes the issue.
  • The description of changes is clear and encompassing.
  • Any required documentation changes (code and docs) are included in this PR.
  • API changes follow the Runbook for Firecracker API changes.
  • User-facing changes are mentioned in CHANGELOG.md.
  • All added/changed functionality is tested.
  • New TODOs link to an issue.
  • Commits meet contribution quality standards.
  • This functionality cannot be added in rust-vmm.

@ShadowCurse ShadowCurse self-assigned this Jul 13, 2023
@ShadowCurse ShadowCurse force-pushed the kvm_and_vcpu_config branch 17 times, most recently from e706260 to 4860e1e Compare July 18, 2023 09:38
@ShadowCurse ShadowCurse mentioned this pull request Jul 18, 2023
Merged
@ShadowCurse ShadowCurse force-pushed the kvm_and_vcpu_config branch 11 times, most recently from 3b463b3 to 3c5cfdd Compare July 21, 2023 17:56
zulinx86
zulinx86 reviewed Aug 21, 2023
View reviewed changes
src/vmm/src/vstate/vcpu/aarch64.rs Outdated Show resolved Hide resolved
tests/integration_tests/functional/test_cpu_features_aarch64.py Outdated Show resolved Hide resolved
src/vmm/src/vstate/vm.rs Outdated Show resolved Hide resolved
@ShadowCurse ShadowCurse force-pushed the kvm_and_vcpu_config branch 2 times, most recently from 44ecaf1 to 140a532 Compare August 21, 2023 11:33
zulinx86
zulinx86 previously approved these changes Aug 21, 2023
View reviewed changes
Copy link
Contributor

@zulinx86 zulinx86 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me say thank you again for your effort on organizing commits! LGTM!

@ShadowCurse ShadowCurse force-pushed the kvm_and_vcpu_config branch 3 times, most recently from da6cb39 to 0ce4f90 Compare August 21, 2023 12:38
@ShadowCurse
refactor(vmm): combined KvmContext and Vm
85541a7 
Moved all code from `KvmContext` into `Vmm::new`.
This simplifies code, because before both struct were always used
together.
Also removed `setup_kvm_vm` method, because it is too
short now.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
feat(cpu templates): ability to specify additional vcpu features
9ae122a 
Added `vcpu_features` field to the cpu templates for aarch64.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
fix(aarch64): minor fix of json strings in cpu templates tests
586d879 
Removed incorrectly placed commas from json strings in the tests.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
refactor(aarch64): removed redundant helper method
3fd4ac6 
Removed `init_vcpu` helper method from aarch64
vcpu tests.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
refactor(vmm): vcpu.init and vcpu.restore updates
ba3b649 
- `vcpu.init` is moved out of `create_vcpus` method.
- `vcpu`s states are now restored before `vcpu`s are put into separate
threads. This in turn removed the need for `VcpuEvent::RestoreState`
`VcpuResponce::RestoreState` and `restore_vcpu_states` method.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
feat(aarch64): added vcpu_features parameter to vcpu.init
7d583ce 
Added `vcpu_features` parameter to `vcpu.init`.
This parameter will be used in the next commit to modify enabled
vcpu features.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
feat(aarch64): added kvi to the vcpu state
f2a318e 
Added `kvi` to the vcpu state on aarch64 to keep
track of enabled vcpu features that could be
modified with cpu templates.
This required updates of `kvm-bindings` and
`kvm-ioctls` dependencies.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
feat: new snapshot version check
292515e 
Added new snapshot version check to prevent creation
of snapshots with version less than 1.5 if `vcpu_features` field
is used in provided cpu template.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
feat(cpu templates): added kvm capabilities to the cpu templates
17984f7 
Added `KvmCapabilities` to the cpu templates for both
x86_64 and aarch64. This allows users to add or remove
KVM capabilities check that FC is performing.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
test: added integration test for snapshot version
de4d24f 
If cpu template uses `vcpu_features` or `kvm_capabilities`
fields then minimal snap version that can be created is 1.5.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
test(aarch64): integration test for vcpu_features and kvm_capabilities
57ed08e 
Added integration test for `vcpu_features` and `kvm_capabilities`
fields in cpu templates.

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse
doc(cpu templates): updated documentation
3eacc79 
- Added new `kvm_capabilities` and `vcpu_features` fields descriptions
to the `schema.json`
- Updated `cpu-templates.md`
- Updated `CHANGELOG.md`

Signed-off-by: Egor Lazarchuk <yegorlz@amazon.co.uk>
@ShadowCurse ShadowCurse merged commit 058983d into firecracker-microvm:main Aug 21, 2023
5 of 7 checks passed
@ShadowCurse ShadowCurse deleted the kvm_and_vcpu_config branch August 21, 2023 13:47
zulinx86 added a commit to zulinx86/firecracker that referenced this pull request Sep 13, 2023
@zulinx86
test: Update fingerprint files
8ca2c5c 
Changes on guest CPU config are as follows:
- CPUID.0x7:EBX
  - ERMS (bit 9) on AMD (marked as reserved in APM)
- CPUID.0x7:EDX
  - FSRM (bit 4) on AMD + kernel 5.10 and 6.10 (marked as reserved in
    APM)
- IA32_ARCH_CAPABILITIES MSR (index: 0x10A)
  - GDS_CTRL (bit 25) on Intel + kernel 4.14
  - GDS_NO (bit 26) on Intel

kvm_capabilities and vcpu_features fields were added in PR firecracker-microvm#3967.

Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
@zulinx86 zulinx86 mentioned this pull request Sep 13, 2023
Merged
4 tasks
wearyzen pushed a commit to zulinx86/firecracker that referenced this pull request Sep 14, 2023
@zulinx86 @wearyzen
test: Update fingerprint files
1bdb6bc 
Changes on guest CPU config are as follows:
- CPUID.0x7:EBX
  - ERMS (bit 9) on AMD (marked as reserved in APM)
- CPUID.0x7:EDX
  - FSRM (bit 4) on AMD + kernel 5.10 and 6.10 (marked as reserved in
    APM)
- IA32_ARCH_CAPABILITIES MSR (index: 0x10A)
  - GDS_CTRL (bit 25) on Intel + kernel 4.14
  - GDS_NO (bit 26) on Intel

kvm_capabilities and vcpu_features fields were added in PR firecracker-microvm#3967.

Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
wearyzen pushed a commit that referenced this pull request Sep 14, 2023
@zulinx86 @wearyzen
test: Update fingerprint files
d6b309a 
Changes on guest CPU config are as follows:
- CPUID.0x7:EBX
  - ERMS (bit 9) on AMD (marked as reserved in APM)
- CPUID.0x7:EDX
  - FSRM (bit 4) on AMD + kernel 5.10 and 6.10 (marked as reserved in
    APM)
- IA32_ARCH_CAPABILITIES MSR (index: 0x10A)
  - GDS_CTRL (bit 25) on Intel + kernel 4.14
  - GDS_NO (bit 26) on Intel

kvm_capabilities and vcpu_features fields were added in PR #3967.

Signed-off-by: Takahiro Itazuri <itazur@amazon.com>
@JonathanWoollett-Light JonathanWoollett-Light added this to the 1.5 milestone Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pb8o pb8o pb8o left review comments

@kalyazin kalyazin kalyazin approved these changes

@zulinx86 zulinx86 zulinx86 approved these changes

@xmarcalx xmarcalx Awaiting requested review from xmarcalx xmarcalx is a code owner

Assignees

@ShadowCurse ShadowCurse

Labels
Status: Awaiting review Indicates that a pull request is ready to be reviewed Type: Enhancement Indicates new feature requests
Projects
None yet
Milestone
1.5.0
Development

Successfully merging this pull request may close these issues.
6 participants
@ShadowCurse @kalyazin @zulinx86 @xmarcalx @pb8o @JonathanWoollett-Light