Azure Generation 2 VM support

[craiglpeters]

/kind feature

Describe the solution you'd like
CAPZ should enable me to create clusters with Azure Generation 2 VMs.

Anything else you would like to add:
I can't think of a use case where mixed gen 1 and gen 2 VMs are needed

[craiglpeters]

This may require changes to packer

[alexeldeib]

kubernetes-sigs/image-builder#422

also requires checking the VM size supports gen2

It would be nice if we could attempt to default gen2 over gen1 when possible similar to #1012 (comment) but doing so has all the current caveats about webhook vs. controller context.

[CecileRobertMichon]

/assign @alexeldeib

[alexeldeib]

@CecileRobertMichon what is left to close this? An example/docs maybe once we publish official gen2 images for the next k8s patch version?

[nader-ziada]

@CecileRobertMichon what is left to close this? An example/docs maybe once we publish official gen2 images for the next k8s patch version?

I can work on this work if you don't have time @alexeldeib

I think we still have to check if the machine size supports Gen2 and set that when creating the VM like you mentioned in the comment above

[alexeldeib]

oops, go for it! I finished the image builder changes but totally forgot we didn't already have the capability check.

/unassign

[nader-ziada]

thanks, will submit a PR soon

/assign

[nader-ziada]

All the images used in tests should work fine with Gen2, except for the GPU one, we are using Standard_NV6, but could probably switch to Standard_NV12s_v3 (will create a PR to test that)

https://docs.microsoft.com/en-us/azure/virtual-machines/generation-2

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue.

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue or PR with /reopen
• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[shysank]

/remove-lifecycle rotten

[mboersma]

We do need to switch to Standard_NV12s_v3 or similar for GPU tests, not only to complete this issue, but because Standard_NV6 goes away in a year:

Based on feedback we've received from customers we're happy to announce we are extending the retirement date by 1 year to 31 August 2023 for the Azure NV6, NV6_Promo, NV12, NV12_Promo, NV24, NV24_Promo virtual machines to give you more time to plan your migration.

The practical issue is getting any sort of quota for the newer SKU types in the subscription that runs CI. So far we've not had any luck doing that.

[nader-ziada]

/unassign

[invidian]

Perhaps it would make sense to close this issue and create a new one for migrating GPU tests?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

[k8s-ci-robot]

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to this:

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Reopen this issue with /reopen
• Mark this issue as fresh with /remove-lifecycle rotten
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[CecileRobertMichon]

/reopen
/remove-lifecycle rotten

@mboersma @willie-yao are we currently publishing gen 2 images or still gen 1? If not is the only remaining item to switch the GPU SKU?

[k8s-ci-robot]

@CecileRobertMichon: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten

@mboersma @willie-yao are we currently publishing gen 2 images or still gen 1? If not is the only remaining item to switch the GPU SKU?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mboersma]

@CecileRobertMichon we are not yet publishing Gen2 images, but IDK specifically what the hitch is (if there is any). We would need to make small changes to the image-builder scripts, since -gen1 is effectively hard-coded right now.

[mboersma]

Re-reading this thread, it would appear that switching GPU SKUs is the only known blocker, besides adjusting the publishing scripts.

[mboersma]

/assign

[mboersma]

/close

This issue as stated is incorrect–gen2 Azure images will work fine in CAPZ, although we currently publish reference images only in gen1 format.

[k8s-ci-robot]

@mboersma: Closing this issue.

In response to this:

/close

This issue as stated is incorrect–gen2 Azure images will work fine in CAPZ, although we currently publish reference images only in gen1 format.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.