Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upcoming: [M3-8960] - Update Kubernetes version upgrade components for LKE-E #11415

Merged
merged 15 commits into from
Dec 17, 2024
Merged

upcoming: [M3-8960] - Update Kubernetes version upgrade components for LKE-E #11415

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
29f4052
Make a hook because this is getting repetitive
mjac0bs Dec 12, 2024
efb61c7
Determine enterprise or standard versions with hook
mjac0bs Dec 12, 2024
492a7ed
Update versions type for util - need to test
mjac0bs Dec 12, 2024
26ee5e3
Improve hook and use in cluster create flow
mjac0bs Dec 13, 2024
0ec7415
Move hook to kubeUtils
mjac0bs Dec 13, 2024
b2ab4fb
Add test coverage and factories
mjac0bs Dec 13, 2024
09d286a
Clean up kubeUtils.test.ts
mjac0bs Dec 13, 2024
d9a3600
Add kubeUtils helper function test coverage for LKE-E
mjac0bs Dec 13, 2024
eb8e9b1
Update Cypress test coverage for enterprise version upgrades
mjac0bs Dec 13, 2024
a1d8be9
Clean up
mjac0bs Dec 13, 2024
8ea979e
Added changeset: Update Kubernetes version upgrade components for LKE-E
mjac0bs Dec 13, 2024
7340553
Fix test failure by mocking the LKE-E capability needed
mjac0bs Dec 13, 2024
1151b7b
Update TODO
mjac0bs Dec 13, 2024
0f35afa
Address PR feedback: increment ids in tiered version factories
mjac0bs Dec 17, 2024
9077ebc
Merge branch 'develop' into M3-8960-update-kube-version-upgrade-compo…
mjac0bs Dec 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 188 additions & 2 deletions packages/manager/cypress/e2e/core/kubernetes/lke-update.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import {
mockGetControlPlaneACL,
mockUpdateControlPlaneACLError,
mockGetControlPlaneACLError,
mockGetTieredKubernetesVersions,
} from 'support/intercepts/lke';
import {
mockGetLinodeType,
Expand Down Expand Up @@ -133,7 +134,7 @@ describe('LKE cluster updates', () => {
* - Confirms that Kubernetes upgrade prompt is shown when not up-to-date.
* - Confirms that Kubernetes upgrade prompt is hidden when up-to-date.
*/
it('can upgrade kubernetes version from the details page', () => {
it('can upgrade standard kubernetes version from the details page', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this distinction! (+ the other test)

const oldVersion = '1.25';
const newVersion = '1.26';

Expand Down Expand Up @@ -235,7 +236,7 @@ describe('LKE cluster updates', () => {
ui.toast.findByMessage('Recycle started successfully.');
});

it('can upgrade the kubernetes version from the landing page', () => {
it('can upgrade the standard kubernetes version from the landing page', () => {
const oldVersion = '1.25';
const newVersion = '1.26';

Expand Down Expand Up @@ -294,6 +295,191 @@ describe('LKE cluster updates', () => {
cy.findByText(newVersion).should('be.visible');
});

/*
mjac0bs marked this conversation as resolved.
Show resolved Hide resolved
* - Confirms UI flow of upgrading Kubernetes enterprise version using mocked API requests.
* - Confirms that Kubernetes upgrade prompt is shown when not up-to-date.
* - Confirms that Kubernetes upgrade prompt is hidden when up-to-date.
*/
it('can upgrade enterprise kubernetes version from the details page', () => {
const oldVersion = '1.31.1+lke1';
const newVersion = '1.31.1+lke2';
Comment on lines +304 to +305
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are new LKE-E versions denoted by a higher number after lke or the "base" standard version number?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both? Versioning is what confuses me the most right now, tbh.

The LKE folks said earier:

LKE-E exposes the patch version, and an overall "lke" version, so the customer is able to specifically choose when to include any upgrade of any component. in both LKE and LKE-E only the "latest" version of each kubernetes minor version is available. In other words, if 1.31.1+lke1 is installed, and then 1.31.2+lke1 comes out. New installs can only choose 1.31.2+lke1, not any arbitrary version

So there are still minor version increases, and patch increases, and then there's this new(ly exposed?) "lke" version and that can change too.

And from some LKE-E docs:

Only one patch version per minor version will be supported at any point in time. For example, at a point in time, the following would be the supported versions “v1.29.8-lke1”, “v1.30.2-lke2”, “v1.31.1-lke1”. Customers will be notified and prompted in Cloud Manager when there is a new patch version available. For example, a customer who created a cluster with ““v1.30.1-lke1” would be notified when ““v1.30.1-lke2” or “v1.30.2-lke2”” is available.

I think that last part is consistent with the test spec here. We have a patch version that gets a new lke version, so that is considered the latest patch to update to.


// TODO LKE-E: Remove once feature is in GA
mockAppendFeatureFlags({
lkeEnterprise: { enabled: true, la: true },
});

const mockCluster = kubernetesClusterFactory.build({
k8s_version: oldVersion,
tier: 'enterprise',
});

const mockClusterUpdated = {
...mockCluster,
k8s_version: newVersion,
};

const upgradePrompt =
'A new version of Kubernetes is available (1.31.1+lke2).';

const upgradeNotes = [
'Once the upgrade is complete you will need to recycle all nodes in your cluster',
// Confirm that the old version and new version are both shown.
oldVersion,
newVersion,
];

mockGetCluster(mockCluster).as('getCluster');
mockGetTieredKubernetesVersions('enterprise', [
{ id: newVersion, tier: 'enterprise' },
{ id: oldVersion, tier: 'enterprise' },
]).as('getTieredVersions');
mockGetClusterPools(mockCluster.id, mockNodePools).as('getNodePools');
mockUpdateCluster(mockCluster.id, mockClusterUpdated).as('updateCluster');
mockGetDashboardUrl(mockCluster.id);
mockGetApiEndpoints(mockCluster.id);

cy.visitWithLogin(`/kubernetes/clusters/${mockCluster.id}`);
cy.wait(['@getCluster', '@getNodePools', '@getTieredVersions']);

// Confirm that upgrade prompt is shown.
cy.findByText(upgradePrompt).should('be.visible');
ui.button
.findByTitle('Upgrade Version')
.should('be.visible')
.should('be.enabled')
.click();

ui.dialog
.findByTitle(
`Step 1: Upgrade ${mockCluster.label} to Kubernetes ${newVersion}`
)
.should('be.visible')
.within(() => {
upgradeNotes.forEach((note: string) => {
cy.findAllByText(note, { exact: false }).should('be.visible');
});

ui.button
.findByTitle('Upgrade Version')
.should('be.visible')
.should('be.enabled')
.click();
});

// Wait for API response and assert toast message is shown.
cy.wait('@updateCluster');

// Verify the banner goes away because the version update has happened
cy.findByText(upgradePrompt).should('not.exist');

mockRecycleAllNodes(mockCluster.id).as('recycleAllNodes');

const stepTwoDialogTitle = 'Step 2: Recycle All Cluster Nodes';

ui.dialog
.findByTitle(stepTwoDialogTitle)
.should('be.visible')
.within(() => {
cy.findByText('Kubernetes version has been updated successfully.', {
exact: false,
}).should('be.visible');

cy.findByText(
'For the changes to take full effect you must recycle the nodes in your cluster.',
{ exact: false }
).should('be.visible');

ui.button
.findByTitle('Recycle All Nodes')
.should('be.visible')
.should('be.enabled')
.click();
});

// Verify clicking the "Recycle All Nodes" makes an API call
cy.wait('@recycleAllNodes');

// Verify the upgrade dialog closed
cy.findByText(stepTwoDialogTitle).should('not.exist');

// Verify the banner is still gone after the flow
cy.findByText(upgradePrompt).should('not.exist');

// Verify the version is correct after the update
cy.findByText(`Version ${newVersion}`);

ui.toast.findByMessage('Recycle started successfully.');
});

it('can upgrade the enterprise kubernetes version from the landing page', () => {
const oldVersion = '1.31.1+lke1';
const newVersion = '1.32.1+lke2';

// TODO LKE-E: Remove once feature is in GA
mockAppendFeatureFlags({
lkeEnterprise: { enabled: true, la: true },
});

const cluster = kubernetesClusterFactory.build({
k8s_version: oldVersion,
tier: 'enterprise',
});

const updatedCluster = { ...cluster, k8s_version: newVersion };

mockGetClusters([cluster]).as('getClusters');
mockGetTieredKubernetesVersions('enterprise', [
{ id: newVersion, tier: 'enterprise' },
{ id: oldVersion, tier: 'enterprise' },
]).as('getTieredVersions');
mockUpdateCluster(cluster.id, updatedCluster).as('updateCluster');
mockRecycleAllNodes(cluster.id).as('recycleAllNodes');

cy.visitWithLogin(`/kubernetes/clusters`);

cy.wait(['@getClusters', '@getTieredVersions']);

cy.findByText(oldVersion).should('be.visible');

cy.findByText('UPGRADE')
.should('be.visible')
.should('be.enabled')
.click();

ui.dialog
.findByTitle(
`Step 1: Upgrade ${cluster.label} to Kubernetes ${newVersion}`
)
.should('be.visible');

mockGetClusters([updatedCluster]).as('getClusters');

ui.button
.findByTitle('Upgrade Version')
.should('be.visible')
.should('be.enabled')
.click();

cy.wait(['@updateCluster', '@getClusters']);

ui.dialog
.findByTitle('Step 2: Recycle All Cluster Nodes')
.should('be.visible');

ui.button
.findByTitle('Recycle All Nodes')
.should('be.visible')
.should('be.enabled')
.click();

cy.wait('@recycleAllNodes');

ui.toast.assertMessage('Recycle started successfully.');

cy.findByText(newVersion).should('be.visible');
});

/*
* - Confirms node, node pool, and cluster recycling UI flow using mocked API data.
* - Confirms that user is warned that recycling recreates nodes and may take a while.
Expand Down
15 changes: 15 additions & 0 deletions packages/manager/src/factories/kubernetesCluster.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import type {
KubernetesControlPlaneACLPayload,
KubernetesDashboardResponse,
KubernetesEndpointResponse,
KubernetesTieredVersion,
KubernetesVersion,
PoolNodeResponse,
} from '@linode/api-v4/lib/kubernetes/types';
Expand Down Expand Up @@ -78,6 +79,20 @@ export const kubernetesVersionFactory = Factory.Sync.makeFactory<KubernetesVersi
}
);

export const kubernetesStandardTierVersionFactory = Factory.Sync.makeFactory<KubernetesTieredVersion>(
{
id: '1.31',
tier: 'standard',
}
);

export const kubernetesEnterpriseTierVersionFactory = Factory.Sync.makeFactory<KubernetesTieredVersion>(
{
id: 'v1.31.1+lke1',
tier: 'enterprise',
}
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make any sense to use Factory.each((i) => ... on these IDs to let the values increment, or would it just complicate things / be inconsistent with the actual data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 0f35afa. 👍🏼 I incremented the minor version in the standard tier factory, and the patch in the enterprise tier factory. Tests continue to pass and I think this will be consistent enough with possible data/not overly complicated.


export const kubernetesControlPlaneACLOptionsFactory = Factory.Sync.makeFactory<ControlPlaneACLOptions>(
{
addresses: {
Expand Down
14 changes: 8 additions & 6 deletions packages/manager/src/features/Kubernetes/ClusterList/KubernetesClusterRow.tsx
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any thoughts about unit tests to confirm that the Upgrade chip is present when a new LKE version is available and is absent when there is no new version? Don't think it needs to block this PR but might be good to have as a small standalone test ticket.

Copy link
Contributor Author

@mjac0bs mjac0bs Dec 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call out - yeah, let's do this in a standalone test ticket. It looks like the LKE landing page spec isn't testing version upgrades chip at all currently. I made M3-9023 and will get that done as part of this LKE-E epic!

Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import { Chip } from '@linode/ui';
import { KubeNodePoolResponse, KubernetesCluster } from '@linode/api-v4';
import Grid from '@mui/material/Unstable_Grid2';
import * as React from 'react';
import { Link } from 'react-router-dom';
Expand All @@ -9,20 +8,20 @@ import { DateTimeDisplay } from 'src/components/DateTimeDisplay';
import { Hidden } from 'src/components/Hidden';
import { TableCell } from 'src/components/TableCell';
import { TableRow } from 'src/components/TableRow';
import {
useAllKubernetesNodePoolQuery,
useKubernetesVersionQuery,
} from 'src/queries/kubernetes';
import { useAllKubernetesNodePoolQuery } from 'src/queries/kubernetes';
import { useRegionsQuery } from 'src/queries/regions/regions';
import { useSpecificTypes } from 'src/queries/types';
import { extendTypesQueryResult } from 'src/utilities/extendType';

import {
getNextVersion,
getTotalClusterMemoryCPUAndStorage,
useLkeStandardOrEnterpriseVersions,
} from '../kubeUtils';
import { ClusterActionMenu } from './ClusterActionMenu';

import type { KubeNodePoolResponse, KubernetesCluster } from '@linode/api-v4';

const useStyles = makeStyles()(() => ({
clusterRow: {
'&:before': {
Expand Down Expand Up @@ -64,14 +63,17 @@ export const KubernetesClusterRow = (props: Props) => {
const { cluster, openDeleteDialog, openUpgradeDialog } = props;
const { classes } = useStyles();

const { data: versions } = useKubernetesVersionQuery();
const { data: pools } = useAllKubernetesNodePoolQuery(cluster.id);
const typesQuery = useSpecificTypes(pools?.map((pool) => pool.type) ?? []);
const types = extendTypesQueryResult(typesQuery);
const { data: regions } = useRegionsQuery();

const region = regions?.find((r) => r.id === cluster.region);

const { versions } = useLkeStandardOrEnterpriseVersions(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We default to standard if the tier is undefined (LKE-E feature not enabled) because all current clusters are 'standard'.

cluster.tier ?? 'standard'
);

const nextVersion = getNextVersion(cluster.k8s_version, versions ?? []);

const hasUpgrade = nextVersion !== null;
Expand Down
36 changes: 8 additions & 28 deletions packages/manager/src/features/Kubernetes/CreateCluster/CreateCluster.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import {
getLatestVersion,
useAPLAvailability,
useIsLkeEnterpriseEnabled,
useLkeStandardOrEnterpriseVersions,
} from 'src/features/Kubernetes/kubeUtils';
import { useAccount } from 'src/queries/account/account';
import {
Expand All @@ -28,9 +29,7 @@ import {
import {
useCreateKubernetesClusterBetaMutation,
useCreateKubernetesClusterMutation,
useKubernetesTieredVersionsQuery,
useKubernetesTypesQuery,
useKubernetesVersionQuery,
} from 'src/queries/kubernetes';
import { useRegionsQuery } from 'src/queries/regions/regions';
import { useAllTypes } from 'src/queries/types';
Expand Down Expand Up @@ -130,31 +129,16 @@ export const CreateCluster = () => {
mutateAsync: createKubernetesClusterBeta,
} = useCreateKubernetesClusterBetaMutation();

const {
data: _versionData,
isError: versionLoadError,
isLoading: versionLoading,
} = useKubernetesVersionQuery();

const {
data: enterpriseTierVersionData,
isLoading: enterpriseTierVersionDataIsLoading,
} = useKubernetesTieredVersionsQuery('enterprise');

const {
isLkeEnterpriseLAFeatureEnabled,
isLkeEnterpriseLAFlagEnabled,
} = useIsLkeEnterpriseEnabled();

/**
* If LKE-E is enabled, use the new /versions/<tier> endpoint data, which supports enterprise tiers.
* If LKE-E is disabled, use the data from the existing /versions endpoint.
* @todo LKE-E: Clean up use of versionData once LKE-E is in GA.
*/
const versionData =
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic was moved inside useLkeStandardOrEnterpriseVersions.

isLkeEnterpriseLAFeatureEnabled && selectedTier === 'enterprise'
? enterpriseTierVersionData
: _versionData;
const {
isLoadingVersions,
versions: versionData,
versionsError,
} = useLkeStandardOrEnterpriseVersions(selectedTier);

const versions = (versionData ?? []).map((thisVersion) => ({
label: thisVersion.id,
Expand Down Expand Up @@ -303,7 +287,7 @@ export const CreateCluster = () => {
selectedRegionID: selectedRegionId,
});

if (typesError || regionsError || versionLoadError) {
if (typesError || regionsError || versionsError) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updates with error and loading variables were just to update to the values returned from the hook.

Now the tiered version query is disabled when the LKE-E feature is disabled (and isFetching will be false), so we shouldn't need to check isLkeEnterpriseLAFeatureEnabled && enterpriseTierVersionDataIsLoading.

// This information is necessary to create a Cluster. Otherwise, show an error state.
return <ErrorState errorText="An unexpected error occurred." />;
}
Expand Down Expand Up @@ -389,14 +373,10 @@ export const CreateCluster = () => {
disableClearable={!!version}
errorText={errorMap.k8s_version}
label="Kubernetes Version"
loading={isLoadingVersions}
options={versions}
placeholder={' '}
value={versions.find((v) => v.value === version) ?? null}
loading={
versionLoading ||
(isLkeEnterpriseLAFeatureEnabled &&
enterpriseTierVersionDataIsLoading)
}
/>
{showAPL && (
<>
Expand Down
1 change: 1 addition & 0 deletions packages/manager/src/features/Kubernetes/KubernetesClusterDetail/KubernetesClusterDetail.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ export const KubernetesClusterDetail = () => {
<UpgradeKubernetesVersionBanner
clusterID={cluster?.id}
clusterLabel={cluster?.label}
clusterTier={cluster?.tier ?? 'standard'}
currentVersion={cluster?.k8s_version}
/>
<LandingHeader
Expand Down
Loading
Loading