Skip to content

Commit

Permalink
[Upgrade Assistant] Update "remove settings" action (#124544)
Browse files Browse the repository at this point in the history
  • Loading branch information
alisonelizabeth authored Feb 28, 2022
1 parent 43b0104 commit b988d3b
Show file tree
Hide file tree
Showing 24 changed files with 886 additions and 102 deletions.
60 changes: 54 additions & 6 deletions x-pack/plugins/upgrade_assistant/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,7 @@ Elasticsearch deprecations can be handled in a number of ways:
* Create an index alias pointing from the original index name to the prefixed index name.
* Reindex from the original index into the prefixed index.
* Delete the old index and rename the prefixed index.
* **Updating index settings.** Some index settings will need to be updated, which doesn't require a
reindex. An example of this is the "Remove deprecated settings" button, which is shown when [deprecated translog settings](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-translog.html#index-modules-translog-retention) are detected ([#93293](https://github.com/elastic/kibana/pull/93293)).
* **Removing settings.** Some index and cluser settings are deprecated and need to be removed. The Upgrade Assistant provides a way to auto-resolve these settings via a "Remove deprecated settings" button.
* **Upgrading or deleting snapshots**. This is specific to Machine Learning. If a user has old Machine Learning job model snapshots, they will need to be upgraded or deleted. The Upgrade Assistant provides a way to resolve this automatically for the user ([#100066](https://github.com/elastic/kibana/pull/100066)).
* **Following the docs.** The Deprecation Info API provides links to the deprecation docs. Users
will follow these docs to address the problem and make these warnings or errors disappear in the
Expand Down Expand Up @@ -107,17 +106,66 @@ To test the Elasticsearch deprecations page ([#107053](https://github.com/elasti
yarn es snapshot --license trial -E path.data=./path_to_6.x_ml_snapshots
```

**3. Removing deprecated index settings**
**3. Removing deprecated settings**

The Upgrade Assistant currently only supports fixing deprecated translog index settings. However [the code](https://github.com/elastic/kibana/blob/main/x-pack/plugins/upgrade_assistant/common/constants.ts#L22) is written in a way to add support for more if necessary. Run the following Console command to trigger the deprecation warning:
The Upgrade Assistant supports removing deprecated index and cluster settings. This is determined based on the `actions` array returned from the deprecation info API. It currently does not support removing affix settings. See https://github.com/elastic/elasticsearch/pull/84246 for more details.

Run the following Console commands to trigger deprecation issues for cluster and index settings:

```
// Can be set as persistent or transient
PUT /_cluster/settings
{
"persistent" : {
"script.context.filter.cache_max_size": 10,
"script.context.update.cache_max_size": 10,
"script.context.update.max_compilations_rate": "10/1m",
"discovery.zen.minimum_master_nodes": 10,
"discovery.zen.commit_timeout": "10s",
"discovery.zen.no_master_block": "all",
"discovery.zen.publish_diff.enable": true,
"discovery.zen.publish_timeout": "10s",
"indices.lifecycle.step.master_timeout": "10s",
"script.context.field.max_compilations_rate": "10/1m",
"script.context.score.max_compilations_rate": "10/1m",
"script.context.interval.cache_expire": "10s",
"script.context.moving-function.cache_expire": "10s",
"xpack.watcher.history.cleaner_service.enabled": true,
"cluster.routing.allocation.exclude._tier": "data_warm",
"cluster.routing.allocation.include._tier": "data_cold",
"cluster.routing.allocation.require._tier": "data_hot",
"xpack.monitoring.elasticsearch.collection.enabled": true,
"xpack.monitoring.collection.enabled": true,
"xpack.monitoring.collection.interval": "20s",
"xpack.monitoring.collection.ccr.stats.timeout": "20s",
"xpack.monitoring.collection.cluster.stats.timeout": "20s",
"xpack.monitoring.collection.enrich.stats.timeout": "20s",
"xpack.monitoring.collection.index.recovery.timeout": "20s",
"xpack.monitoring.collection.index.stats.timeout": "20s",
"xpack.monitoring.collection.ml.job.stats.timeout": "20s",
"xpack.monitoring.collection.node.stats.timeout": "20s",
"xpack.monitoring.collection.index.recovery.active_only": true,
"xpack.monitoring.history.duration": "2d",
"xpack.monitoring.migration.decommission_alerts": true,
"cluster.routing.allocation.shard_state.reroute.priority": "HIGH",
"cluster.routing.allocation.disk.include_relocations": true
}
}
```

```
PUT deprecated_settings
{
"settings": {
"translog.retention.size": "1b",
"translog.retention.age": "5m",
"index.indexing.slowlog.level": "warn",
"index.max_adjacency_matrix_filters": 10,
"index.routing.allocation.exclude._tier": "data_warm",
"index.routing.allocation.include._tier": "data_hot",
"index.routing.allocation.require._tier": "data_cold",
"index.search.slowlog.level": "warn",
"index.soft_deletes.enabled": true,
"index.translog.retention.size": "1mb",
"index.translog.retention.age": "5s"
}
}
```
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

import { act } from 'react-dom/test-utils';

import { setupEnvironment } from '../helpers';
import { ElasticsearchTestBed, setupElasticsearchPage } from './es_deprecations.helpers';
import { esDeprecationsMockResponse } from './mocked_responses';

describe('Cluster settings deprecation flyout', () => {
let testBed: ElasticsearchTestBed;
const { server, httpRequestsMockHelpers } = setupEnvironment();
const clusterSettingDeprecation = esDeprecationsMockResponse.deprecations[4];

afterAll(() => {
server.restore();
});

beforeEach(async () => {
httpRequestsMockHelpers.setLoadEsDeprecationsResponse(esDeprecationsMockResponse);
httpRequestsMockHelpers.setReindexStatusResponse({
reindexOp: null,
warnings: [],
hasRequiredPrivileges: true,
meta: {
indexName: 'foo',
reindexName: 'reindexed-foo',
aliases: [],
},
});

await act(async () => {
testBed = await setupElasticsearchPage({ isReadOnlyMode: false });
});

const { actions, component } = testBed;
component.update();
await actions.table.clickDeprecationRowAt('clusterSetting', 0);
});

test('renders a flyout with deprecation details', async () => {
const { find, exists } = testBed;

expect(exists('clusterSettingsDetails')).toBe(true);
expect(find('clusterSettingsDetails.flyoutTitle').text()).toContain(
clusterSettingDeprecation.message
);
expect(find('clusterSettingsDetails.documentationLink').props().href).toBe(
clusterSettingDeprecation.url
);
expect(exists('removeClusterSettingsPrompt')).toBe(true);
});

it('removes deprecated cluster settings', async () => {
const { find, actions, exists } = testBed;

httpRequestsMockHelpers.setClusterSettingsResponse({
acknowledged: true,
persistent: {},
transietn: {},
});

expect(exists('clusterSettingsDetails.warningDeprecationBadge')).toBe(true);

await actions.clusterSettingsDeprecationFlyout.clickDeleteSettingsButton();

const request = server.requests[server.requests.length - 1];

expect(request.method).toBe('POST');
expect(request.url).toBe(`/api/upgrade_assistant/cluster_settings`);
expect(request.status).toEqual(200);

// Verify the "Resolution" column of the table is updated
expect(find('clusterSettingsResolutionStatusCell').at(0).text()).toEqual(
'Deprecated settings removed'
);

// Reopen the flyout
await actions.table.clickDeprecationRowAt('clusterSetting', 0);

// Verify prompt to remove setting no longer displays
expect(find('removeSettingsPrompt').length).toEqual(0);
// Verify the action button no longer displays
expect(find('clusterSettingsDetails.deleteSettingsButton').length).toEqual(0);
// Verify the badge got marked as resolved
expect(exists('clusterSettingsDetails.resolvedDeprecationBadge')).toBe(true);
});

it('handles failure', async () => {
const { find, actions } = testBed;
const error = {
statusCode: 500,
error: 'Remove cluster settings error',
message: 'Remove cluster settings error',
};

httpRequestsMockHelpers.setClusterSettingsResponse(undefined, error);

await actions.clusterSettingsDeprecationFlyout.clickDeleteSettingsButton();

const request = server.requests[server.requests.length - 1];

expect(request.method).toBe('POST');
expect(request.url).toBe(`/api/upgrade_assistant/cluster_settings`);
expect(request.status).toEqual(500);

// Verify the "Resolution" column of the table is updated
expect(find('clusterSettingsResolutionStatusCell').at(0).text()).toEqual(
'Settings removal failed'
);

// Reopen the flyout
await actions.table.clickDeprecationRowAt('clusterSetting', 0);

// Verify the flyout shows an error message
expect(find('clusterSettingsDetails.deleteClusterSettingsError').text()).toContain(
'Error deleting cluster settings'
);
// Verify the remove settings button text changes
expect(find('clusterSettingsDetails.deleteClusterSettingsButton').text()).toEqual(
'Retry removing deprecated settings'
);
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ const createActions = (testBed: TestBed) => {
component.update();
},
clickDeprecationRowAt: async (
deprecationType: 'mlSnapshot' | 'indexSetting' | 'reindex' | 'default',
deprecationType: 'mlSnapshot' | 'indexSetting' | 'reindex' | 'default' | 'clusterSetting',
index: number
) => {
await act(async () => {
Expand Down Expand Up @@ -125,6 +125,16 @@ const createActions = (testBed: TestBed) => {
},
};

const clusterSettingsDeprecationFlyout = {
clickDeleteSettingsButton: async () => {
await act(async () => {
find('deleteClusterSettingsButton').simulate('click');
});

component.update();
},
};

const reindexDeprecationFlyout = {
clickReindexButton: async () => {
await act(async () => {
Expand All @@ -142,6 +152,7 @@ const createActions = (testBed: TestBed) => {
mlDeprecationFlyout,
reindexDeprecationFlyout,
indexSettingsDeprecationFlyout,
clusterSettingsDeprecationFlyout,
};
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
*/

import type { ESUpgradeStatus, EnrichedDeprecationInfo } from '../../../common/types';
import { indexSettingDeprecations } from '../../../common/constants';

export const MOCK_SNAPSHOT_ID = '1';
export const MOCK_JOB_ID = 'deprecation_check_job';
Expand Down Expand Up @@ -43,13 +42,26 @@ const MOCK_INDEX_SETTING_DEPRECATION: EnrichedDeprecationInfo = {
isCritical: false,
resolveDuringUpgrade: false,
type: 'index_settings',
message: indexSettingDeprecations.translog.deprecationMessage,
message: 'Setting [index.routing.allocation.include._tier] is deprecated',
details: 'deprecation details',
url: 'doc_url',
index: 'my_index',
correctiveAction: {
type: 'indexSetting',
deprecatedSettings: indexSettingDeprecations.translog.settings,
deprecatedSettings: ['index.routing.allocation.include._tier'],
},
};

const MOCK_CLUSTER_SETTING_DEPRECATION: EnrichedDeprecationInfo = {
isCritical: false,
resolveDuringUpgrade: false,
type: 'cluster_settings',
message: 'Setting [cluster.routing.allocation.require._tier] is deprecated',
details: 'deprecation details',
url: 'doc_url',
correctiveAction: {
type: 'clusterSetting',
deprecatedSettings: ['cluster.routing.allocation.require._tier'],
},
};

Expand All @@ -70,6 +82,7 @@ export const esDeprecationsMockResponse: ESUpgradeStatus = {
MOCK_INDEX_SETTING_DEPRECATION,
MOCK_DEFAULT_DEPRECATION,
MOCK_REINDEX_DEPRECATION,
MOCK_CLUSTER_SETTING_DEPRECATION,
],
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,16 @@ const registerHttpRequestMockHelpers = (server: SinonFakeServer) => {
]);
};

const setClusterSettingsResponse = (response?: object, error?: ResponseError) => {
const status = error ? error.statusCode || 400 : 200;
const body = error ? error : response;
server.respondWith('POST', `${API_BASE_PATH}/cluster_settings`, [
status,
{ 'Content-Type': 'application/json' },
JSON.stringify(body),
]);
};

return {
setLoadCloudBackupStatusResponse,
setLoadEsDeprecationsResponse,
Expand All @@ -232,6 +242,7 @@ const registerHttpRequestMockHelpers = (server: SinonFakeServer) => {
setLoadMlUpgradeModeResponse,
setGetUpgradeStatusResponse,
setLoadRemoteClustersResponse,
setClusterSettingsResponse,
};
};

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,15 @@ export const esCriticalAndWarningDeprecations: ESUpgradeStatus = {
isCritical: false,
type: 'index_settings',
resolveDuringUpgrade: false,
message: 'Translog retention settings are deprecated',
url: 'https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-translog.html',
message: 'Setting [index.routing.allocation.include._tier] is deprecated',
url: 'https://www.elastic.co/guide/en/elasticsearch/reference/7.16/migrating-7.13.html#deprecate-tier-filter-setting',
details:
'translog retention settings [index.translog.retention.size] and [index.translog.retention.age] are ignored because translog is no longer used in peer recoveries with soft-deletes enabled (default in 7.0 or later)',
'Remove the [index.routing.allocation.include._tier] setting. Use [index.routing.allocation.include._tier_preference] to control allocation to data tiers.',
index: 'settings',
correctiveAction: {
type: 'indexSetting',
deprecatedSettings: ['index.routing.allocation.include._tier'],
},
},
],
};
Expand Down
12 changes: 0 additions & 12 deletions x-pack/plugins/upgrade_assistant/common/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,6 @@
*/
export const MAJOR_VERSION = '7.16.0';

/*
* Map of 7.0 --> 8.0 index setting deprecation log messages and associated settings
* We currently only support one setting deprecation (translog retention), but the code is written
* in a way to be able to support any number of deprecated index settings defined here
*/
export const indexSettingDeprecations = {
translog: {
deprecationMessage: 'Translog retention settings are deprecated', // expected message from ES deprecation info API
settings: ['translog.retention.size', 'translog.retention.age'],
},
};

export const API_BASE_PATH = '/api/upgrade_assistant';

// Telemetry constants
Expand Down
8 changes: 7 additions & 1 deletion x-pack/plugins/upgrade_assistant/common/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,12 +199,18 @@ export interface IndexSettingAction {
type: 'indexSetting';
deprecatedSettings: string[];
}

export interface ClusterSettingAction {
type: 'clusterSetting';
deprecatedSettings: string[];
}

export interface EnrichedDeprecationInfo
extends Omit<estypes.MigrationDeprecationsDeprecation, 'level'> {
type: keyof estypes.MigrationDeprecationsResponse;
isCritical: boolean;
index?: string;
correctiveAction?: ReindexAction | MlAction | IndexSettingAction;
correctiveAction?: ReindexAction | MlAction | IndexSettingAction | ClusterSettingAction;
resolveDuringUpgrade: boolean;
}

Expand Down
Loading

0 comments on commit b988d3b

Please sign in to comment.