Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MD]Use placeholder for data source credentials fields when export saved object #6928

Merged
merged 2 commits into from
Jun 5, 2024
Merged

[MD]Use placeholder for data source credentials fields when export saved object #6928

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2fd5603
[MD]Use placeholder for data source credentials fields when exportin…
zhongnansu Jun 5, 2024
fe8a800
Changeset file for PR #6928 created/updated
opensearch-changeset-bot[bot] Jun 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions changelogs/fragments/6928.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
feat:
- [MD]Use placeholder for data source credentials fields when export saved object ([#6928](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/6928))
49 changes: 48 additions & 1 deletion src/core/server/saved_objects/export/get_sorted_objects_for_export.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,10 @@
* under the License.
*/

import { exportSavedObjectsToStream } from './get_sorted_objects_for_export';
import {
DATA_SOURCE_CREDENTIALS_PLACEHOLDER,
exportSavedObjectsToStream,
} from './get_sorted_objects_for_export';
import { savedObjectsClientMock } from '../service/saved_objects_client.mock';
import { Readable } from 'stream';
import { createPromiseFromStreams, createConcatStream } from '../../utils/streams';
Expand Down Expand Up @@ -706,6 +709,50 @@ describe('getSortedObjectsForExport()', () => {
]);
});

test('modifies return results to update `credentials` of data-source to use placeholder', async () => {
const createDataSourceSavedObject = (id: string, auth: any) => ({
id,
type: 'data-source',
attributes: { auth },
references: [],
});

const dataSourceNoAuthInfo = { type: 'no_auth' };
const dataSourceBasicAuthInfo = {
type: 'username_password',
credentials: { username: 'foo', password: 'bar' },
};

const redactedDataSourceBasicAuthInfo = {
type: 'username_password',
credentials: {
username: DATA_SOURCE_CREDENTIALS_PLACEHOLDER,
password: DATA_SOURCE_CREDENTIALS_PLACEHOLDER,
},
};

savedObjectsClient.bulkGet.mockResolvedValueOnce({
saved_objects: [
createDataSourceSavedObject('1', dataSourceNoAuthInfo),
createDataSourceSavedObject('2', dataSourceBasicAuthInfo),
],
});
const exportStream = await exportSavedObjectsToStream({
exportSizeLimit: 10000,
savedObjectsClient,
objects: [
{ type: 'data-source', id: '1' },
{ type: 'data-source', id: '2' },
],
});
const response = await readStreamToCompletion(exportStream);
expect(response).toEqual([
createDataSourceSavedObject('1', dataSourceNoAuthInfo),
createDataSourceSavedObject('2', redactedDataSourceBasicAuthInfo),
expect.objectContaining({ exportedCount: 2 }),
]);
});

test('includes nested dependencies when passed in', async () => {
savedObjectsClient.bulkGet.mockResolvedValueOnce({
saved_objects: [
Expand Down
34 changes: 33 additions & 1 deletion src/core/server/saved_objects/export/get_sorted_objects_for_export.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ import { SavedObjectsClientContract, SavedObject, SavedObjectsBaseOptions } from
import { fetchNestedDependencies } from './inject_nested_depdendencies';
import { sortObjects } from './sort_objects';

export const DATA_SOURCE_CREDENTIALS_PLACEHOLDER = 'pleaseUpdateCredentials';

/**
* Options controlling the export operation.
* @public
Expand Down Expand Up @@ -185,10 +187,40 @@ export async function exportSavedObjectsToStream({
({ namespaces, ...object }) => object
);

// update the credential fields from "data-source" saved object to use placeholder to avoid exporting sensitive information
const redactedObjectsWithoutCredentials = redactedObjects.map<SavedObject<unknown>>((object) => {
if (object.type === 'data-source') {
const { auth, ...rest } = object.attributes as {
auth: { type: string; credentials?: any };
};
const hasCredentials = auth && auth.credentials;
const updatedCredentials = hasCredentials
? Object.keys(auth.credentials).reduce((acc, key) => {
acc[key] = DATA_SOURCE_CREDENTIALS_PLACEHOLDER;
Copy link
Member

@xinruiba xinruiba Jun 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this change~
Just one question, SigV4 and TokenExchange also have regions under credential attribute, will this logic have any side effect?

Maybe we can also test SigV4 scenario while waiting CI to complete? Thanks!

Copy link
Member Author

@zhongnansu zhongnansu Jun 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

every field under "credentials" attributes will be replaced to use placeholder value, including region. I have tested sigV4 case, you can find it from the video. The exported sigV4 data source will look like this

{
   "attributes": {
      "auth": {
         "credentials": {
            "accessKey": "pleaseUpdateCredentials",
            "region": "pleaseUpdateCredentials",
            "secretKey": "pleaseUpdateCredentials",
            "service": "pleaseUpdateCredentials"
         },
         "type": "sigv4"
      },
      "dataSourceVersion": "",
      "description": "",
      "endpoint": "https://mjdty4727oifwax8ycdk.us-west-2.aoss.amazonaws.com",
      "installedPlugins": [],
      "title": "testSigV4"
   },
   "id": "7f8e13c0-2378-11ef-a345-bd55de69299f",
   "migrationVersion": {
      "data-source": "2.4.0"
   },
   "references": [],
   "type": "data-source",
   "updated_at": "2024-06-05T20:16:31.484Z",
   "version": "WzQsMV0="
}

return acc;
}, {} as { [key: string]: any })
: undefined;
return {
...object,
attributes: {
...rest,
auth: {
type: auth.type,
...(hasCredentials && { credentials: updatedCredentials }),
},
},
};
}
return object;
});

const exportDetails: SavedObjectsExportResultDetails = {
exportedCount: exportedObjects.length,
missingRefCount: missingReferences.length,
missingReferences,
};
return createListStream([...redactedObjects, ...(excludeExportDetails ? [] : [exportDetails])]);
return createListStream([
...redactedObjectsWithoutCredentials,
...(excludeExportDetails ? [] : [exportDetails]),
]);
}
Loading