Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new advanced settings for vis augmenter #3961

Merged
5 changes: 4 additions & 1 deletion config/opensearch_dashboards.yml
Original file line number Diff line number Diff line change
Expand Up @@ -239,4 +239,7 @@
#data_source.encryption.wrappingKey: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

# Set the value of this setting to false to hide the help menu link to the OpenSearch Dashboards user survey
# opensearchDashboards.survey.url: "https://survey.opensearch.org"
# opensearchDashboards.survey.url: "https://survey.opensearch.org"

# Set the value of this setting to false to disable plugin augmentation on Dashboard
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we're consistent in this file, but I prefer the form of L232-234; because we provide the commented out default value, the instructions should be about setting it to the non-default:

Suggested change
# Set the value of this setting to false to disable plugin augmentation on Dashboard
# Set the value of this setting to true to enable plugin augmentation on Dashboard visualizations

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see the default is intended to be true. In that case, keep the text and change the commented value.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the standard of the other settings and the commented value should be false based on that.

# vis_augmenter.pluginAugmentationEnabled: false
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ export const stackManagementSchema: MakeSchemaFrom<UsageStats> = {
'visualization:regionmap:showWarnings': { type: 'boolean' },
'visualization:dimmingOpacity': { type: 'float' },
'visualization:tileMap:maxPrecision': { type: 'long' },
'visualization:enablePluginAugmentation': { type: 'boolean' },
'visualization:enablePluginAugmentation.maxPluginObjects': { type: 'number' },
'securitySolution:ipReputationLinks': { type: 'text' },
'csv:separator': { type: 'keyword' },
'visualization:tileMap:WMSdefaults': { type: 'text' },
Expand Down
6 changes: 6 additions & 0 deletions src/plugins/telemetry/schema/oss_plugins.json
Original file line number Diff line number Diff line change
Expand Up @@ -1372,6 +1372,12 @@
"visualization:tileMap:maxPrecision": {
"type": "long"
},
"visualization:enablePluginAugmentation": {
"type": "boolean"
},
"visualization:enablePluginAugmentation.maxPluginObjects": {
"type": "number"
},
"securitySolution:ipReputationLinks": {
"type": "text"
},
Expand Down
12 changes: 12 additions & 0 deletions src/plugins/vis_augmenter/config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

import { schema, TypeOf } from '@osd/config-schema';

export const configSchema = schema.object({
pluginAugmentationEnabled: schema.boolean({ defaultValue: true }),
});

export type VisAugmenterPluginConfigType = TypeOf<typeof configSchema>;
3 changes: 2 additions & 1 deletion src/plugins/vis_augmenter/public/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { ExpressionsSetup } from '../../expressions/public';
import { PluginInitializerContext, CoreSetup, CoreStart, Plugin } from '../../../core/public';
import { DataPublicPluginSetup, DataPublicPluginStart } from '../../data/public';
import { visLayers } from './expressions';
import { setSavedAugmentVisLoader } from './services';
import { setSavedAugmentVisLoader, setUISettings } from './services';
import { createSavedAugmentVisLoader, SavedAugmentVisLoader } from './saved_augment_vis';

// eslint-disable-next-line @typescript-eslint/no-empty-interface
Expand Down Expand Up @@ -40,6 +40,7 @@ export class VisAugmenterPlugin
}

public start(core: CoreStart, { data }: VisAugmenterStartDeps): VisAugmenterStart {
setUISettings(core.uiSettings);
const savedAugmentVisLoader = createSavedAugmentVisLoader({
savedObjectsClient: core.savedObjects.client,
indexPatterns: data.indexPatterns,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,59 +4,87 @@
*/

import { get, isEmpty } from 'lodash';
import { IUiSettingsClient } from 'opensearch-dashboards/public';
import {
SavedObjectLoader,
SavedObjectOpenSearchDashboardsServices,
} from '../../../saved_objects/public';
import { createSavedAugmentVisClass } from './_saved_augment_vis';
import { VisLayerTypes } from '../types';
import { getUISettings } from '../services';

// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface SavedObjectOpenSearchDashboardsServicesWithAugmentVis
extends SavedObjectOpenSearchDashboardsServices {}
export type SavedAugmentVisLoader = ReturnType<typeof createSavedAugmentVisLoader>;

export class SavedObjectLoaderAugmentVis extends SavedObjectLoader {
private readonly config: IUiSettingsClient = getUISettings();

mapHitSource = (source: Record<string, any>, id: string) => {
source.id = id;
source.visId = get(source, 'visReference.id', '');

if (isEmpty(source.visReference)) {
source.error = 'visReference is missing in augment-vis saved object';
return source;
}
if (isEmpty(source.visLayerExpressionFn)) {
source.error = 'visLayerExpressionFn is missing in augment-vis saved object';
return source;
}
if (!(get(source, 'visLayerExpressionFn.type', '') in VisLayerTypes)) {
source.error = 'Unknown VisLayer expression function type';
return source;
}
return source;
};

/**
* Updates hit.attributes to contain an id related to the referenced visualization
* (visId) and returns the updated attributes object.
* @param hit
* @returns {hit.attributes} The modified hit.attributes object, with an id and url field.
*/
mapSavedObjectApiHits(hit: {
references: any[];
attributes: Record<string, unknown>;
id: string;
}) {
// For now we are assuming only one vis reference per saved object.
// If we change to multiple, we will need to dynamically handle that
const visReference = hit.references[0];
return this.mapHitSource({ ...hit.attributes, visReference }, hit.id);
}

/**
* Retrieve a saved object by id or create new one.
* Returns a promise that completes when the object finishes
* initializing. Throws exception when the setting is set to false.
* @param opts
* @returns {Promise<SavedObject>}
*/
get(opts?: Record<string, unknown> | string) {
const isAugmentationEnabled = this.config.get('visualization:enablePluginAugmentation');

if (!isAugmentationEnabled) {
// eslint-disable-next-line no-throw-literal
throw 'Visualization augmentation is disabled, please enable visualization:enablePluginAugmentation.';
lezzago marked this conversation as resolved.
Show resolved Hide resolved
}

// can accept object as argument in accordance to SavedVis class
// see src/plugins/saved_objects/public/saved_object/saved_object_loader.ts
// @ts-ignore
const obj = new this.Class(opts);
return obj.init();
}
}

export function createSavedAugmentVisLoader(
services: SavedObjectOpenSearchDashboardsServicesWithAugmentVis
) {
const { savedObjectsClient } = services;

class SavedObjectLoaderAugmentVis extends SavedObjectLoader {
mapHitSource = (source: Record<string, any>, id: string) => {
source.id = id;
source.visId = get(source, 'visReference.id', '');

if (isEmpty(source.visReference)) {
source.error = 'visReference is missing in augment-vis saved object';
return source;
}
if (isEmpty(source.visLayerExpressionFn)) {
source.error = 'visLayerExpressionFn is missing in augment-vis saved object';
return source;
}
if (!(get(source, 'visLayerExpressionFn.type', '') in VisLayerTypes)) {
source.error = 'Unknown VisLayer expression function type';
return source;
}
return source;
};

/**
* Updates hit.attributes to contain an id related to the referenced visualization
* (visId) and returns the updated attributes object.
* @param hit
* @returns {hit.attributes} The modified hit.attributes object, with an id and url field.
*/
mapSavedObjectApiHits(hit: {
references: any[];
attributes: Record<string, unknown>;
id: string;
}) {
// For now we are assuming only one vis reference per saved object.
// If we change to multiple, we will need to dynamically handle that
const visReference = hit.references[0];
return this.mapHitSource({ ...hit.attributes, visReference }, hit.id);
}
}
const SavedAugmentVis = createSavedAugmentVisClass(services);
return new SavedObjectLoaderAugmentVis(SavedAugmentVis, savedObjectsClient) as SavedObjectLoader;
return new SavedObjectLoaderAugmentVis(SavedAugmentVis, savedObjectsClient);
}
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,46 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { getSavedAugmentVisLoader } from '../../services';
import { ISavedAugmentVis } from '../../types';
import { get } from 'lodash';
import { getSavedAugmentVisLoader, getUISettings } from '../../services';
import { ISavedAugmentVis } from '../types';

/**
* Create an augment vis saved object given an object that
* implements the ISavedAugmentVis interface
*/
export const createAugmentVisSavedObject = async (AugmentVis: ISavedAugmentVis): Promise<any> => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add unit tests here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will follow up in another pr

const loader = getSavedAugmentVisLoader();
const config = getUISettings();

const isAugmentationEnabled = config.get('visualization:enablePluginAugmentation');
if (!isAugmentationEnabled) {
// eslint-disable-next-line no-throw-literal
throw 'Visualization augmentation is disabled, please enable visualization:enablePluginAugmentation.';
}
const maxAssociatedCount = config.get('visualization:enablePluginAugmentation.maxPluginObjects');

await loader.findAll().then(async (resp) => {
if (resp !== undefined) {
const savedAugmentObjects = get(resp, 'hits', []);
// gets all the saved object for this visualization
const savedObjectsForThisVisualization = savedAugmentObjects.filter(
(savedObj) => get(savedObj, 'visId', '') === AugmentVis.visId
);

if (maxAssociatedCount <= savedObjectsForThisVisualization.length) {
// eslint-disable-next-line no-throw-literal
throw (
'Cannot associate the plugin resource to the visualization due to the limit of the' +
'max amount of associated plugin resources (' +
maxAssociatedCount +
') with ' +
savedObjectsForThisVisualization.length +
' associated to the visualization'
);
}
}
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the more I'm thinking about this, I think we will need to make this check on the server-side at the low-level api call made by the saved object client. Otherwise we still aren't covering all scenarios, and the limit could be circumvented by making alternate calls to create & save the object.

This function createAugmentVisSavedObject() is essentially just a helper fn / wrapper that is using the saved object loader under the hood. The loader itself is an abstraction on top of the saved object client.

To my knowledge there hasn't been set limitations on creating a saved object from the client, other than the fact that type and attributes can't be empty - see https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/core/public/saved_objects/saved_objects_client.ts#L244

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @ashwin-pc @joshuarrrr for thoughts on this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe having this at the loader-level is still appropriate, and if users circumvent by creating these saved objs in unexpected ways (e.g., direct api calls), that is ok. Typically saved objs are quite fragile and are highly recommended not to be created manually, and altering them later on could lead to weird/unexpected behavior. I'd still like to get OSD folks opinion on this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea this is a real issue as its bypassing the setting. More enforcement would be hard too since they can bypass the normal saved object api and write to the index directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the reason to move this to the server side is to prevent the user from circumventing the limit, I'd love to know more about how risky that is. But in general it's better to put a warning near the setting than to move it to the server. Moving it to the server is more useful when we want to reduce latency or access secrets. For real safeguards, that has to be controlled on OS itself. As for saved object safeguards, yeah we don't have many today and that's a sorely needed feature.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the same time if you think it's necessary, feel free to add your own safeguards for your saved objects on the server. It's a judgement call

Copy link
Member

@ohltyler ohltyler May 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ashwin-pc thanks for your input! Sounds good. This is helpful in understanding the current state of saved object safeguarding/limits. Agreed it would need to have logic in OpenSearch itself to truly safeguard. If the limit is breached unexpectedly, this does not have extreme impacts. It would just mean errors when trying to link more plugin resources to the particular vis that is over the limit, and potential confusion if users deep dive and see there are >limit resources associated to a particular vis. AND if possible, we could add clear messaging when re-rendering a vis that is over such limit (e.g., unlinking plugin resources to free up space).

In my opinion, based on what's currently available, the overall low likelihood of this manual behavior, and the overall low criticality of this limit being breached unexpectedly, I think I'm ok with keeping this at the saved-object-loader level of checks. Moving to server-side and/or having actual blocking checks on the OpenSearch side I think is a bigger discussion regarding efforts to lock down saved objects. Lmk your thoughts @lezzago

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea I think leaving as is would be best. Later the roadmap is that OSD would have its own data store separate from OS, so having this logic in OS would cause more problems in the future. I believe adding some documentation here would be helpful for the users.


return await loader.get((AugmentVis as any) as Record<string, unknown>);
};
9 changes: 6 additions & 3 deletions src/plugins/vis_augmenter/public/services.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { createGetterSetter } from '../../opensearch_dashboards_utils/common';
import { SavedObjectLoader } from '../../saved_objects/public';
import { createGetterSetter } from '../../opensearch_dashboards_utils/public';
import { IUiSettingsClient } from '../../../core/public';
import { SavedObjectLoaderAugmentVis } from './saved_augment_vis';

export const [getSavedAugmentVisLoader, setSavedAugmentVisLoader] = createGetterSetter<
SavedObjectLoader
SavedObjectLoaderAugmentVis
>('savedAugmentVisLoader');

export const [getUISettings, setUISettings] = createGetterSetter<IUiSettingsClient>('UISettings');
15 changes: 14 additions & 1 deletion src/plugins/vis_augmenter/public/utils/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
VisLayer,
isVisLayerWithError,
} from '../';
import { getUISettings } from '../services';

export const isEligibleForVisLayers = (vis: Vis): boolean => {
// Only support date histogram and ensure there is only 1 x-axis and it has to be on the bottom.
Expand All @@ -38,7 +39,10 @@ export const isEligibleForVisLayers = (vis: Vis): boolean => {
const hasOnlyLineSeries =
vis.params.seriesParams.every((seriesParam: { type: string }) => seriesParam.type === 'line') &&
vis.params.type === 'line';
return hasValidXaxis && hasCorrectAggregationCount && hasOnlyLineSeries;
// Checks if the augmentation setting is enabled
const config = getUISettings();
const isAugmentationEnabled = config.get('visualization:enablePluginAugmentation');
lezzago marked this conversation as resolved.
Show resolved Hide resolved
return isAugmentationEnabled && hasValidXaxis && hasCorrectAggregationCount && hasOnlyLineSeries;
};

/**
Expand All @@ -50,6 +54,15 @@ export const getAugmentVisSavedObjs = async (
visId: string | undefined,
loader: SavedAugmentVisLoader | undefined
): Promise<ISavedAugmentVis[]> => {
if (visId === undefined || loader === undefined) {
return [] as ISavedAugmentVis[];
}
lezzago marked this conversation as resolved.
Show resolved Hide resolved
const config = getUISettings();
const isAugmentationEnabled = config.get('visualization:enablePluginAugmentation');
if (!isAugmentationEnabled) {
// eslint-disable-next-line no-throw-literal
throw 'Visualization augmentation is disabled, please enable visualization:enablePluginAugmentation.';
}
lezzago marked this conversation as resolved.
Show resolved Hide resolved
try {
const resp = await loader?.findAll();
const allSavedObjects = (get(resp, 'hits', []) as any[]) as ISavedAugmentVis[];
Expand Down
9 changes: 8 additions & 1 deletion src/plugins/vis_augmenter/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,16 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { PluginInitializerContext } from '../../../core/server';
import { PluginConfigDescriptor, PluginInitializerContext } from '../../../core/server';
import { VisAugmenterPlugin } from './plugin';
import { configSchema, VisAugmenterPluginConfigType } from '../config';

export const config: PluginConfigDescriptor<VisAugmenterPluginConfigType> = {
exposeToBrowser: {
pluginAugmentationEnabled: true,
},
schema: configSchema,
};
export function plugin(initializerContext: PluginInitializerContext) {
return new VisAugmenterPlugin(initializerContext);
}
Expand Down
45 changes: 44 additions & 1 deletion src/plugins/vis_augmenter/server/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { i18n } from '@osd/i18n';
import { schema } from '@osd/config-schema';
import { Observable } from 'rxjs';
import { first } from 'rxjs/operators';
import {
PluginInitializerContext,
CoreSetup,
Expand All @@ -12,6 +16,7 @@ import {
} from '../../../core/server';
import { augmentVisSavedObjectType } from './saved_objects';
import { capabilitiesProvider } from './capabilities_provider';
import { VisAugmenterPluginConfigType } from '../config';

// eslint-disable-next-line @typescript-eslint/no-empty-interface
export interface VisAugmenterPluginSetup {}
Expand All @@ -21,15 +26,53 @@ export interface VisAugmenterPluginStart {}
export class VisAugmenterPlugin
implements Plugin<VisAugmenterPluginSetup, VisAugmenterPluginStart> {
private readonly logger: Logger;
private readonly config$: Observable<VisAugmenterPluginConfigType>;

constructor(initializerContext: PluginInitializerContext) {
this.logger = initializerContext.logger.get();
this.config$ = initializerContext.config.create<VisAugmenterPluginConfigType>();
}

public setup(core: CoreSetup) {
public async setup(core: CoreSetup) {
this.logger.debug('VisAugmenter: Setup');
core.savedObjects.registerType(augmentVisSavedObjectType);
core.capabilities.registerProvider(capabilitiesProvider);

const config: VisAugmenterPluginConfigType = await this.config$.pipe(first()).toPromise();
const isAugmentationEnabled =
config.pluginAugmentationEnabled === undefined ? true : config.pluginAugmentationEnabled;

if (isAugmentationEnabled) {
lezzago marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually pretty neat that you are relying on the missing UI settings to disable it in the plugin. Calling it out in the comment here will be really helpful in the future though since it may not be obvious that you are doing it this way and its anon standard pattern in the repo.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will follow up in another pr with the tests

core.uiSettings.register({
['visualization:enablePluginAugmentation']: {
name: i18n.translate('visualization.enablePluginAugmentationTitle', {
defaultMessage: 'Enable plugin augmentation',
}),
value: true,
description: i18n.translate('visualization.enablePluginAugmentationText', {
defaultMessage: 'Plugin functionality can be accessed from line chart visualizations',
}),
category: ['visualization'],
schema: schema.boolean(),
},
['visualization:enablePluginAugmentation.maxPluginObjects']: {
name: i18n.translate('visualization.enablePluginAugmentation.maxPluginObjectsTitle', {
defaultMessage: 'Max number of associated augmentations',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message seems a little weird, should it be something like 'max number of associated plugin objects / plugin resources'? Not sure if this was finalized by tech writer

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was based on mockups and we can improve this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure - no blocker here. We can probably follow up with one larger PR to update wording after UX has a full walk through.

}),
value: 10,
description: i18n.translate(
'visualization.enablePluginAugmentation.maxPluginObjectsText',
{
defaultMessage:
'Associating more than 10 plugin resources per visualization can lead to performance ' +
'issues and increase the cost of running clusters.',
}
),
category: ['visualization'],
schema: schema.number({ min: 0 }),
},
});
}
return {};
}

Expand Down
Loading