Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NP] Inline buildPointSeriesData and buildHierarchicalData dependencies #61575

Merged
merged 38 commits into from
Apr 9, 2020
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
7fafc54
Move buildHierarchicalData to vislib
maryia-lapata Mar 27, 2020
4cd75d1
Move shortened version of buildPointSeriesData to Discover
maryia-lapata Mar 27, 2020
2bc346b
Move buildPointSeriesData to vis_type_vislib
maryia-lapata Mar 27, 2020
ba83566
Convert unit tests to jest
maryia-lapata Mar 30, 2020
8283cad
Remove ui/agg_response
maryia-lapata Mar 30, 2020
45b33f8
Merge remote-tracking branch 'remotes/kibana/master' into agg-respons…
maryia-lapata Mar 30, 2020
6d3b22e
Convert point_series files to TS
maryia-lapata Mar 31, 2020
2020a63
Update TS in unit tests
maryia-lapata Apr 1, 2020
9c2cc97
Merge branch 'master' into agg-response-cleanup
elasticmachine Apr 1, 2020
d05da68
Convert buildHierarchicalData to TS
maryia-lapata Apr 2, 2020
8e39fbd
Convert buildPointSeriesData to TS in Discover
maryia-lapata Apr 2, 2020
abcc032
Clean TS in Discover
maryia-lapata Apr 3, 2020
f7609a4
Update TS for buildHierarchicalData
maryia-lapata Apr 3, 2020
4eb83d9
Update buildHierarchicalData unit tests
maryia-lapata Apr 3, 2020
994b93e
Clean up TS in point_series
maryia-lapata Apr 3, 2020
207d32c
Merge branch 'master' into agg-response-cleanup
elasticmachine Apr 3, 2020
3de79f7
Add unit tests fro response_handler.js
maryia-lapata Apr 4, 2020
76f0266
Simplify point_series for Discover
maryia-lapata Apr 4, 2020
5bd3a5d
Return array for data
maryia-lapata Apr 6, 2020
f610ffd
Merge branch 'master' into agg-response-cleanup
elasticmachine Apr 6, 2020
15af087
Add check for empty row
maryia-lapata Apr 6, 2020
da332e0
Merge branch 'agg-response-cleanup' of https://github.com/maryia-lapa…
maryia-lapata Apr 6, 2020
f6a4ed5
Simplify point_series for Discover
maryia-lapata Apr 6, 2020
84fdace
Merge branch 'master' into agg-response-cleanup
elasticmachine Apr 6, 2020
bbc4624
Return all points
maryia-lapata Apr 6, 2020
e7c88f6
Merge remote-tracking branch 'kibana/master' into agg-response-cleanup
maryia-lapata Apr 6, 2020
ad8ab46
Merge remote-tracking branch 'kibana/master' into agg-response-cleanup
maryia-lapata Apr 8, 2020
10eca3f
Specify TS
maryia-lapata Apr 8, 2020
a72aa56
Merge branch 'agg-response-cleanup' of https://github.com/maryia-lapa…
maryia-lapata Apr 8, 2020
1168932
Refactoring
maryia-lapata Apr 8, 2020
acef340
Merge remote-tracking branch 'upstream/master' into agg-response-cleanup
flash1293 Apr 9, 2020
b0d9e60
Merge remote-tracking branch 'kibana/master' into agg-response-cleanup
maryia-lapata Apr 9, 2020
16028af
Merge remote-tracking branch 'kibana/master' into agg-response-cleanup
maryia-lapata Apr 9, 2020
5617d0c
Simplifying
maryia-lapata Apr 9, 2020
c0d6422
Merge branch 'agg-response-cleanup' of github.com:maryia-lapata/kiban…
flash1293 Apr 9, 2020
5f7b63e
improve types
flash1293 Apr 9, 2020
d7e5043
Merge pull request #12 from flash1293/agg-response-cleanup-types
maryia-lapata Apr 9, 2020
838f5cf
Update _get_point.test.ts
maryia-lapata Apr 9, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -78,5 +78,3 @@ export {
EsQuerySortValue,
SortDirection,
} from '../../../../../plugins/data/public';
// @ts-ignore
export { buildPointSeriesData } from 'ui/agg_response/point_series/point_series';
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ import { IUiSettingsClient } from 'kibana/public';
import { EuiChartThemeType } from '@elastic/eui/dist/eui_charts_theme';
import { Subscription } from 'rxjs';
import { getServices } from '../../../kibana_services';
import { Chart as IChart } from '../helpers/point_series';

export interface DiscoverHistogramProps {
chartData: any;
chartData: IChart;
timefilterUpdateHandler: (ranges: { from: number; to: number }) => void;
}

Expand Down Expand Up @@ -163,7 +164,7 @@ export class DiscoverHistogram extends Component<DiscoverHistogramProps, Discove
};

public formatXValue = (val: string) => {
const xAxisFormat = this.props.chartData.xAxisFormat.params.pattern;
const xAxisFormat = this.props.chartData.xAxisFormat.params!.pattern;

return moment(val).format(xAxisFormat);
};
Expand Down Expand Up @@ -219,14 +220,15 @@ export class DiscoverHistogram extends Component<DiscoverHistogramProps, Discove
* see https://github.com/elastic/kibana/issues/27410
* TODO: Once the Discover query has been update, we should change the below to use the new field
*/
const { intervalESValue, intervalESUnit, interval: xInterval } = chartData.ordered;
const { intervalESValue, intervalESUnit, interval } = chartData.ordered;
const xInterval = interval as number;

const xValues = chartData.xAxisOrderedValues;
const lastXValue = xValues[xValues.length - 1];

const domain = chartData.ordered;
const domainStart = domain.min.valueOf();
const domainEnd = domain.max.valueOf();
const domainStart = domain.min!.valueOf();
const domainEnd = domain.max!.valueOf();

const domainMin = data[0].x > domainStart ? domainStart : data[0].x;
const domainMax = domainEnd - xInterval > lastXValue ? domainEnd - xInterval : lastXValue;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,23 @@
* under the License.
*/

export function addToSiri(series, point, id, yLabel, yFormat, zFormat, zLabel) {
import { Point } from './_get_point';

export interface Serie {
id: string;
values: Point[];
}

export function addToSiri(series: Map<string, Serie>, point: Point, id: string) {
id = id == null ? '' : id + '';

if (series.has(id)) {
series.get(id).values.push(point);
(series.get(id) as Serie).values.push(point);
return;
}

series.set(id, {
id: id.split('-').pop(),
rawId: id,
label: yLabel == null ? id : yLabel,
count: 0,
id: id.split('-').pop() as string,
values: [point],
Copy link
Contributor Author

@maryia-lapata maryia-lapata Apr 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the component the values is used only, so I removed all other fields.

format: yFormat,
zLabel,
zFormat,
});
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import { Table, Dimensions, Aspects } from './point_series';

/**
* Identify and group the columns based on the aspect of the pointSeries
* they represent.
*
* @return {object} - an object with a key for each aspect (see map). The values
* may be undefined or an array of aspects.
*/
export function getAspects(table: Table, dimensions: Dimensions) {
const aspects: Aspects = {} as Aspects;
(Object.keys(dimensions) as Array<keyof Dimensions>).forEach(name => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we know Discover's dimensions object exactly, I simplified all functions for Discover to get rid of unnecessary code.

const d = dimensions[name];
const column = table.columns[d.accessor];
if (!column) {
return;
}
aspects[name] = [
{
accessor: column.id,
column: d.accessor,
title: column.name,
format: d.format,
params: d.params,
},
];
});

return aspects;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import { Aspect, Table, Row } from './point_series';

type RowValue = number | 'NaN';
interface Raw {
table: Table;
column: number;
row: number;
value: RowValue;
}
export interface Point {
x: RowValue;
y: RowValue;
extraMetrics: [];
maryia-lapata marked this conversation as resolved.
Show resolved Hide resolved
xRaw: Raw;
yRaw: Raw;
tableRaw?: {
table: Table;
column: number;
row: number;
value: number;
title: string;
};
parent: null;
series?: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for the series concept - as there is always just a single series, we can just pass it through directly as values or something and getting rid of the nested structure. All of the things put together here are just used in a single place - https://github.com/elastic/kibana/blob/master/src/legacy/core_plugins/kibana/public/discover/np_ready/angular/directives/histogram.tsx#L208

If it's not used there, we can just throw it out.

seriesId?: string;
}

export function getPoint(table: Table, x: Aspect, row: Row, rowIndex: number, y: Aspect) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the Aspect concept is overkill for the discover case - we can probably just hardcode the x and y accessors in this function (and get rid of the raw stuff)

const xRow = row[x.accessor];
const yRow = row[y.accessor];

const point = {
x: xRow,
y: yRow,
extraMetrics: [],
xRaw: {
table,
column: x.column,
row: rowIndex,
value: xRow,
},
yRaw: {
table,
column: y.column,
row: rowIndex,
value: yRow,
},
tableRaw: table.$parent && {
table: table.$parent.table,
column: table.$parent.column,
row: table.$parent.row,
value: table.$parent.key,
title: table.$parent.name,
},
parent: null,
} as Point;

if (point.y === 'NaN') {
// filter out NaN from stats
// from metrics that are not based at zero
return;
}

if (y) {
// If the data is not split up with a series aspect, then
// each point's "series" becomes the y-agg that produced it
point.series = y.title;
}

return point;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import _ from 'lodash';
import { getPoint, Point } from './_get_point';
import { addToSiri, Serie } from './_add_to_siri';
import { Chart, Table } from './point_series';

export function getSeries(table: Table, chart: Chart) {
const aspects = chart.aspects;
const xAspect = aspects.x[0];
const yAspect = aspects.y[0];

const partGetPoint = _.partial(getPoint, table, xAspect);

return _(table.rows)
.transform((series: any, row, rowIndex) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about it even further - I think we can completely remove the logic around "series" and "points". In the special case of discover we are just copying over x and y values, so we can just use the table as it is, no need for all of these abstraction layers.

const point: Point = partGetPoint(row, rowIndex, yAspect);
if (point) {
const id = `${point.series}-${yAspect.accessor}`;
point.seriesId = id;
addToSiri(series as Map<string, Serie>, point, id);
}
}, new Map<string, Serie>() as any)
.thru(series => [...series.values()])
.value() as Serie[];
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import { uniq } from 'lodash';
import moment, { Duration } from 'moment';
import { Unit } from '@elastic/datemath';

import { SerializedFieldFormat } from '../../../../../../../../plugins/expressions/common/types';
import { getSeries } from './_get_series';
import { getAspects } from './_get_aspects';
import { Serie } from './_add_to_siri';

export interface Column {
id: string;
name: string;
}

export interface Row {
[key: string]: number;
}

export interface Table {
columns: Column[];
rows: Row[];
$parent: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we never have the $parent case in discover.

table: Table;
column: number;
row: number;
key: number;
name: string;
};
}

interface HistogramParams {
date: true;
interval: Duration;
intervalESValue: number;
intervalESUnit: Unit;
format: string;
bounds?: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't bounds always set in discover when there is a histogram?
Also, it seems like the actual type of this is a moment instance:

const bounds = agg.params.timeRange ? timefilter.calculateBounds(agg.params.timeRange) : null;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems so. Please take a look at 10eca3f.

min: string | number;
max: string | number;
};
}
export interface Dimension {
accessor: 0 | 1;
format: SerializedFieldFormat<{ pattern: string }>;
params?: HistogramParams;
}

export interface Dimensions {
x: Dimension;
y: Dimension;
}
export interface Aspect {
accessor: Column['id'];
column?: Dimension['accessor'];
title: Column['name'];
format: Dimension['format'];
params: Dimension['params'];
}
export interface Aspects {
x: Aspect[];
y: Aspect[];
}
interface Ordered {
date: true;
interval: Duration | number;
intervalESUnit: string;
intervalESValue: number;
min?: number;
max?: number;
}
export interface Chart {
aspects: Aspects;
series: Serie[];
xAxisOrderedValues: number[];
xAxisFormat: Dimension['format'];
xAxisLabel: Column['name'];
yAxisLabel?: Column['name'];
ordered: Ordered;
}

export const buildPointSeriesData = (table: Table, dimensions: Dimensions) => {
const chart: Chart = {
aspects: getAspects(table, dimensions),
} as Chart;

const { format, title, params, accessor } = chart.aspects.x[0];
chart.xAxisOrderedValues = uniq(table.rows.map(r => r[accessor]));
chart.xAxisFormat = format;
chart.xAxisLabel = title;

const { intervalESUnit, intervalESValue, interval, bounds } = params as HistogramParams;
chart.ordered = {
date: true,
interval: moment.duration(interval),
intervalESUnit,
intervalESValue,
};

if (bounds) {
chart.ordered.min = isNaN(bounds.min as number)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the types from the comments above are fixed, then these type casts should go away.

? Date.parse(bounds.min as string)
: (bounds.min as number);
chart.ordered.max = isNaN(bounds.max as number)
? Date.parse(bounds.max as string)
: (bounds.max as number);
}

chart.yAxisLabel = chart.aspects.y && chart.aspects.y[0].title;

chart.series = getSeries(table, chart);

delete chart.aspects;
return chart;
};
Loading