Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(editor): Add new /templates/search endpoint #8227

Merged
merged 27 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
cded13e
feat(editor): Update search UI to work with the new back-end
MiloradFilipovic Jan 2, 2024
d656a68
Merge branch 'master' into ADO-1555-use-new-search-enpoint
MiloradFilipovic Jan 2, 2024
e3bc01a
⚡ Showing typesense filters in UI
MiloradFilipovic Jan 2, 2024
b5ecfad
⚡ Make filtering work with new back-end
MiloradFilipovic Jan 2, 2024
ab1f3ff
Lint fix
MiloradFilipovic Jan 2, 2024
c4f052b
fix all categories count
RicardoE105 Jan 8, 2024
080212c
fix category querystring filter
RicardoE105 Jan 8, 2024
cd11dd8
sync master
RicardoE105 Jan 8, 2024
71b9c93
fix linting issues
RicardoE105 Jan 8, 2024
77849c2
Merge branch 'master' into ADO-1555-use-new-search-enpoint
MiloradFilipovic Jan 10, 2024
2cda63c
⚡ Removing total template count from All Categories filter
MiloradFilipovic Jan 10, 2024
093cca0
👌 Renaming search parameters to match typsense, not showing 0 in work…
MiloradFilipovic Jan 10, 2024
f679cab
⚡ Revert categories filter so it uses static lists
MiloradFilipovic Jan 10, 2024
2a5df55
👕 Removing leftover interface, formatting code, adding defaults for p…
MiloradFilipovic Jan 10, 2024
1d6b3d6
👌 Fixing collection filtering and URL search restore
MiloradFilipovic Jan 10, 2024
3a279e9
⚡ Promoting selected categories on top, minor refactoring
MiloradFilipovic Jan 11, 2024
dde2a6f
🔥 Removing leftover comments
MiloradFilipovic Jan 11, 2024
33102a8
Merge branch 'master' into ADO-1555-use-new-search-enpoint
MiloradFilipovic Jan 11, 2024
e1156f1
🔨 Using computed props instead of watchers, fixing search query resto…
MiloradFilipovic Jan 11, 2024
bc21b55
✅ Added templates search e2e tests
MiloradFilipovic Jan 11, 2024
620c060
⚡ Add back category loading state
MiloradFilipovic Jan 11, 2024
bf0010a
🔨 Using mocks in template search tests
MiloradFilipovic Jan 11, 2024
bc54f21
⚡ Fixing parentheses in template count labels
MiloradFilipovic Jan 11, 2024
f66cc45
🐛 Only sort categories one time, when they are populated
MiloradFilipovic Jan 11, 2024
e25f983
✔️ Updating tests after components have been updated
MiloradFilipovic Jan 11, 2024
4377ccd
⚡ Updating template search page size to20
MiloradFilipovic Jan 11, 2024
2b90208
✔️ Updating intercepted urls
MiloradFilipovic Jan 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion packages/editor-ui/src/Interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -830,6 +830,24 @@ export interface ITemplatesWorkflowInfo {
};
}

export type TemplateCategoryFilter = {
name: string;
result_count: number;
};

export type TemplateSearchFacet = {
field_name: string;
sampled: boolean;
stats: {
total_values: number;
};
counts: Array<{
count: number;
highlighted: string;
value: string;
}>;
};

export interface ITemplatesWorkflowResponse extends ITemplatesWorkflow, IWorkflowTemplate {
description: string | null;
image: ITemplatesImage[];
Expand Down Expand Up @@ -1357,14 +1375,15 @@ export interface INodeTypesState {
}

export interface ITemplateState {
categories: { [id: string]: ITemplatesCategory };
categories: TemplateCategoryFilter[];
collections: { [id: string]: ITemplatesCollection };
workflows: { [id: string]: ITemplatesWorkflow | ITemplatesWorkflowFull };
workflowSearches: {
[search: string]: {
workflowIds: string[];
totalWorkflows: number;
loadingMore?: boolean;
categories?: TemplateCategoryFilter[];
};
};
collectionSearches: {
Expand Down
10 changes: 8 additions & 2 deletions packages/editor-ui/src/api/templates.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import type {
ITemplatesCollectionResponse,
ITemplatesWorkflowResponse,
IWorkflowTemplate,
TemplateSearchFacet,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we drop the endpoint to retrieve the categories since we are not using it anymore? @MiloradFilipovic

Copy link
Contributor

@MiloradFilipovic MiloradFilipovic Jan 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need it for backward compatibility

} from '@/Interface';
import type { IDataObject } from 'n8n-workflow';
import { get } from '@/utils/apiUtils';
Expand Down Expand Up @@ -42,10 +43,15 @@ export async function getWorkflows(
apiEndpoint: string,
query: { skip: number; limit: number; categories: number[]; search: string },
headers?: IDataObject,
): Promise<{ totalWorkflows: number; workflows: ITemplatesWorkflow[] }> {
): Promise<{
totalWorkflows: number;
workflows: ITemplatesWorkflow[];
filters: TemplateSearchFacet[];
out_of: number;
MiloradFilipovic marked this conversation as resolved.
Show resolved Hide resolved
}> {
return get(
apiEndpoint,
'/templates/workflows',
'/templates/search',
{
skip: query.skip,
rows: query.limit,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this map to the page parameter typesense expects? https://typesense.org/docs/0.24.0/api/search.html#pagination-parameters

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is mapped on the backend:
per_page --> rows
page --> skip

Expand Down
40 changes: 25 additions & 15 deletions packages/editor-ui/src/components/TemplateFilters.vue
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,25 @@
<ul v-if="!loading" :class="$style.categories">
<li :class="$style.item">
<el-checkbox
:label="$locale.baseText('templates.allCategories')"
:model-value="allSelected"
@update:modelValue="(value) => resetCategories(value)"
/>
>
{{ $locale.baseText('templates.allCategories') }}
</el-checkbox>
</li>
<li
v-for="category in collapsed ? sortedCategories.slice(0, expandLimit) : sortedCategories"
:key="category.id"
v-for="(category, index) in collapsed
? sortedCategories.slice(0, expandLimit)
: sortedCategories"
:key="index"
:class="$style.item"
>
<el-checkbox
:label="category.name"
:model-value="isSelected(category.id)"
:model-value="isSelected(category.name)"
@update:modelValue="(value) => handleCheckboxChanged(value, category)"
/>
>
{{ category.name }} <n8n-tag :text="String(category.result_count)" />
</el-checkbox>
</li>
</ul>
<div
Expand All @@ -38,7 +42,10 @@

<script lang="ts">
import { defineComponent } from 'vue';
import type { ITemplatesCategory } from '@/Interface';
import type { TemplateCategoryFilter } from '@/Interface';
import type { PropType } from 'vue';
import { useTemplatesStore } from '@/stores/templates.store';
import { mapStores } from 'pinia';

export default defineComponent({
name: 'TemplateFilters',
Expand All @@ -48,7 +55,8 @@ export default defineComponent({
default: false,
},
categories: {
type: Array,
type: Array as PropType<TemplateCategoryFilter[]>,
default: () => [],
},
expandLimit: {
type: Number,
Expand All @@ -59,28 +67,30 @@ export default defineComponent({
},
selected: {
type: Array,
default: () => [],
},
},
data() {
return {
collapsed: true,
sortedCategories: [] as ITemplatesCategory[],
sortedCategories: [] as TemplateCategoryFilter[],
};
},
computed: {
...mapStores(useTemplatesStore),
allSelected(): boolean {
return this.selected.length === 0;
},
},
watch: {
categories: {
handler(categories: ITemplatesCategory[]) {
handler(categories: TemplateCategoryFilter[]) {
if (!this.sortOnPopulate) {
this.sortedCategories = categories;
} else {
const selected = this.selected || [];
const selectedCategories = categories.filter(({ id }) => selected.includes(id));
const notSelectedCategories = categories.filter(({ id }) => !selected.includes(id));
const selectedCategories = categories.filter(({ name }) => selected.includes(name));
const notSelectedCategories = categories.filter(({ name }) => !selected.includes(name));
this.sortedCategories = selectedCategories.concat(notSelectedCategories);
}
},
Expand All @@ -91,8 +101,8 @@ export default defineComponent({
collapseAction() {
this.collapsed = false;
},
handleCheckboxChanged(value: boolean, selectedCategory: ITemplatesCategory) {
this.$emit(value ? 'select' : 'clear', selectedCategory.id);
handleCheckboxChanged(value: boolean, selectedCategory: TemplateCategoryFilter) {
this.$emit(value ? 'select' : 'clear', selectedCategory.name);
},
isSelected(categoryId: string) {
return this.selected.includes(categoryId);
Expand Down
6 changes: 5 additions & 1 deletion packages/editor-ui/src/components/TemplateList.vue
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<div v-if="loading || workflows.length" :class="$style.list">
<div v-if="!simpleView" :class="$style.header">
<n8n-heading :bold="true" size="medium" color="text-light">
{{ $locale.baseText('templates.workflows') }}
{{ $locale.baseText('templates.workflows') }} ({{ totalCount }})
MiloradFilipovic marked this conversation as resolved.
Show resolved Hide resolved
<span v-if="!loading && totalWorkflows" v-text="`(${totalWorkflows})`" />
</n8n-heading>
</div>
Expand Down Expand Up @@ -63,6 +63,10 @@ export default defineComponent({
type: Boolean,
default: false,
},
totalCount: {
type: Number,
default: 0,
},
},
mounted() {
if (this.infiniteScrollEnabled) {
Expand Down
52 changes: 22 additions & 30 deletions packages/editor-ui/src/stores/templates.store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@ import type {
ITemplatesWorkflow,
ITemplatesWorkflowFull,
IWorkflowTemplate,
TemplateCategoryFilter,
TemplateSearchFacet,
} from '@/Interface';
import { useSettingsStore } from './settings.store';
import {
getCategories,
getCollectionById,
getCollections,
getTemplateById,
Expand All @@ -32,7 +33,7 @@ export type TemplatesStore = ReturnType<typeof useTemplatesStore>;

export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
state: (): ITemplateState => ({
categories: {},
categories: [],
collections: {},
workflows: {},
collectionSearches: {},
Expand All @@ -41,10 +42,8 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
previousSessionId: '',
}),
getters: {
allCategories(): ITemplatesCategory[] {
return Object.values(this.categories).sort((a: ITemplatesCategory, b: ITemplatesCategory) =>
a.name > b.name ? 1 : -1,
);
allCategories(): TemplateCategoryFilter[] {
return this.categories;
},
getTemplateById() {
return (id: string): null | ITemplatesWorkflow => this.workflows[id];
Expand Down Expand Up @@ -111,14 +110,6 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
},
},
actions: {
addCategories(categories: ITemplatesCategory[]): void {
categories.forEach((category: ITemplatesCategory) => {
this.categories = {
...this.categories,
[category.id]: category,
};
});
},
addCollections(collections: Array<ITemplatesCollection | ITemplatesCollectionFull>): void {
collections.forEach((collection) => {
const workflows = (collection.workflows || []).map((workflow) => ({ id: workflow.id }));
Expand Down Expand Up @@ -175,6 +166,7 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
[searchKey]: {
workflowIds: workflowIds as unknown as string[],
totalWorkflows: data.totalWorkflows,
categories: this.categories,
},
};

Expand All @@ -186,6 +178,7 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
[searchKey]: {
workflowIds: [...cachedResults.workflowIds, ...workflowIds] as string[],
totalWorkflows: data.totalWorkflows,
categories: this.categories,
},
};
},
Expand Down Expand Up @@ -254,20 +247,6 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
this.addWorkflows(response.collection.workflows);
return this.getCollectionById(collectionId);
},
async getCategories(): Promise<ITemplatesCategory[]> {
const cachedCategories = this.allCategories;
if (cachedCategories.length) {
return cachedCategories;
}
const settingsStore = useSettingsStore();
const apiEndpoint: string = settingsStore.templatesHost;
const versionCli: string = settingsStore.versionCli;
const response = await getCategories(apiEndpoint, { 'n8n-version': versionCli });
const categories = response.categories;

this.addCategories(categories);
return categories;
},
async getCollections(query: ITemplatesQuery): Promise<ITemplatesCollection[]> {
const cachedResults = this.getSearchedCollections(query);
if (cachedResults) {
Expand All @@ -291,6 +270,7 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
async getWorkflows(query: ITemplatesQuery): Promise<ITemplatesWorkflow[]> {
const cachedResults = this.getSearchedWorkflows(query);
if (cachedResults) {
this.categories = this.workflowSearches[getSearchKey(query)].categories ?? [];
return cachedResults;
}

Expand All @@ -300,14 +280,25 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {

const payload = await getWorkflows(
apiEndpoint,
{ ...query, skip: 0, limit: TEMPLATES_PAGE_SIZE },
{ ...query, skip: 1, limit: TEMPLATES_PAGE_SIZE },
{ 'n8n-version': versionCli },
);

this.addWorkflows(payload.workflows);
this.setCategories(payload.filters);
this.addWorkflowsSearch({ ...payload, query });
return this.getSearchedWorkflows(query) || [];
},
setCategories(facets: TemplateSearchFacet[]): void {
const categories = facets.find((facet) => facet.field_name === 'categories');
if (!categories) {
return;
}
this.categories = categories.counts.map((category) => ({
name: category.value,
result_count: category.count,
}));
},
async getMoreWorkflows(query: ITemplatesQuery): Promise<ITemplatesWorkflow[]> {
if (this.isSearchLoadingMore(query) && !this.isSearchFinished(query)) {
return [];
Expand All @@ -320,12 +311,13 @@ export const useTemplatesStore = defineStore(STORES.TEMPLATES, {
try {
const payload = await getWorkflows(apiEndpoint, {
...query,
skip: cachedResults.length,
skip: cachedResults.length / TEMPLATES_PAGE_SIZE + 1,
MiloradFilipovic marked this conversation as resolved.
Show resolved Hide resolved
limit: TEMPLATES_PAGE_SIZE,
});

this.setWorkflowSearchLoaded(query);
this.addWorkflows(payload.workflows);
this.setCategories(payload.filters);
this.addWorkflowsSearch({ ...payload, query });

return this.getSearchedWorkflows(query) || [];
Expand Down
21 changes: 4 additions & 17 deletions packages/editor-ui/src/views/TemplatesSearchView.vue
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,7 @@
<div :class="$style.filters">
<TemplateFilters
:categories="templatesStore.allCategories"
:sort-on-populate="areCategoriesPrepopulated"
:loading="loadingCategories"
:loading="loadingWorkflows"
:selected="categories"
@clear="onCategoryUnselected"
@clearAll="onCategoriesCleared"
Expand Down Expand Up @@ -60,8 +59,8 @@
<TemplateList
:infinite-scroll-enabled="true"
:loading="loadingWorkflows"
:total-workflows="totalWorkflows"
:workflows="workflows"
:total-count="totalWorkflows"
@loadMore="onLoadMore"
@openTemplate="onOpenTemplate"
/>
Expand Down Expand Up @@ -127,10 +126,8 @@ export default defineComponent({
},
data() {
return {
areCategoriesPrepopulated: false,
categories: [] as number[],
categories: [] as string[],
loading: true,
loadingCategories: true,
loadingCollections: true,
loadingWorkflows: true,
search: '',
Expand Down Expand Up @@ -191,7 +188,6 @@ export default defineComponent({
},
async mounted() {
setPageTitle('n8n - Templates');
void this.loadCategories();
void this.loadWorkflowsAndCollections(true);
void this.usersStore.showPersonalizationSurvey();

Expand All @@ -208,10 +204,7 @@ export default defineComponent({
}

if (typeof this.$route.query.categories === 'string' && this.$route.query.categories.length) {
this.categories = this.$route.query.categories
.split(',')
.map((categoryId) => parseInt(categoryId, 10));
this.areCategoriesPrepopulated = true;
this.categories = this.$route.query.categories.split(',');
}
},
methods: {
Expand Down Expand Up @@ -336,12 +329,6 @@ export default defineComponent({
this.loadingWorkflows = false;
}
},
async loadCategories() {
try {
await this.templatesStore.getCategories();
} catch (e) {}
this.loadingCategories = false;
},
async loadCollections() {
try {
this.loadingCollections = true;
Expand Down
Loading