Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestions and Spellchecking #326

Closed
wants to merge 64 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
f517658
Wip. Lucene suggestion api.
nzdev Jan 16, 2023
dd9c84e
wip retrieve results
nzdev Jan 16, 2023
e37cf1f
Move tests
nzdev Jan 17, 2023
e4d7a8c
Suggester registration
nzdev Jan 17, 2023
458bb28
Remove unused methods.
nzdev Jan 17, 2023
183abda
Tidy options
nzdev Jan 17, 2023
1bc17d7
Tidy inteface for suggestion
nzdev Jan 17, 2023
4a786be
Revert "Tidy inteface for suggestion"
nzdev Jan 17, 2023
783b1b2
Revert "Revert "Tidy inteface for suggestion""
nzdev Jan 17, 2023
2c4a66f
Add suggester names
nzdev Jan 17, 2023
aba2cda
Add FuzzySuggester.
nzdev Jan 17, 2023
2fcadfc
overrride
nzdev Jan 17, 2023
ac60442
Add suggester and searcher information to demo
nzdev Jan 18, 2023
78d3500
Support registering suggester.
nzdev Jan 18, 2023
577775a
set manager
nzdev Jan 18, 2023
b18c0d0
Empty results
nzdev Jan 18, 2023
d49480f
Doc comments and query time analyzer
nzdev Jan 18, 2023
bebe64c
Support query time analyzer
nzdev Jan 18, 2023
b72340e
Only more popular support
nzdev Jan 18, 2023
96b87c9
Allow overriding suggestion execution
nzdev Jan 18, 2023
9e1dd0a
Support non lucene dictionary for suggestion helpers
nzdev Jan 18, 2023
9883179
Highligher support
nzdev Jan 18, 2023
37f72f2
dispose
nzdev Jan 18, 2023
368299f
docs
nzdev Jan 18, 2023
4d13a44
Feedback from pr
nzdev Jan 18, 2023
83a28a1
Search time analyzer must be set at index time.
nzdev Jan 20, 2023
f377203
Make search Analyzer available.
nzdev Jan 20, 2023
abc0347
Reorder.
nzdev Jan 20, 2023
59b138f
Refactor suggester.
nzdev Jan 20, 2023
c66e2c2
update docs
nzdev Jan 20, 2023
fd166b6
Analyzing infix suggester.
nzdev Jan 20, 2023
da5e452
fix up demo
nzdev Jan 20, 2023
20b20d8
Documentation
nzdev Jan 20, 2023
42fd508
Add Suggester directory factory.
nzdev Jan 20, 2023
be515ac
Build once on startup
nzdev Jan 20, 2023
eef76b3
Rebuild suggestions
nzdev Jan 20, 2023
392cd71
Comments.
nzdev Jan 20, 2023
e7c777b
Open up Suggesters for extension.
nzdev Mar 20, 2023
9a34253
bugfix
nzdev Mar 20, 2023
49b7c51
null check
nzdev Mar 23, 2023
499c7d8
replace on rebuild
nzdev Mar 23, 2023
3837581
enable suggestions on fullname
nzdev Mar 24, 2023
9996e35
Query time analyzer support
nzdev Mar 24, 2023
1a653b7
v3 compatibilty fixes
nzdev Jul 30, 2023
948c3a2
Merge branch 'release/3.0' of https://github.com/Shazwazza/Examine in…
nzdev Jul 30, 2023
271fe4d
v3.2 api compat
nzdev Jul 30, 2023
49fe8e5
v4 merge
nzdev Jul 30, 2023
18bc46f
doc fixes
nzdev Jul 30, 2023
aa046a9
remove unused usings
nzdev Jul 30, 2023
de95690
Null checks
nzdev Jul 30, 2023
1785444
docfx documentation for suggestions
nzdev Jul 30, 2023
77a97e3
Avoid optional params
nzdev Jul 30, 2023
b876574
Merge branch 'release/4.0' into v3/feature/suggestions
nzdev Aug 16, 2023
4278634
docfx modern
nzdev Aug 16, 2023
6967348
merge
nzdev Aug 16, 2023
2970f07
v4
nzdev Oct 30, 2023
109cacd
Merge branch 'feature/docfx' of https://github.com/nzdev/Examine into…
nzdev Oct 30, 2023
643237e
Update roadmap
nzdev Oct 30, 2023
511e3d1
Update docfx action
nzdev Oct 30, 2023
8f8ac27
use nunit docfx
nzdev Oct 30, 2023
12cadec
docfx action
nzdev Oct 30, 2023
8f15799
2.10 action
nzdev Oct 31, 2023
009a449
merge v4
nzdev Oct 31, 2023
20fdbfd
Merge branch 'feature/docfx' into v3/feature/suggestions
nzdev Oct 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docfx-gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
- name: Setup Github Pages
uses: actions/configure-pages@v3
- name: Build DocFX Site
uses: VakuWare/docfx-pdf-action@v1.4.0
uses: nunit/docfx-action@v2.10.0
with:
args: docs/docfx.json
- name: Upload docfx built site artifact
Expand Down
2 changes: 1 addition & 1 deletion docs/api/index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
uid: apidocsindex
---
# Examine V3 API Documentation
# Examine V4 API Documentation

API documentation is automatically generated.

Expand Down
1 change: 0 additions & 1 deletion docs/articles/configuration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: Configuration
permalink: /configuration
uid: configuration
Expand Down
1 change: 0 additions & 1 deletion docs/articles/indexing.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: Indexing
permalink: /indexing
uid: indexing
Expand Down
1 change: 0 additions & 1 deletion docs/articles/searching.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: Searching
permalink: /searching
uid: searching
Expand Down
1 change: 0 additions & 1 deletion docs/articles/sorting.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: Sorting
permalink: /sorting
uid: sorting
Expand Down
87 changes: 87 additions & 0 deletions docs/articles/suggesting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
layout: page
title: Suggesting
permalink: /suggesting
ref: suggesting
order: 2
---
Suggesting and Spell Checking
===

_**Tip**: There are many examples of searching in the [`SuggesterApiTests` source code](https://github.com/Shazwazza/Examine/blob/master/src/Examine.Test/Lucene/Suggest/SuggesterApiTests.cs) to use as examples/reference._


## Registering Suggesters

On the index to register the Suggesters, create a SuggesterDefinitionCollection and set it on IndexOptions.SuggesterDefinitions

Examples for suggesting or spellchecking based on the "fullName" index field.

```cs
var suggesters = new SuggesterDefinitionCollection();
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.AnalyzingInfixSuggester, ExamineLuceneSuggesterNames.AnalyzingInfixSuggester, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.AnalyzingSuggester, ExamineLuceneSuggesterNames.AnalyzingSuggester, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker, ExamineLuceneSuggesterNames.DirectSpellChecker, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker_LevensteinDistance, ExamineLuceneSuggesterNames.DirectSpellChecker_LevensteinDistance, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker_JaroWinklerDistance, ExamineLuceneSuggesterNames.DirectSpellChecker_JaroWinklerDistance, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker_NGramDistance, ExamineLuceneSuggesterNames.DirectSpellChecker_NGramDistance, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.FuzzySuggester, ExamineLuceneSuggesterNames.FuzzySuggester, new string[] { "fullName" }));
```

## Suggester API

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new SuggestionOptions(5,ExamineLuceneSuggesterNames.AnalyzingSuggester));
```

This code will run a suggestion for the input text "Sam", returning up to 5 suggestions.

## Lucene Suggesters

To generate suggestions for input text, retreive the ISuggester from the index IIndex.Suggester.

### Analyzing Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.AnalyzingInfixSuggester));
```

This code will run a suggestion for the input text "Sam", returning up to 5 suggestions, and will highlight the result text which matches the input text..

### Analyzing Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.AnalyzingSuggester));
```

This code will run a suggestion for the input text "Sam", returning up to 5 suggestions.

### Fuzzy Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.FuzzySuggester));
```

This code will run a Fuzzy suggestion for the input text "Sam", returning up to 5 suggestions.

## Lucene Spellcheckers

To generate spellchecker suggestions for input text, retreive the ISuggester from the index IIndex.Suggester.

### Direct SpellChecker Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.DirectSpellChecker));
```

This code will run a spellchecker suggestion for the input text "Sam" returning up to 5 suggestions.
6 changes: 5 additions & 1 deletion docs/articles/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@
- name: Sorting
href: sorting.md
- name: Paging
href: sorting.md#paging-and-limiting-results
href: sorting.md#paging-and-limiting-results
- name: Suggesters
href: suggesting.md
- name: Spellchecking
href: suggesting.md#lucene-spellcheckers
1 change: 0 additions & 1 deletion docs/developerguides/buildtest.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: Build and Test
permalink: /devbuildtest
uid: devbuildtest
Expand Down
3 changes: 1 addition & 2 deletions docs/developerguides/codestructure.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
---
layout: page
title: Code Structure
permalink: /devcodestructure
uid: devcodestructure
order: 1
order: 2
---

Code Structure
Expand Down
5 changes: 2 additions & 3 deletions docs/developerguides/docsite.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
---
layout: page
title: Documentation Site
permalink: /devdocsite
uid: devdocsite
order: 1
order: 3
---

Documentation Site
Expand All @@ -16,7 +15,7 @@ The easiest way to get started is to edit an existing page by clicking the Impro

## Building documentation

1. Download [docfx](https://github.com/dotnet/docfx/releases).
1. Install [docfx](https://github.com/dotnet/docfx/releases) dotnet tool update -g docfx.
2. Unzip the release and add the folder to your system path variables.
3. Open a terminal, for example PowerShell or the VS Code terminal.
4. Change directory to /docs
Expand Down
17 changes: 11 additions & 6 deletions docs/developerguides/roadmap.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
---
layout: page
title: Roadmap
permalink: /devroadmap
uid: devroadmap
order: 1
order: 4
---

Examine.Lucene Roadmap
Expand All @@ -13,21 +12,27 @@ This page covers the roadmap for the core Examine abstractions and the Examine.L

## Ongoing and future work

- [ ] [Facet Deep Paging](https://github.com/Shazwazza/Examine/pull/321)
- [ ] [Hierarchical Faceting API (Taxonomy)](https://github.com/Shazwazza/Examine/pull/323)
- [ ] [Suggestions API and Spellchecking API](https://github.com/Shazwazza/Examine/pull/326)
- [ ] [Similarity API (BM25)](https://github.com/Shazwazza/Examine/pull/327)
- [ ] [GeoSpatial API](https://github.com/Shazwazza/Examine/pull/328)
- [ ] [Custom Scoring API](https://github.com/Shazwazza/Examine/pull/338)

## Past and completed work

- [x] [Facet Deep Paging](https://github.com/Shazwazza/Examine/pull/321)
- [x] [Hierarchical Faceting API (Taxonomy)](https://github.com/Shazwazza/Examine/pull/323)
- [x] [Nullable Support](https://github.com/Shazwazza/Examine/pull/313)
- [x] [Faceting API (SortedDocValues)](https://github.com/Shazwazza/Examine/pull/311)
- [x] [Docfx website](https://github.com/Shazwazza/Examine/pull/322)
- [x] [Deep Paging](https://github.com/Shazwazza/Examine/pull/320)

## Release Notes

### Examine 3.1
[Examine 3.1 Release Notes](https://github.com/Shazwazza/Examine/releases/tag/v3.1.0)
### Examine 4.0.0 beta 1
[Examine 4.0.0 beta 1 Release Notes](https://github.com/Shazwazza/Examine/releases/tag/v4.0.0-beta.1)

### Examine v3.2.0 beta 9
[Examine v3.2.0 beta 9 Release Notes](https://github.com/Shazwazza/Examine/releases/tag/v3.2.0-beta.9)

### Examine 3.1.0
[Examine 3.1.0 Release Notes](https://github.com/Shazwazza/Examine/releases/tag/v3.1.0)
11 changes: 10 additions & 1 deletion docs/docfx.json
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,14 @@
"fileMetadataFiles": [],
"template": [
"default",
"templates/material"
"modern",
"templates/examinemodern"
],
"sitemap": {
"baseUrl": "https://shazwazza.github.io/Examine",
"priority": 0.1,
"changefreq": "monthly"
},
"postProcessors": ["ExtractSearchIndex"],
"markdownEngineName": "markdig",
"noLangKeyword": false,
Expand All @@ -76,6 +82,7 @@
"disableGitFeatures": false,
"globalMetadata": {
"_appTitle": "Examine",
"_appName": "Examine",
"_appFooter": "Examine",
"_enableSearch": true,
"_gitContribute": {
Expand All @@ -85,6 +92,8 @@
},
"_appLogoPath": "images/headerlogo.png",
"_appFaviconPath": "images/favicon.ico"
},
"fileMetadata": {
}
}
}
1 change: 0 additions & 1 deletion docs/docs-v1-v2/configuration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: V1/V2 Configuration
permalink: /configuration
uid: v2configuration
Expand Down
1 change: 0 additions & 1 deletion docs/docs-v1-v2/indexing.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: V1/V2 Indexing
permalink: /indexing
uid: v2indexing
Expand Down
1 change: 0 additions & 1 deletion docs/docs-v1-v2/searching.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: V1/V2 Searching
permalink: /searching
uid: v2searching
Expand Down
1 change: 0 additions & 1 deletion docs/docs-v1-v2/sorting.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
---
layout: page
title: V1/V2 Sorting
permalink: /sorting
uid: v2sorting
Expand Down
3 changes: 2 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ Releases are available [here](https://github.com/Shazwazza/Examine/releases) and

| Examine Version | .NET |
| --------------- | ---- |
| V4 | .NET Standard 2.0 |
| V3 | .NET Standard 2.0 |
| V2 | .NET Standard 2.0 |
| V1 | .NET Framework 4.5.2 |
Expand All @@ -47,7 +48,7 @@ Releases are available [here](https://github.com/Shazwazza/Examine/releases) and
1. Install

```powershell
> dotnet add package Examine --version 3.0.1
> dotnet add package Examine --version 4.0.0-beta1
```

1. Configure Services and create an index
Expand Down
83 changes: 83 additions & 0 deletions docs/suggesting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
layout: page
title: Suggesting
permalink: /suggesting
ref: suggesting
order: 2
---
Suggesting and Spell Checking
===

_**Tip**: There are many examples of searching in the [`SuggesterApiTests` source code](https://github.com/Shazwazza/Examine/blob/master/src/Examine.Test/Lucene/Suggest/SuggesterApiTests.cs) to use as examples/reference._


## Registering Suggesters

On the index to register the Suggesters, create a SuggesterDefinitionCollection and set it on IndexOptions.SuggesterDefinitions

Example

```cs
var suggesters = new SuggesterDefinitionCollection();
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.AnalyzingInfixSuggester, ExamineLuceneSuggesterNames.AnalyzingInfixSuggester, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.AnalyzingSuggester, ExamineLuceneSuggesterNames.AnalyzingSuggester, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker, ExamineLuceneSuggesterNames.DirectSpellChecker, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker_LevensteinDistance, ExamineLuceneSuggesterNames.DirectSpellChecker_LevensteinDistance, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker_JaroWinklerDistance, ExamineLuceneSuggesterNames.DirectSpellChecker_JaroWinklerDistance, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.DirectSpellChecker_NGramDistance, ExamineLuceneSuggesterNames.DirectSpellChecker_NGramDistance, new string[] { "fullName" }));
suggesters.AddOrUpdate(new SuggesterDefinition(ExamineLuceneSuggesterNames.FuzzySuggester, ExamineLuceneSuggesterNames.FuzzySuggester, new string[] { "fullName" }));
```

## Suggester API

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new SuggestionOptions(5,ExamineLuceneSuggesterNames.AnalyzingSuggester));
```

This code will run a suggestion for the input text "Sam", returning up to 5 suggestions.

## Lucene Suggesters

To generate suggestions for input text, retreive the ISuggester from the index IIndex.Suggester.

### Analyzing Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.AnalyzingInfixSuggester));
```

This code will run a suggestion for the input text "Sam", returning up to 5 suggestions, and will highlight the result text which matches the input text..

### Analyzing Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.AnalyzingSuggester));
```

This code will run a suggestion for the input text "Sam", returning up to 5 suggestions.

### Fuzzy Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.FuzzySuggester));
```

This code will run a Fuzzy suggestion for the input text "Sam", returning up to 5 suggestions.

### Direct SpellChecker Suggester

```cs
var suggester = index.Suggester;
var query = suggester.CreateSuggestionQuery();
var results = query.Execute("Sam", new LuceneSuggestionOptions(5, ExamineLuceneSuggesterNames.DirectSpellChecker));
```

This code will run a spellchecker suggestion for the input text "Sam" returning up to 5 suggestions.
Loading
Loading