Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate docs site to docfx #322

Merged
merged 44 commits into from
Mar 2, 2023
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
0eae6a0
Docfx setup
nzdev Dec 12, 2022
44544d5
Material
nzdev Dec 12, 2022
685b595
refs
nzdev Dec 12, 2022
c297e6d
Improve url
nzdev Dec 12, 2022
b9e3a07
Set logo and favicon
nzdev Dec 12, 2022
644bf7d
Filter apidocs.
nzdev Dec 12, 2022
1b3d496
API docs links
nzdev Dec 12, 2022
91fc704
split out v1/2 docs
nzdev Dec 12, 2022
cd08aba
Remove v1 from v3 docs.
nzdev Dec 13, 2022
a737ebf
Link Lucene.NET, retitle v1/v2 pages
nzdev Dec 13, 2022
0bf8fe0
Update nuget cmd version
nzdev Dec 13, 2022
31acdd5
versions and links
nzdev Dec 13, 2022
0fc7357
links
nzdev Dec 13, 2022
96c7e69
Link configuration doc to api docs.
nzdev Dec 13, 2022
71e3cd5
Link indexing
nzdev Dec 13, 2022
385e544
linking
nzdev Dec 13, 2022
1d3a2f2
links
nzdev Dec 13, 2022
26e4d35
Link and update searching.
nzdev Dec 13, 2022
e738c56
link and remove out of date paging section.
nzdev Dec 13, 2022
0dbb363
Fix links
nzdev Dec 13, 2022
516262f
Add toc for paging
nzdev Dec 13, 2022
20d662d
favicon
nzdev Dec 13, 2022
6ade7ab
Rename
nzdev Dec 13, 2022
88267b8
Move docfx to docs/v2
nzdev Dec 30, 2022
ab58959
Restore original doc files.
nzdev Dec 30, 2022
a9c7654
Fix metadata gen path
nzdev Dec 30, 2022
63458c0
Create docfx.yml
nzdev Jan 28, 2023
d6fc9ad
Update docfx.yml
nzdev Jan 28, 2023
ab78dc3
Update docfx.yml
nzdev Jan 28, 2023
d417851
Update docfx.yml
nzdev Jan 28, 2023
33a8f94
Update docfx.yml
nzdev Jan 28, 2023
904563e
Update docfx.yml
nzdev Jan 28, 2023
fc0ec5e
Update docfx.yml
nzdev Feb 6, 2023
73271b6
Rename docfx.yml to actions.yml
nzdev Feb 6, 2023
f9fbeb6
update dfx
nzdev Feb 6, 2023
519c110
Update actions.yml
nzdev Feb 6, 2023
d861a06
Create docfx-gh-pages.yml
nzdev Feb 10, 2023
6c94feb
Merge pull request #2 from nzdev/docfx/pipeline
nzdev Feb 10, 2023
de09408
Update docfx-gh-pages.yml
nzdev Feb 10, 2023
a762fd8
Update docfx-gh-pages.yml
nzdev Feb 10, 2023
ff9aa20
tidy
nzdev Feb 11, 2023
6a345fe
Update docfx-gh-pages.yml
nzdev Feb 28, 2023
49a5fd3
placeholder
nzdev Feb 28, 2023
9c805d3
Revert "Update docfx-gh-pages.yml"
nzdev Feb 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/actions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
on:
nzdev marked this conversation as resolved.
Show resolved Hide resolved
push:
branches:
- main
- release/*
- feature/docfx
jobs:
examine_docfx:
runs-on: windows-latest
name: DocFX
steps:
- name: Checkout Examine
uses: actions/checkout@v3.3.0
with:
ref: feature/docfx
- name: Setup .NET 6 SDK
uses: actions/setup-dotnet@v1
with:
dotnet-version: '6.0.101'
- name: Build Documentation
uses: nikeee/docfx-action@v1.0.0
with:
args: docs/v2/docfx.json
# Publish generated site using GitHub Pages
- name: Publish Documentation on GitHub Pages
uses: nzdev/examine@feature/docfx
env:
BUILD_DIR: _site # docfx's default output directory is _site
GH_PAT: ${{ secrets.GH_PAT }} # See https://github.com/examine/ghpages
54 changes: 54 additions & 0 deletions .github/workflows/docfx-gh-pages.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Sample workflow for building and deploying a Jekyll site to GitHub Pages
name: Deploy Jekyll with GitHub Pages dependencies preinstalled

on:
# Runs on pushes targeting the default branch
push:
branches: ["feature/docfx"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow one concurrent deployment
concurrency:
group: "pages"
cancel-in-progress: true

jobs:
# Build job
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Pages
uses: actions/configure-pages@v3

- name: docfx-pdf-action
# You may pin to the exact commit or the version.
# uses: VakuWare/docfx-pdf-action@c8edf99121b8375c6a0920068de633a8d0fe071f
uses: VakuWare/docfx-pdf-action@v1.4.0
Shazwazza marked this conversation as resolved.
Show resolved Hide resolved
with:
args: docs/v2/docfx.json
- name: Upload artifact
uses: actions/upload-pages-artifact@v1
with:
path: docs/v2/_site
Shazwazza marked this conversation as resolved.
Show resolved Hide resolved

# Deployment job
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v1
9 changes: 9 additions & 0 deletions docs/v2/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
###############
# folder #
###############
/**/DROP/
/**/TEMP/
/**/packages/
/**/bin/
/**/obj/
_site
25 changes: 25 additions & 0 deletions docs/v2/404.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
---
permalink: /404.html
layout: default
---

<style type="text/css" media="screen">
.container {
margin: 10px auto;
max-width: 600px;
text-align: center;
}
h1 {
margin: 30px 0;
font-size: 4em;
line-height: 1;
letter-spacing: -1px;
}
</style>

<div class="container">
<h1>404</h1>

<p><strong>Page not found :(</strong></p>
<p>The requested page could not be found.</p>
</div>
10 changes: 10 additions & 0 deletions docs/v2/404.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
layout: page
title: 404 Error
tagline: Ah snap! We could not find what you are looking for!
permalink: /404.html
---

Try got get back to the Home Page and start over!

[Go to the Home Page]({{ '/' | absolute_url }})
Binary file added docs/v2/android-chrome-192x192.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/v2/android-chrome-384x384.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
5 changes: 5 additions & 0 deletions docs/v2/api/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
###############
# temp file #
###############
*.yml
.manifest
8 changes: 8 additions & 0 deletions docs/v2/api/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
uid: apidocsindex
---
# Examine V3 API Documentation

API documentation is automatically generated.

_**Tip**: There are many unit tests in the source code that can be used as Examples of how to do things. There is also a test web project that has plenty of examples of how to configure indexes and search them._
194 changes: 194 additions & 0 deletions docs/v2/articles/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
---
layout: page
title: Configuration
permalink: /configuration
uid: configuration
order: 1
---

Configuration
===

An index can be configured in many ways including different configurations per field such as how those values are analyzed, indexed, tokenized ... basically how the data is stored and retrieved. This is done via Examine ["Value Types"](#value-types).

_**Note**: This documentation refers to using Lucene based indexes in Examine (the default index type shipped in Examine)._

## Field definitions

A Field Definition is a mapping of a field name to a ["Value Types"](#value-types). By default all fields are mapped to the default Value Type: [`FieldDefinitionTypes.FullText`](xref:Examine.FieldDefinitionTypes#Examine_FieldDefinitionTypes_FullText).

You can map a field to any value type when configuring the index.

### IConfigureNamedOptions

Configuration of Examine indexes is done with [.NET's Options pattern](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options?view=aspnetcore-5.0). For Examine, this is done with named options: [`IConfigureNamedOptions`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.iconfigurenamedoptions-1).

There are several options that can be configured, the most common ones are:

* __FieldDefinitions__ _[`FieldDefinitionCollection`](xref:Examine.FieldDefinitionCollection)_ - Manages the mappings between a field name and it's index value type
* __Analyzer__ _`Analyzer`_ - The default Lucene Analyzer to use for each field (default = [`StandardAnalyzer`](https://lucenenet.apache.org/docs/4.8.0-beta00016/api/analysis-common/Lucene.Net.Analysis.Standard.StandardAnalyzer.html))
* __Validator__ _[`IValueSetValidator`]([`IValueSetValidator`](xref:Examine.IValueSetValidator))_ - Used to validate a value set to be indexed, if validation fails it will not be indexed
* __[IndexValueTypesFactory](xref:Examine.Lucene.IFieldValueTypeFactory)__ _`IReadOnlyDictionary<string, IFieldValueTypeFactory>`_ - Allows you to define custom Value Types

```cs
/// <summary>
/// Configure Examine indexes using .NET IOptions
/// </summary>
public sealed class ConfigureIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
public void Configure(string name, LuceneDirectoryIndexOptions options)
{
switch (name)
{
case "MyIndex":
// Set the "Price" field to map to the 'Double' value type.
options.FieldDefinitions.AddOrUpdate(
new FieldDefinition("Price", FieldDefinitionTypes.Double));
break;
}
}

public void Configure(LuceneDirectoryIndexOptions options)
=> Configure(string.Empty, options);
}
```

### After construction

You can modify the field definitions [FieldDefinitionCollection](xref:Examine.FieldDefinitionCollection) for an index after it is constructed by using any of the following methods:

* `myIndex.FieldDefinitionCollection.TryAdd`
* `myIndex.FieldDefinitionCollection.AddOrUpdate`
* `myIndex.FieldDefinitionCollection.GetOrAdd`

These modifications __must__ be done before any indexing or searching is executed.

### Add a field value type after construction

It is possible to add custom field value types after the construction of the index, but this must be done before the index is used. Some people may prefer this method of adding custom field value types. Generally, these should be modified directly after the construction of the index.

```cs
// Create the index with all of the defaults
var myIndex = new LuceneIndex(
"MyIndex",
new SimpleFSDirectory(new DirectoryInfo("C:\\TestIndexes")));

// Add a custom field value type
myIndex.FieldValueTypeCollection.ValueTypeFactories
.TryAdd(
"phonenumber",
name => new GenericAnalyzerFieldValueType(
name,
new PhoneNumberAnalyzer()));

// Map a field to use the custom field value type
myIndex.FieldDefinitionCollection.TryAdd(
new FieldDefinition("Phone", "phonenumber"));
```

## Value types

Value types are responsible for:

* Defining a field name and if the field should be sortable, the field to store the sortable data
* Adding a field value to an index document
* Configuring how the value will be stored in the index
* Configuring the analyzer for the field
* Generating the Query for the field

These are the default field value types provided with Examine. Each value type can be resolved from the static class [`Examine.FieldDefinitionTypes`](xref:Examine.FieldDefinitionTypes) (i.e. [`Examine.FieldDefinitionTypes.FullText`](xref:Examine.FieldDefinitionTypes#Examine_FieldDefinitionTypes_FullText)).

| Value Type | Description | Sortable |
|----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|
| FullText | __Default__.<br />The field will be indexed with the index's <br />default Analyzer without any sortability. <br />Generally this is fine for normal text searching. | ❌ |
| FullTextSortable | Will be indexed with FullText but also <br />enable sorting on this field for search results. <br />_FullText sortability adds additional overhead <br />since it requires an additional index field._ | ✅ |
| Integer | Stored as a numerical structure. | ✅ |
| Float | Stored as a numerical structure. | ✅ |
| Double | Stored as a numerical structure. | ✅ |
| Long | Stored as a numerical structure. | ✅ |
| DateTime | Stored as a DateTime, <br />represented by a numerical structure. | ✅ |
| DateYear | Just like DateTime but with <br />precision only to the year. | ✅ |
| DateMonth | Just like DateTime but with <br />precision only to the month. | ✅ |
| DateDay | Just like DateTime but with <br />precision only to the day. | ✅ |
| DateHour | Just like DateTime but with <br />precision only to the hour. | ✅ |
| DateMinute | Just like DateTime but with <br />precision only to the minute. | ✅ |
| EmailAddress | Uses custom analyzers for dealing <br />with email address searching. | ❌ |
| InvariantCultureIgnoreCase | Uses custom analyzers for dealing with text so it<br /> can be searched on regardless of the culture/casing. | ❌ |
| Raw | Will be indexed without analysis, searching will<br /> only match with an exact value. | ❌ |

### Custom field value types

A field value type is defined by [`IIndexFieldValueType`](xref:Examine.Lucene.Indexing.IIndexFieldValueType)

_**Tip**: There are many implementations of IIndexFieldValueType in the source code to use as examples/reference._

A common base class that can be used for field value types is: [`IndexFieldValueTypeBase`](xref:Examine.Lucene.Indexing.IndexFieldValueTypeBase).

A common implementation that can be used for field value types for custom Analyzers is: [`GenericAnalyzerFieldValueType`](xref:Examine.Lucene.Indexing.GenericAnalyzerFieldValueType).

#### Example - Phone Number

A phone number stored in Lucene could require a custom analyzer to index and search it properly. So the best way to set this up in Examine would be to have a custom field value type for it. Since this field value type doesn't need to do anything more fancy than to provide a custom analyzer, we can create it with the [`GenericAnalyzerFieldValueType`](xref:Examine.Lucene.Indexing.GenericAnalyzerFieldValueType).

```cs
/// <summary>
/// Configure Examine indexes using .NET IOptions
/// </summary>
public sealed class ConfigureIndexOptions : IConfigureNamedOptions<LuceneDirectoryIndexOptions>
{
private readonly ILoggerFactory _loggerFactory;

public ConfigureIndexOptions(ILoggerFactory loggerFactory)
=> _loggerFactory = loggerFactory;

public void Configure(string name, LuceneDirectoryIndexOptions options)
{
switch (name)
{
case "MyIndex":
// Create a dictionary for custom value types.
// They keys are the value type names.
options.IndexValueTypesFactory = new Dictionary<string, IFieldValueTypeFactory>
{
// Create a phone number value type using the GenericAnalyzerFieldValueType
// to pass in a custom analyzer. As an example, it could use Examine's
// PatternAnalyzer to pass in a phone number pattern to match.
["phone"] = new DelegateFieldValueTypeFactory(name =>
new GenericAnalyzerFieldValueType(
name,
_loggerFactory,
new PatternAnalyzer(@"\d{3}\s\d{3}\s\d{4}", 0)))
};

// Add the field definition for a field called "phone" which maps
// to a Value Type called "phone" defined above.
options.FieldDefinitions.AddOrUpdate(new FieldDefinition("phone", "phone"));
break;
}
}

public void Configure(LuceneDirectoryIndexOptions options)
=> throw new NotImplementedException("This is never called and is just part of the interface");
}
```

The above creates a custom field value type using a custom analyzer and maps the "Phone" field to use this value type.

## ValueSet validators

An [`IValueSetValidator`](xref:Examine.IValueSetValidator) is a simple interface:

```cs
public interface IValueSetValidator
{
ValueSetValidationResult Validate(ValueSet valueSet);
}
```

That returns an result [`ValueSetValidationResult`](xref:Examine.ValueSetValidationResult) with an enum of [`ValueSetValidationStatus`](xref:Examine.ValueSetValidationStatus) values:

* `Valid` - The ValueSet is valid and will be indexed
* `Failed` - The ValueSet was invalid and will not be indexed
* `Filtered` - The ValueSet has been filtered/modified by the validator and will be indexed

Examine only has one implementation: [`ValueSetValidatorDelegate`](xref:Examine.Lucene.Providers.ValueSetValidatorDelegate) which can be used by developers as a simple way to create a validator based on a callback, else developers can implement this interface if required. By default, no ValueSet validation is done with Examine.
Loading