Using filter_by and other Typesense search parameters

[furnnl]

Description

As the documentation says, it should be possible to specify additional search parameters. queryBy works, but filterBy doesn’t.

Steps to reproduce

I tried the following parameters:

additionalSearchParameters: {
queryBy: 'name',
filterBy: 'myfilter:0',
'filter_by': 'myfilter:0',
}

Expected Behavior

Filtered results

Actual Behavior

Unfiltered results

Metadata

Typsense Version:

OS:

[jasonbosco]

@furnnl The reason filterBy doesn't work when set in additionalSearchParameters is because it's overridden by InstantSearch's refinement widgets internally. So you want to manage default filters through InstantSearch's configure widget. (Also works similarly in React, Vue, Angular, etc flavors of InstantSearch).

And specifically for filters, you want to use the facetFilters key and define the filters in the format described here. The adapter will then take care of translating that into the Typesense filter format.

So in your example, you want to do something like this:

instantsearch.widgets.configure({ 
  facetFilters: ['myfilter:0']
});

Does that help?

[jasonbosco]

Closing this for now, but please feel free to re-open if you're still running into issues.

[shrugs]

Adding additional info for any react users:

const facetFilters = useMemo(() => [`myfilter:0`], [])

// ...

<Configure facetFilters={facetFilters} />

does the trick (making sure to keep the identity of that facetFilters array as constant as possible, since it'll trigger re-fetches when it changes, and providing that inline can cause an infinite fetch-render loop)

[ctorgalson]

I don't know if it's worth re-opening this but in my tests, adding a correctly-formatted facet filter to <Configure /> in React adds a property like configure: {facetFilters: ['meal:lunch']} to searchState, and adds e.g. &configure[facetFilters][0]=meal%3Alunch to the query string, but the actual query results are completely unaffected.

/shrug

[jasonbosco]

@ctorgalson Could you post the full <Configure /> line, showing the facetFilters string?

Could you also copy-as-curl the request that gets made to the Typesense server from the browser and post it?

[ctorgalson]

@jasonbosco I will, though I'll have to recreate the issue as I found an alternate solution :)

---

Adding to this thread because I didn't find it on my first ten trips through Algolia's docs: Virtual Widgets.

TL;DR: There's a way in React to provide default values without adding a "real" widget, like this (though <ClearRefinements /> will clear a refinement set this way):

<VirtualRefinementList attribute="categories" defaultRefinement="Cell Phones" />

[reinoldus]

Has anyone solved it for vue-instantsearch?

If I do: <ais-configure :filters="'my_filter:true'"> the filter is not picked up, for <ais-configure :facetFilters="['my_filter:true']"> the filter is picked up, but overwritten once I select another filter

[jasonbosco]

@reinoldus Filters need to be specified in the format mentioned here. The adapter then converts them to the Typesense filter string format internally.

the filter is picked up, but overwritten once I select another filter

This is expected behavior for the configure widget. It's only meant to set a default filter on load. If you need to enforce a filter for all searches, then you want to use the helper object that's exposed via the search function.

[krisimmig]

Hi @jasonbosco , I have tried to use the search helper you mentioned which gets injected into the searchFunction as described here. Now I want to add a default filter to all searches based on a language field we have in our typesense index so the user only sees english or german or french results. The field in the index is defined as a facet but when I try to use for example this code:

searchFunction(helper) {
  helper.addFacetRefinement("language", "de").search();
},

I get the error that the facet 'language' is not defined. It is mentioned that these facets need to be declared when initializing the helper like so:

var helper = AlgoliasearchHelper(client, indexName, {
  facets: ['nameOfTheAttribute']
});

Is there a way to use facets like that at all using the helper object? I am not sure how I can get to this using the vue version of the adapter and the instansearch-js widget set for vue.

[jasonbosco]

@krisimmig I just tried this out myself in Vue and I saw the same error you saw: "Error: X is not defined in the facets attribute of the helper configuration". Didn't find anything useful googling around. I was almost going to give up, but by some trial-and-error serendipity I found a configuration that made that error go away:

export default {
  data() {
    return {
      searchClient,
      searchParameters: {
        facets: ['language'],
      },
      searchFunction(helper) {
        helper.addFacetRefinement('language', 'de').search();
      },
    };
  },
};

Here's the full diff to get static filters working in my demo Vue app: typesense/typesense-vue-instantsearch-demo@typesense:c4d2400...typesense:7edf1bf

[krisimmig]

That´s it, awesome! :) I didn't think to find the facets param in the ais-configure widget, but it is there and like that it seems to work as expected. Thanks for your help!
For anybody else looking for something like this, here is a list of all configure options that can be used in the ais-configure widget: https://www.algolia.com/doc/api-reference/search-api-parameters/

Note: After testing this a bit more, we noticed that the load-more is not working anymore when adding facets to all searches. Not sure what is going on but we didn't have any more time to find the cause for this.
We are now using a hidden ais-toggle-refinement component to always set the language to a certain value which the user can not directly change. This works well but seems like a hack 🤷

[pjatx]

@jasonbosco Thanks for all the tips, this has been driving me insane for days.

Can you think of a way to conditionally apply the filterBy logic if a given input has a value in it? Use case is being able to filter by an exact record ID should the user want to do that.

[jasonbosco]

@pjatx You could do this outside of the context of the adapter, from the Typesense backend. You can turn off typo tolerance and prefix search for just the record ID field, and add it to the query_by fields may be in the beginning of the list. Then if a user searches for an exact record ID, that will be surfaced first.

[pjatx]

@jasonbosco thanks that's super helpful - will give it a shot!

[britisharmy]

Supported operands, less than is not supported on numeric filter

  methods: {
     searchFunction(helper) {
        //helper.addFacetRefinement('deal', 'deal').search();

        helper.addNumericRefinement('price_per_night','>',this.slider_amount).search();

     //helper.addFacetRefinement('price_per_night','>'+this.slider_amount).search();
      },

Even the commented code works, but for numeric filter, less than is not supported.

[britisharmy]

@krisimmig I just tried this out myself in Vue and I saw the same error you saw: "Error: X is not defined in the facets attribute of the helper configuration". Didn't find anything useful googling around. I was almost going to give up, but by some trial-and-error serendipity I found a configuration that made that error go away:

export default {
  data() {
    return {
      searchClient,
      searchParameters: {
        facets: ['language'],
      },
      searchFunction(helper) {
        helper.addFacetRefinement('language', 'de').search();
      },
    };
  },
};

Here's the full diff to get static filters working in my demo Vue app: typesense/typesense-vue-instantsearch-demo@typesense:c4d2400...typesense:7edf1bf

The searchFunction should be placed inside the functions block in vue js. It still works where its placed but if you have a variable, its changeable once its in the methods block._

[jasonbosco]

@britisharmy

You want to call addNumericRefinement in a method chain and also add it to facets like this:

typesense/typesense-vue-instantsearch-demo@0b30a5b

Side note: instantsearch only issues <= and >= filters from its widgets, so those are the only two operators supported by the adapter.

Separately, good call out on moving searchFunction to methods. I've updated the demo.

[pjatx]

@jasonbosco any way to have a separate search field that only filters on a single attribute within the context of the search adapter? Still wrestling with the same use case above.

More context:

• Currently using vue-instantsearch (works great)
• Using it to search through about 50K film records - fields include title, abstract, release date, film ID, etc...
• Want to use main search field (AisSearchBox) to search across all fields, but want to include a separate field that would filter all the results by an exact match on the film id. Using a refinement, but not the best UX.

Any way this is achievable in my current setup?

[jasonbosco]

@pjatx Any particular reason you want a separate search box just to search for Film ID? You could use the same search box, and turn off prefix search and typo tolerance (num_typos=0) for just the film_id field using the additionalSearchParameters parameter in the adapter.

Here's more info on these two params: https://typesense.org/docs/0.21.0/api/documents.html#arguments

[pjatx]

@jasonbosco it's more of a client request, but if I can get it working well in a single search field that should work!

This seems to be working:

  additionalSearchParameters: {
    queryBy: 'filmId,title,abstract,keywords,director,operators,otherCreators',
    prefix: 'false,true,true,true,true,true,true',
    numTypos: 0
  }

Does that only constrain the numTypos param to the first field? Tried the following and couldn't get it to work:

 numTypos: '0,2,2,2,2,2,2'

[jasonbosco]

@pjatx I see, if you really wanted to, you can have multiple search boxes, you'd need multiple instances of InstantSearch as described here.

numTypos: 0 will disable typo tolerance for all fields. numTypos: '0,2,2,2,2,2,2' is what you need. Could you expand on what you mean by it doesn't work? Did you see an error or did you get results which included typo corrected ones? In any case, could you open a separate issue in the main typesense repo for this?

[pjatx]

@jasonbosco thanks, that's definitely helpful.

Yeah no problem, here's the error I get when I try to pass a string of comma-separated values:

Error: 400 - Parameter num_typos must be an unsigned integer.

Would be happy to open a separate issue if helpful!

[jasonbosco]

Could you make sure you're running the latest version of Typesense v0.21.0 - that's the one that has support for per-field num_typos. If you're running on Typesense Cloud, if you email contact @ typesense d0t org with your cluster id, we can upgrade it in place for you.

If you're already running v0.21.0, then this sounds like a bug. Could you open a separate issue?

[pjatx]

@jasonbosco ah - that's it. I'm still on 0.19.0. Will upgrade and let you know if I have any more issues. Can't thank you enough for the help! Love Typesense : )

[panoet]

@furnnl The reason filterBy doesn't work when set in additionalSearchParameters is because it's overridden by InstantSearch's refinement widgets internally. So you want to manage default filters through InstantSearch's configure widget. (Also works similarly in React, Vue, Angular, etc flavors of InstantSearch).

And specifically for filters, you want to use the facetFilters key and define the filters in the format described here. The adapter will then take care of translating that into the Typesense filter format.

So in your example, you want to do something like this:

instantsearch.widgets.configure({ 
  facetFilters: ['myfilter:0']
});

Does that help?

So, we only can filter by field that has beed declared "facet"? In my case, user want to filter by some value that I didnot make it as facet when creating the collection. So, do I need to recreated the collection and import all the data? I didn't find any "edit schema" menu.

[jasonbosco]

So, we only can filter by field that has beed declared "facet"?

In Typesense Server, you can filter by fields that are not set as facets. However, (unfortunately) InstantSearch.js's widgets makes the assumption that all filter fields are set as facets, since that's how Algolia works. So if you're using InstantSearch with Typesense, you would also have to set any fields you want to filter by as a facet.

So, do I need to recreated the collection and import all the data?

Correct. I'd recommend using collection aliases to do this with zero downtime. We have an open item to add support for editing an existing schema: typesense/typesense#96

[Koniouchine]

For those trying to configure Angular ~12, here's a working example.
It also includes a geosearch solution that @jasonbosco helped me solve.

component.ts

// region Typesense Configuration
config = {
  indexName: 'events',
  searchClient
} as InstantSearchConfig;

searchParameters = {
  hitsPerPage: 10,
  q         : '*',
  query_by  : 'title',
  aroundLatLng: '45.406431, -78.6848739',
  aroundRadius: 1000
}
// endregion

component.html

<ais-instantsearch [config]="config" (change)="setHits($event)">
  <ais-configure
    [searchParameters]="searchParameters"
  ></ais-configure>

  <ais-search-box [searchAsYouType]="false"></ais-search-box>
  <ais-geo-search></ais-geo-search>

  <ais-hits>
    <ng-template #searchResults let-hits="hits" let-results="results">
      <div *ngFor="let hit of hits">
        {{hit.title}}
      </div>
    </ng-template>
  </ais-hits>
</ais-instantsearch>

[Koniouchine]

Further insight for filtering with numbers within Angular:
As previously mentioned, when doing a numeric search, only ">= and <=" are supported.
However you must also be use them with the 'numericFilters' field instead of 'facetFilters'.

// assign to <ais-config> widget
  searchParameters: any = {
    hitsPerPage: 10,
    numericFilters: ['date.startDate >= 1647068400'],
    q         : '*',
    query_by  : 'title',
    aroundLatLng: '44.406431, -79.6848739',
    aroundRadius: 14000 // in meters
  }

[jasonbosco]

Quick update: I just added the ability for the adapter to handle the filters prop in the Configure widget in addition to facetFilters and numericFilters in v2.4.1-3. Here's documentation for the format to use with filters.

[uncvrd]

@jasonbosco came here via a Slack Typesense thread. I just gave numericFilters a shot on 2.4.1-3 and I'm receiving the following error:

Which is weird because I'm passing an array to numericFilters. Per typsense docs, I think I'm implementing it correctly by following Algolia's syntax, right?:

<Configure 
    numericFilters={[
        `teamId = 1`
    ]}
/>

I was basing the above implementation off of this example https://www.algolia.com/doc/api-reference/api-parameters/numericFilters/#apply-numeric-filters-on-a-search-query At least, with this adapter version, I can at least get away with using filters as that seems to work in 2.4.1-3!:

<Configure 
    filters='teamId:= 3'
/>

Am I missing something obvious in the numericFilters implementation?

---

Edit: hmm, is the regex missing a match for the equal (=) sign? I am far away from a pro at regex but maybe that's why the regex is returning null and causing an error 🤔

[jasonbosco]

@uncvrd Thank you for catching that and thank you for PR #114. I've published this in v2.4.1-4.

[weiklr]

this was really helpful in figuring out how to use typesense search parameters with configure i managed to get facet refinement working as follows, (using js version of instantsearch):

 facetWidget = instantsearch.widgets.configure({
            facets: ['your facet column']
   })
searchInstance = instantsearch({indexName, typesenseSearchClient})
searchInstance.addWidget(facetWidget)
searchInstance.start()
searchInstance.helper.addFacetRefinement('facet column name', 'facet value'))

[braincomb]

Hi @jasonbosco ,

Is there a way to do this with negation? For example we want to show all items EXCEPT those that have someAttr:true. How would we do this :facetFilters?

Thanks.

@reinoldus Filters need to be specified in the format mentioned here. The adapter then converts them to the Typesense filter string format internally.

the filter is picked up, but overwritten once I select another filter

This is expected behavior for the configure widget. It's only meant to set a default filter on load. If you need to enforce a filter for all searches, then you want to use the helper object that's exposed via the search function.

[jasonbosco]

@braincomb Does addFacetExclusion help: https://instantsearchjs.netlify.app/algoliasearch-helper-js/reference#AlgoliaSearchHelper#addFacetExclusion

[braincomb]

@braincomb Does addFacetExclusion help: https://instantsearchjs.netlify.app/algoliasearch-helper-js/reference#AlgoliaSearchHelper#addFacetExclusion

Thanks @jasonbosco, that worked perfectly.

For anyone looking for the same solution, here's what I did for vue-instantsearch to do an exclusion filter:

<ais-instant-search index-name="index" :search-client="searchClient" :search-function="searchHelper">

<ais-configure :facets="['facetToExclude']"></ais-configure>

const searchHelper= (helper: any) => {
  helper.addFacetExclusion("facetToExclude", "value").search();
};