Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Advanced Search Queries #407

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 34 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ To add PgSearch to an Active Record model, simply include the PgSearch module.
class Shape < ActiveRecord::Base
include PgSearch
end
```
```

### Multi-search vs. search scopes

Expand Down Expand Up @@ -160,7 +160,7 @@ problematic_record.published? # => true
PgSearch.multisearch("timestamp") # => Includes problematic_record
```

#### More Options
#### More Options

**Conditionally update pg_search_documents**

Expand Down Expand Up @@ -611,7 +611,7 @@ Animal.with_name_matching("fish !red !blue") # => [one_fish, two_fish]

PostgreSQL full text search also support multiple dictionaries for stemming.
You can learn more about how dictionaries work by reading the [PostgreSQL
documention](http://www.postgresql.org/docs/current/static/textsearch-dictionaries.html).
documention](http://www.postgresql.org/docs/current/static/textsearch-dictionaries.html).
If you use one of the language dictionaries, such as "english",
then variants of words (e.g. "jumping" and "jumped") will match each other. If
you don't want stemming, you should pick the "simple" dictionary which does
Expand Down Expand Up @@ -772,6 +772,30 @@ See the
[documentation](https://www.postgresql.org/docs/current/static/textsearch-controls.html)
for details on the meaning of each option.

##### :advanced

Setting this attribute to true will allow you to pass advanced search queries directly to postgres, without pg_search doing any processing on the terms.

```ruby
class Number < ActiveRecord::Base
include PgSearch
pg_search_scope :advanced_search,
against: :text,
using: {
tsearch: {advanced: true}
}
end

one = Number.create! text: 'one two three'
one = Number.create! text: 'one three two'

Number.advanced_search('one<->two') # => ['one two three']
```

See the
[documentation](https://www.postgresql.org/docs/current/static/functions-textsearch.html)
for details on the meaning of each option.

#### :dmetaphone (Double Metaphone soundalike search)

[Double Metaphone](http://en.wikipedia.org/wiki/Double_Metaphone) is an
Expand All @@ -783,7 +807,7 @@ used for searching.
Double Metaphone support is currently available as part of the [fuzzystrmatch
extension](http://www.postgresql.org/docs/current/static/fuzzystrmatch.html)
that must be installed before this feature can be used. In addition to the
extension, you must install a utility function into your database. To generate
extension, you must install a utility function into your database. To generate
and run a migration for this, run:

$ rails g pg_search:migration:dmetaphone
Expand Down Expand Up @@ -818,7 +842,7 @@ Trigram search works by counting how many three-letter substrings (or
Trigram search has some ability to work even with typos and misspellings in
the query or text.

Trigram support is currently available as part of the
Trigram support is currently available as part of the
[pg_trgm extension](http://www.postgresql.org/docs/current/static/pgtrgm.html) that must be installed before this
feature can be used.

Expand Down Expand Up @@ -882,7 +906,7 @@ Vegetable.strictly_spelled_like("collyflower") # => []
Allows you to match words in longer strings.
By default, trigram searches use `%` or `similarity()` as a similarity value.
Set `word_similarity` to `true` to opt for `<%` and `word_similarity` instead.
This causes the trigram search to use the similarity of the query term
This causes the trigram search to use the similarity of the query term
and the word with greatest similarity.

```ruby
Expand All @@ -908,12 +932,12 @@ Sentence.similarity_like("word") # => []
Sentence.word_similarity_like("word") # => [sentence]
```

### Limiting Fields When Combining Features
### Limiting Fields When Combining Features

Sometimes when doing queries combining different features you
Sometimes when doing queries combining different features you
might want to searching against only some of the fields with certain features.
For example perhaps you want to only do a trigram search against the shorter fields
so that you don't need to reduce the threshold excessively. You can specify
so that you don't need to reduce the threshold excessively. You can specify
which fields using the 'only' option:

```ruby
Expand All @@ -932,7 +956,7 @@ class Image < ActiveRecord::Base
end
```

Now you can succesfully retrieve an Image with a file_name: 'image_foo.jpg'
Now you can succesfully retrieve an Image with a file_name: 'image_foo.jpg'
and long_description: 'This description is so long that it would make a trigram search
fail any reasonable threshold limit' with:

Expand Down
32 changes: 21 additions & 11 deletions lib/pg_search/features/tsearch.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ module PgSearch
module Features
class TSearch < Feature # rubocop:disable Metrics/ClassLength
def self.valid_options
super + %i[dictionary prefix negation any_word normalization tsvector_column highlight]
super + %i[advanced dictionary prefix negation any_word normalization tsvector_column highlight]
end

def conditions
Expand Down Expand Up @@ -110,13 +110,19 @@ def tsquery_for_term(unsanitized_term) # rubocop:disable Metrics/AbcSize
# After this, the SQL expression evaluates to a string containing the term surrounded by single-quotes.
# If :prefix is true, then the term will have :* appended to the end.
# If :negated is true, then the term will have ! prepended to the front.
terms = [
(Arel::Nodes.build_quoted('!') if negated),
Arel::Nodes.build_quoted("' "),
term_sql,
Arel::Nodes.build_quoted(" '"),
(Arel::Nodes.build_quoted(":*") if options[:prefix])
].compact
terms = \
if options[:advanced]
[
term_sql,
]
else
[ (Arel::Nodes.build_quoted('!') if negated),
Arel::Nodes.build_quoted("' ") ,
term_sql,
Arel::Nodes.build_quoted(" '"),
(Arel::Nodes.build_quoted(":*") if options[:prefix])
].compact
end

tsquery_sql = terms.inject do |memo, term|
Arel::Nodes::InfixOperation.new("||", memo, Arel::Nodes.build_quoted(term))
Expand All @@ -131,9 +137,13 @@ def tsquery_for_term(unsanitized_term) # rubocop:disable Metrics/AbcSize
def tsquery
return "''" if query.blank?

query_terms = query.split(" ").compact
tsquery_terms = query_terms.map { |term| tsquery_for_term(term) }
tsquery_terms.join(options[:any_word] ? ' || ' : ' && ')
if options[:advanced]
tsquery_for_term(query)
else
query_terms = query.split(" ").compact
tsquery_terms = query_terms.map { |term| tsquery_for_term(term) }
tsquery_terms.join(options[:any_word] ? ' || ' : ' && ')
end
end

def tsdocument
Expand Down
46 changes: 46 additions & 0 deletions spec/lib/pg_search/features/tsearch_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -235,4 +235,50 @@
end
end
end
describe 'advanced' do
with_model :Model do
table do |t|
t.string :name
t.text :content
end
end

context "when options[:advanced] is true" do

it "does not wrap the query in quotes or join query terms using logical operators" do
query = "advanced query"
columns = [
PgSearch::Configuration::Column.new(:name, nil, Model),
PgSearch::Configuration::Column.new(:content, nil, Model)
]
options = { advanced: true }
config = double(:config, ignore: [])
normalizer = PgSearch::Normalizer.new(config)

feature = described_class.new(query, options, columns, Model, normalizer)
expect(feature.conditions.to_sql).to eq(
%{((to_tsvector('simple', coalesce(#{Model.quoted_table_name}."name"::text, '')) || to_tsvector('simple', coalesce(#{Model.quoted_table_name}."content"::text, ''))) @@ (to_tsquery('simple', 'advanced query')))}
)
end

end

context "when options[:advanced] is false" do
it "wraps the query in quotes and joins the terms using logical operators" do
query = "advanced query"
columns = [
PgSearch::Configuration::Column.new(:name, nil, Model),
PgSearch::Configuration::Column.new(:content, nil, Model)
]
options = { advanced: false }
config = double(:config, ignore: [])
normalizer = PgSearch::Normalizer.new(config)

feature = described_class.new(query, options, columns, Model, normalizer)
expect(feature.conditions.to_sql).to eq(
%{((to_tsvector('simple', coalesce(#{Model.quoted_table_name}."name"::text, '')) || to_tsvector('simple', coalesce(#{Model.quoted_table_name}."content"::text, ''))) @@ (to_tsquery('simple', ''' ' || 'advanced' || ' ''') && to_tsquery('simple', ''' ' || 'query' || ' ''')))}
)
end
end
end
end