Elastic search was recently used to index the Firefox4 twitter stream and make it searchable. It's based on Lucene and has a simple JSON based interface that you can use to store objects and search through them (for instance even with CURL).
This also makes it easy to have your search indexes be updated in realtime whenever your CakePHP models change data. Cause basically all we'd have to do is do a Curl PUT, DELETE, etc to also make the change in Elastisearch with every afterSave and afterDelete.
This plugin provides
- a behavior to automatically update your indexes
- a shell task to do full index fills
- a generic search component that you can attach to your AppController and will intercept search actions on enabled models. Will return results in JSON format for easy AJAX integration.
You'll need the PHP curl libraries installed.
As a fake submodule
cd ${YOURAPP}/Plugin
git clone git://github.com/kvz/cakephp-elasticsearch-plugin.git ElasticsearchAs a real submodule
cd ${REPO_ROOT}
git submodule add git://github.com/kvz/cakephp-elasticsearch-plugin.git ${YOURAPP}/Plugin/ElasticsearchUsing composer
"require": {
"kvz/elasticsearch": "dev-master"
}Config/database.php
<?php
class DATABASE_CONFIG {
public $elastic = array(
'url' => 'http://127.0.0.1:9200',
);
// ... etc
?>Models/Ticket.php (minimal example)
<?php
public $actsAs = array(
'Elasticsearch.Searchable' => array(
),
// ... etc
);
?>Models/Ticket.php (with raw sql for huge datasets)
<?php
public $actsAs = array(
'Elasticsearch.Searchable' => array(
'index_chunksize' => 1000, // per row, not per parent object anymore..
'index_find_params' => '
SELECT
`tickets`.`cc` AS \'Ticket/cc\',
`tickets`.`id` AS \'Ticket/id\',
`tickets`.`subject` AS \'Ticket/subject\',
`tickets`.`from` AS \'Ticket/from\',
`tickets`.`created` AS \'Ticket/created\',
`customers`.`customer_id` AS \'Customer/customer_id\',
`customers`.`name` AS \'Customer/name\',
`ticket_responses`.`id` AS \'TicketResponse/{n}/id\',
`ticket_responses`.`from` AS \'TicketResponse/{n}/from\',
`ticket_responses`.`created` AS \'TicketResponse/{n}/created\'
FROM `tickets`
LEFT JOIN `ticket_responses` ON `ticket_responses`.`ticket_id` = `tickets.id`
LEFT JOIN `customers` ON `customers`.`customer_id` = `tickets`.`customer_id`
WHERE 1=1
{single_placeholder}
{offset_limit_placeholder}
',
),
// ... etc
);
?>Models/Ticket.php (full example)
<?php
public $actsAs = array(
'Elasticsearch.Searchable' => array(
'debug_traces' => false,
'searcher_enabled' => false,
'searcher_action' => 'searcher',
'searcher_param' => 'q',
'searcher_serializer' => 'json_encode',
'fake_fields' => array(
'_label' => array('Product/description', 'BasketItem/description'),
),
'index_name' => 'main',
'index_chunksize' => 10000,
'index_find_params' => array(
'limit' => 1,
'fields' => array(
// It's important you name your fields.
'subject',
'from',
),
'contain' => array(
'Customer' => array(
// It's important you name your fields.
'fields' => array(
'id',
'name',
),
),
'TicketResponse' => array(
// It's important you name your fields.
'fields' => array(
'id',
'content',
'created',
),
),
'TicketObjectLink' => array(
// It's important you name your fields.
'fields' => array(
'foreign_model',
'foreign_id',
),
),
'TicketPriority' => array(
// It's important you name your fields.
'fields' => array(
'code',
'from',
),
),
'TicketQueue' => array(
// It's important you name your fields.
'fields' => array(
'name',
),
),
),
'order' => array(
'Ticket.id' => 'DESC',
),
),
'highlight' => array(
'pre_tags' => array('<em class="highlight">'),
'post_tags' => array('</em>'),
'fields' => array(
'_all' => array(
'fragment_size' => 200,
'number_of_fragments' => 1,
),
),
),
'realtime_update' => false,
'error_handler' => 'php',
'static_url_generator' => array('{model}', 'url'),
'enforce' => array(
'Customer/id' => 123,
// or a callback: '#Customer/id' => array('LiveUser', 'id'),
),
'highlight_excludes' => array(
// if you're always restricting results by customer, that
// query should probably not be part of your highlight
// instead of dumping _all and going over all fields except Customer/id,
// you can also exclude it:
'Customer/id',
),
),
);
?>To automatically enable a /<controller>/searcher url on all models
that have elastic search enabled, use:
Controller/AppController.php
<?php
public $components = array(
'Elasticsearch.Searcher',
// ... etc
);
?>This component will only actually fire when the Controller->modelClass has the searchable behavior attached.
I chose for this method (vs a dedicated SearchesController) so ACLing is easier. e.g. You may already have an ACL for /tickets/*, so /tickets/search will automatically be restricted the same way.
If you want to search on all models, you could make a dedicated search controller and instruct to search on everything like so:
<?php
class SearchersController extends AppController
{
public $components = array(
'Elasticsearch.Searcher' => array(
'model' => '_all',
'leading_model' => 'Ticket'
)
// ... etc
);
public function searcher()
{
$this->Searcher->searchAction($this->RequestHandler->isAjax());
}
}
?>One known limitation is that the Elasticsearch plugin will only look at the
first configured Model for configuration parameters like searcher_param
and searcher_action.
From your shell:
# Fill all indexes
./cake Elasticsearch.indexer fill _all
# Fill index with tickets
./cake Elasticsearch.indexer fill Ticket
# Try a ticket search from commandline
./cake Elasticsearch.indexer search Ticket HelloFrom your browser
http://www.example.com/tickets/searcher/q:*kevin*Let's look at an integration example that uses jQuery UI's autocomplete.
Assuming you have included that library, and have an input field with attributes
id="main-search" and target="/tickets/searcher/q:*{query}*":
// Main-search
$(document).ready(function() {
$("#main-search")
.autocomplete({
source: function(request, response) {
$.getJSON(
$("#main-search")
.attr("target")
.replace("{query}", request.term),
null,
response
);
},
delay: 100,
select: function(event, ui) {
var id = 0;
if ((id = ui.item.id)) {
location.href = ui.item.url;
alert("Selected: #" + id + ": " + ui.item.url);
}
return false;
}
})
.data("autocomplete")._renderItem = function(ul, item) {
return $("<li></li>")
.data("item.autocomplete", item)
.append(
"<a href='" + item.url + "'>" + item.html + "<br>" + item.descr + "</a>"
)
.appendTo(ul);
};
});- There also is an unmaintained legacy cakephp 1.3 branch
# Get Status
curl -XGET 'http://127.0.0.1:9200/_all/_status?pretty=true'
# Dangerous: Delete an entire index
curl -XDELETE 'http://127.0.0.1:9200/main'
# Dangerous: Delete an entire type
curl -XDELETE 'http://127.0.0.1:9200/main/ticket'
# Get all tickets
curl -XGET http://127.0.0.1:9200/main/ticket/_search -d '{
"query" : {
"field" : {
"_all" : "**"
}
}
}'
# Get everything
curl -XGET http://127.0.0.1:9200/main/_search?pretty=true -d '{
"query" : {
"field" : {
"_all" : "**"
}
},
"size" : 1000000
}'
# Dangerous: Delete an entire type
curl -XDELETE 'http://127.0.0.1:9200/main/ticket'
# Refresh index
curl -XPOST 'http://127.0.0.1:9200/main/_refresh'