Es5 compat

[fifthestateinc]

The tests here will fail unless pelias/config#45 is accepted because the fixture sets an expectation for the index_concurrency field, but that is moot for 5.x compat.

[orangejulius]

Hey @fifthestateinc /@cag, can you rebase this on top of master? I just merged #196 which includes an updated version of pelias-config and the changes to support the removal of the index_concurrency setting. The tests should pass once you do that!

[cag]

@orangejulius Hey I didn't know about rebasing until now. Thanks!

[kradem]

Hey everybody!

It's not allowed in ES5 to create index by POST, here's the output of both res and err:

~/repos/pelias/schema$ node scripts/create_index.js

--------------
 create index 
--------------

Bad Request 

No handler found for uri [/pelias] and method [POST]

~/repos/pelias/schema/scripts/create_index.js

...
client.indices.create( { index: indexName, body: schema }, function( err, res ){
  if( err ){
    console.error( err.message || err, '\n' );
    console.log(res);
    process.exit(1);
  }
  console.log( '[put mapping]', '\t', indexName, res, '\n' );
  process.exit( !!err );
});

[missinglink]

@kradem I was looking at this a couple days ago and this branch had some issues such as the one you mentioned. I merged the code in to a branch called es5-compat on this repo and fixed the issues, have a look at that branch.

Also, ensure that your ~/pelias.json has esclient.apiVersion set to 5.0.

  "esclient": {
    "apiVersion": "5.0",

[missinglink]

see the new docker repo for an easy way of experimenting with es5

[cag]

@missinglink LGTM, let's close this PR in favor of your branch.

[kradem]

Hi @missinglink!

I successfuly installed and run importers (guided by https://github.com/pelias/dockerfiles/blob/master/openstreetmap/Dockerfile), and api service started with warning:

$ sudo systemctl status pelias
● pelias.service - Pelias API server
   Loaded: loaded (/lib/systemd/system/pelias.service; enabled; vendor preset: enabled)
   Active: active (running) since Čet 2017-04-27 22:36:51 CEST; 5s ago
 Main PID: 5168 (npm)
    Tasks: 17
   Memory: 1.9G
      CPU: 4.219s
   CGroup: /system.slice/pelias.service
           ├─5168 npm                             
           ├─5179 sh -c node index.js
           └─5180 node index.js

Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.049Z - info: [api] language: using null transport
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.050Z - warn: [api] language: configuration not found
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.050Z - info: [api] language: using null transport
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.050Z - warn: [api] language: configuration not found
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.050Z - info: [api] language: using null transport
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.051Z - warn: [api] language: configuration not found
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.051Z - info: [api] language: using null transport
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.051Z - warn: [api] language: configuration not found
Tra 27 22:36:56 npm[5168]: 2017-04-27T20:36:56.051Z - info: [api] language: using null transport
Tra 27 22:36:56 npm[5168]: pelias is now running on port 3100

And upon search an error occurs (excerpt from status):

Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.117Z - warn: [api] unknown geocoding error string: [parsing_exception] failed to parse [function_score] query. [you can either define [functions] array or a single function, not both. already found [functions] array, now encountering [filter].], with { line=1 & col=627 }

And excerpt from journalctl -u):

Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.105Z - info: [api] [lang] 'en' via 'header'
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.105Z - info: [api] [req] endpoint=/v1/autocomplete text=zad, , tokens=[zad], tokens_complete=[zad], tokens_incomplete=[], size=10, private=false, focus.point.lat=45.24782097102814, focus.point.lon=15.308074951171877, name=English, iso6391=en, iso6393=eng, defaulted=false
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.106Z - warn: [api] chaos monkey asks: what happens now?
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.106Z - info: [api] [query:autocomplete] [parser:addressit] [param:focus_point]
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.106Z - debug: [api] [ES req] index=pelias, searchType=dfs_query_then_fetch, must=[$=peliasQueryFullToken, type=phrase, $=1, $=3, $=zad], should=[$=peliasPhrase, type=phrase, $=1, $=3, $=zad, $=peliasQueryPartialToken, $=100, $=zad, type=phrase, operator=and, $=3, functions=[$=15, $=45.24782097102814, $=15.308074951171877, $=0km, $=250km, $=0.5], $=avg, $=replace, or=[layer=venue, layer=address], , $=20, functions=[$=log1p, $=popularity, missing=1, $=1], score_mode=first, boost_mode=replace, , $=20, functions=[$=log1p, $=population, missing=1, $=3], score_mode=first, boost_mode=replace], $=20, $=true, sort=[_score]
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.114Z - error: [api] elasticsearch error [parsing_exception] failed to parse [function_score] query. [you can either define [functions] array or a single function, not both. already found [functions] array, now encountering [filter].], with { line=1 & col=627 } :: {"path":"/pelias/_search","query":{"search_type":"dfs_query_then_fetch"},"body":"{\"query\":{\"bool\":{\"must\":[{\"match\":{\"name.default\":{\"analyzer\":\"peliasQueryFullToken\",\"type\":\"phrase\",\"boost\":1,\"slop\":3,\"query\":\"zad\"}}}],\"should\":[{\"match\":{\"phrase.default\":{\"analyzer\":\"peliasPhrase\",\"type\":\"phrase\",\"boost\":1,\"slop\":3,\"query\":\"zad\"}}},{\"function_score\":{\"query\":{\"match\":{\"name.default\":{\"analyzer\":\"peliasQueryPartialToken\",\"boost\":100,\"query\":\"zad\",\"type\":\"phrase\",\"operator\":\"and\",\"slop\":3}}},\"functions\":[{\"weight\":15,\"linear\":{\"center_point\":{\"origin\":{\"lat\":45.24782097102814,\"lon\":15.308074951171877},\"offset\":\"0km\",\"scale\":\"250km\",\"decay\":0.5}}}],\"score_mode\":\"avg\",\"boost_mode\":\"replace\",\"filter\":{\"or\":[{\"term\":{\"layer\":\"venue\"}},{\"term\":{\"layer\":\"address\"}}]}}},{\"function_score\":{\"query\":{\"match_all\":{}},\"max_boost\":20,\"functions\":[{\"field_value_factor\":{\"modifier\":\"log1p\",\"field\":\"popularity\",\"missing\":1},\"weight\":1}],\"score_mode\":\"first\",\"boost_mode\":\"replace\"}},{\"function_score\":{\"query\":{\"match_all\":{}},\"max_boost\":20,\"functions\":[{\"field_value_factor\":{\"modifier\":\"log1p\",\"field\":\"population\",\"missing\":1},\"weight\":3}],\"score_mode\":\"first\",\"boost_mode\":\"replace\"}}]}},\"size\":20,\"track_scores\":true,\"sort\":[\"_score\"]}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"parsing_exception\",\"reason\":\"failed to parse [function_score] query. [you can either define [functions] array or a single function, not both. already found [functions] array, now encountering [filter].]\",\"line\":1,\"col\":627}],\"type\":\"par
Tra 27 22:41:06 npm[5168]: sing_exception\",\"reason\":\"failed to parse [function_score] query. [you can either define [functions] array or a single function, not both. already found [functions] array, now encountering [filter].]\",\"line\":1,\"col\":627},\"status\":400}"}
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.114Z - debug: [api] [ES response]
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.115Z - warn: [api] unknown geocoding error string: [parsing_exception] failed to parse [function_score] query. [you can either define [functions] array or a single function, not both. already found [functions] array, now encountering [filter].], with { line=1 & col=627 }
Tra 27 22:41:06 npm[5168]: 2017-04-27T20:41:06.115Z - warn: [api] unknown geocoding error string: [parsing_exception] failed to parse [function_score] query. [you can either define [functions] array or a single function, not both. already found [functions] array, now encountering [filter].], with { line=1 & col=627 }

[trescube]

This is on hold until we're ready to upgrade to elasticsearch 5 (which is not on our calendar as a firm date but it's definitely important to us).

[orangejulius]

It turns out the geohash parameters that this PR allowed us to enable for ES2 while disabling for ES5 were actually deprecated in ES 2.3 and 2.4 and so we were able to remove them completely in #321. So we don't need this PR any more.