Fuller featured API for get_relative_account_history

[wmbutler]

In order to fully satisfy bitshares/bitshares-ui#68, we need to be able to optionally request:

• type of operation ( transfer / cancel order / place order .... )
• date/time range
• trading pair

in addition to the features of get_relative_account_history

[abitmore]

Or perhaps improve the get_market_order_history API in market_history plugin, but not account_history plugin.

[wmbutler]

Happy to call whatever API you think it most appropriate to achieve this.

[oxarbitrage]

With the addition of the elasticsearch plugin this queries are very easy to do and much more powerful than what can currently do with a traditional full node.

I was talking with @xeroc today to add at least 1 public ES node if the infrastructure worker get voted in. In the meantime i will show some examples and provide access by request to tmp ES nodes.

get only transfers from account:

curl -X GET 'http://localhost:9200/graphene-*/data/_search?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.282"}},{"term": { "operation_type": 0}}] }
    }
}
'

getting today operations from account:

curl -X GET 'http://localhost:9200/graphene-*/data/_search?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.439704"}}, {"range": {"block_data.block_time": {"gte": "2017-11-15T00:00:00", "lte": "2017-11-15T23:59:59"}}}] }
    }
}
'

get count:

curl -X GET 'http://localhost:9200/graphene-*/data/_count?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.439704"}}, {"range": {"block_data.block_time": {"gte": "2017-11-15T00:00:00", "lte": "2017-11-15T23:59:59"}}}] }
    }
}
'

going lower into first ops of account by date:

curl -X GET 'http://localhost:9200/graphene-*/data/_search?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.439704"}}, {"range": {"block_data.block_time": {"gte": "2017-10-13T17:39:00", "lte": "2017-10-13T17:41:59"}}}] }
    }
}
'

count:

curl -X GET 'http://localhost:9200/graphene-*/data/_count?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.439704"}}, {"range": {"block_data.block_time": {"gte": "2017-10-13T17:39:00", "lte": "2017-10-13T17:41:59"}}}] }
    }
}
'

use elasticsearch date math ranges, lets get the ops in the last 1 hour:

curl -X GET 'http://localhost:9200/graphene-*/data/_count?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.439704"}}, {"range": {"block_data.block_time": {"gte": "now-1h", "lte": "now"}}}] }
    }
}
'

and in last 1 day:

curl -X GET 'http://localhost:9200/graphene-*/data/_count?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.439704"}}, {"range": {"block_data.block_time": {"gte": "now-1d", "lte": "now"}}}] }
    }
}
'

lets add operation filter, getting all the transfers in the last month from account:

curl -X GET 'http://localhost:9200/graphene-*/data/_count?pretty=true' -d '
{
    "query" : {
        "bool" : { "must" : [{"term": { "account_history.account.keyword": "1.2.24484"}},{"term": { "operation_type": 0}}, {"range": {"block_data.block_time": {"gte": "now-1M", "lte": "now"}}}] }
    }
}
'

more about date math ranges at: https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#date-math

[abitmore]

Please document all these ES related queries somewhere. And leave a link to every issue/PR. Replying with details in each issue is a bit waste of efforts.

[oxarbitrage]

yes, understood and on it. thanks.

[pmconrad]

I was talking with @xeroc today to add at least 1 public ES node if the infrastructure worker get voted in.

Please be careful. ES has security issues, you should not expose the query API to the public internet.

[oxarbitrage]

interesting, we can make something like: https://github.com/oxarbitrage/bitshares-python-api-backend/blob/master/api.py with flask and python to expose some calls with ES running on localhost. i can develop this for a full ES node with some calls, pretty easy. let me know what do you think.

[pmconrad]

Yes, a small wrapper application should do the trick. But I'm not really an expert there, perhaps spend some time googling ES security.

[oxarbitrage]

ok, thanks @pmconrad

[oxarbitrage]

linking work in progress small elasticsearch flask wrapper: https://github.com/oxarbitrage/bitshares-es-wrapper
documentation and fine tuning is in progress. Setup will be an elasticsearch database running in localhost and this wrapper exposed to public to make calls. Idea is to have 1 call that can do everything in the wrapper with a lot of arguments or a reduced number of calls instead of making 1 for every need but this is open to discussion.