Server backend for the EUDAT resource accounting service
An aggregator for accounting records in the EUDAT Common Data Infrastructure. Records are made to an account - typically created per registered (storage) resource - and accounts can be grouped into domains. Each domain has its own user base.
This package implements the back-end service. There is no user interface except for a few primitiv web-based admin views to create domains, accounts and users.
Record creation is easiest with the accompanying command line client available from https://github.com/EUDAT-DPMT/eudat.accounting.client but any way to generate a suitble HTTP request will work (see Adding Records below).
Accumulated records are available per account in JSON format.
All accumulated data are stored in the schemaless database ZODB http://www.zodb.org/ made web-accessible by the web application server Zope (now largely maintained by the Plone community). The server application itself only provides a few marker interfaces to which browser views are registered implementing the functionality needed beyond what Zope provides out-of-the-box
The application is rather minimalistic. So far, there are only
methods for creating and viewing records: addRecord
and
listRecords
- to be invoked on accounts - respectively.
All else needs to be set-up through-the-web.
This will almost certainly change in the future.
The recommended way to install the service is by using buildout within a virtual environment. There is a ready-made buildout configuration available on GitHub (eudat.accounting.server_buildout) but you can just as well roll out your own:
- Create a virtual environment with a recent version of Python 2.7
- In there, install
zc.buildout
:
$ bin/pip install zc.buildout
3. Add a buildout configuration file (buldout.cfg
) to
the virtualenv folder. An example configuration could look like this:
[buildout]
parts =
zeo
instance
debug-client
eggs =
eudat.accounting.server
develop =
src/eudat.accounting.server
zcml =
eudat.accounting.server
[zeo]
recipe = plone.recipe.zeoserver
[instance]
recipe = plone.recipe.zope2instance
user = admin:secret
zeo-client = on
shared-blob = on
eggs = ${buildout:eggs}
zcml = ${buildout:zcml}
[debug-client]
recipe = plone.recipe.zope2instance
http-address = 8081
user = admin:secret
zeo-client = on
shared-blob = on
eggs = ${buildout:eggs}
zcml = ${buildout:zcml}
This example already illustrates how to install two server clients
sharing the same database instance via ZEO. It also shows how to
install the code base of the application direclty from GitHub -
assuming you have cloned the repository into src/
. When
working with a release, the develop
directive can be skipped.
Running
$ bin/buildout [-v]
will generate the entire application. To run it make sure to start ZEO first followed by the application server instance:
$ bin/zeo start
$ bin/instance start
Leave the debug-client
alone. It can be useful later when
you want to get interactive access to the database while the
application is running or if you want to invoke server scripts
from the command line while the application is running.
Now point your browser to localhost:8080
(or wherever you
bound the application to) and start configuring your app.
And be sure to change the admin password ;-)
First thing to do after completing the steps outlined above is
to create one or several domains. This can be achieved in the
ZMI (Zope Management Interface) by adding a BTreeFolder
and
assigning it the marker interface eudat.accounting.server.interfaces.IDomain
available under the Interfaces
tab. It is also recommended to
add a User Folder
(available from the Add
drop-down)
where you define the users for this domain.
Next, create one or several accounts within the domain. Again these
are just BTreeFolders
but this time marked with
eudat.accounting.server.interfaces.IAccount
That's it. Now, records can be added to these accouts using
the client mentioned above and records can be listed by invoking
<base_url>/<domain_id>/<account_id>/listRecords
While it is most convenient to add records using one of the clients it is also possible to do this by hand. One way is to call an appropriately crafted URL including all data to be transmitted like in
$ curl -u "probe:<secret>" "http://localhost:8080/demo/12345/addRecord?core.value:record=321&core.unit:record=TB&core.type:record=storage"
This is assuming that you have an accounting service running
at localhost
on port 8080
and a domain called demo
with
an account 12345
and a user probe
with the corresponding
password <secret>
.
Using the :record
qualifier tells Zope to group the type
, value
and unit
into a dictionary called core
as can be seen when
recalling the data just submitted:
$ curl -u "probe:<secret>" http://localhost:8080/demo/12345/listRecords
[
{
"core": {
"type": "storage",
"value": "321",
"unit": "TB"
},
"meta": {
"ts": 1489476898,
"submission_time": "2017-03-14 07:34:58"
}
},
[...]
Note how the submission time (in UTC) has been added as meta data to the record. You can also provide additional meta data yourself as in
$ curl -u "probe:<secret>" "http://localhost:8080/demo/12345/addRecord?core.value:record=321&core.unit:record=TB&core.type:record=storage&meta.comment:record=This+is+a+demo"
resulting in
$ curl -u "probe:<secret>" "http://localhost:8080/demo/12345/listRecords
[
{
"core": {
"type": "storage",
"value": "325",
"unit": "TB"
},
"meta": {
"comment": "This is a comment",
"ts": 1489477707,
"submission_time": "2017-03-14 07:48:27"
}
},
[..]
Note that you have to take care that data are encoded correctly yourself.
Rather than embedding the record data in the URL directly you can
also collect them in a file and pass it via curl. Assume you have
a file record.txt
with the following conent:
core.value:record=321&
core.unit:record=GB&
core.type:record=storage&
meta.comment:record=This+is+a+comment&
meta.number:record=42&
meta.object_type:record=registered+object
Calling then
$ curl -u "probe:<secret>" -d @record.txt https://accounting.eudat.eu/demo/12345/addRecord
will result in
$ curl -u "probe:<secret>" "http://localhost:8080/demo/12345/listRecords
[
{
"core": {
"type": "storage",
"value": "321",
"unit": "GB"
},
"meta": {
"comment": "This is a comment",
"object_type": "registered object",
"number": "42",
"ts": 1489493390,
"submission_time": "2017-03-14 12:09:50"
}
},
[..]
With respect to the keys supported (value
, unit
, comment
) and
the grouping (what goes into core
and what goes into meta
) you
are completely free to use whatever you want thereby implementing
any policy you like.
In EUDAT, we have adopted the convention that core
needs to have
a value
, a unit
and a type
whereas everything in meta
is
optional. You can look at the clients if you want to see this
demonstrated further.
Please use a virtualenv to maintain this package, but I should not need to say that.
The installation instructions above already show how to setup a development environment.
Project home page
https://github.com/EUDAT-DPMT/eudat.accounting.server
Source code
https://github.com/EUDAT-DPMT/eudat.accounting.server
Issues tracker
https://github.com/EUDAT-DPMT/eudat.accounting.server/issues