I'ts a super-simple API server demo using sinatra.
Let consider this requirements:
- Realize a fast API server, publishing some API endpoints, returning data in JSON format.
- Resources are data already stored in multiple RDBMS database tables, by example from TWO different ALREADY EXISTING (postgreSQL) databases.
- Access data using activerecord ORM, using sinatra-activerecord gem that allow to interact with DBs through activerecord ORM.
As proof of concept I supplied some Sinatra endpoints in a pseudo-REST way, a very-very simple authorization key management (just a toy!), some data query and routing, files upload/download. I used Oj fast JSON implementation!
.----------------------------------.
| |
| API Client (curl/webclient/etc.) |
| |
.----------------------------------.
| ^
| |
| 2: JSON data
1: HTTP requests
| | API SERVER
.--------------------------------------------------------------.
| | | |
| v | |
| .------------------------------. |
| | Sinatra API Server | |
| | | .--------------------. |
| | Controller/Router logic |<----| AUTH-KEY Archive: | |
| | | | .../db/app.keys | |
| | | .--------------------. |
| +------------------------------+ |
| | ActiveRecord ORM | |
| .---+--------------------------. |
| | |
| | .-------------------------------------------------. |
| +---| postgreSQL DB 1 (default/base) with tables: | |
| | | | |
| | | .------. .------. .--------. | |
| | | | Exam | | User | | Course | | |
| | | .------. .------. .--------. | |
| | .-------------------------------------------------. |
| | .-------------------------------------------------. |
| +---| postgreSQL DB 2 (external/remote) with tables: | |
| | | |
| | .------. | |
| | | Note | | |
| | .------. | |
| .-------------------------------------------------. |
.--------------------------------------------------------------.
Developement Tips & Tools:
- Command line Client tests: I used [curl](http://$ curl.haxx.se/docs/httpscripting.html) as default command line tool for client-side calls examples.
- Web client tests: I supplied a simple front-end web page using using jQuery AJAX.
To reproduce the Rails developer usual experience:
- I enjoyed use of shotgun to automatically reload rack development server (you can so edit source code on the fly, without restart server)!
- I enjoyed also very useful tux developement environment, a sortt of a la rails console substitute, I mainly used to browse ActiveRecord models and doying queries.
- I display record data set in pretty print data tables using Hirb gem.
In this demo application, I want connect with two already living databases (It's the case when you already have in production some legacy database and you want to access these data with an API server):
As prrof of concept, I used a real case scenario, where I have a default/primary postgreSQL database with name: esamiAnatomia_development, that contain three tables:
ExamUserCourse
The source code to connect the database is so simple as:
ActiveRecord::Base.establish_connection(ENV['ESAMIANATOMIA_DB_URL'] || \
'postgres://YOURUSERNAME:YOURPASSWORD@HOSTIPADDRESS/esamiAnatomia_development')
class Exam < ActiveRecord::Base
end
class User < ActiveRecord::Base
end
class Course < ActiveRecord::Base
endNow, let say I want sinatra server be able to connect also to a second different postgreSQL database named sar, containing the table:
Note
Here below the table columns details:
sudo -u db_username psql sarpsql (9.1.9)
Type "help" for help.
sar=# \d notes
Table "public.notes"
Column | Type | Modifiers
------------+-----------------------------+----------------------------------------------------
id | integer | not null default nextval('notes_id_seq'::regclass)
title | character varying(255) |
body | text |
created_at | timestamp without time zone |
updated_at | timestamp without time zone |
Indexes:
"notes_pkey" PRIMARY KEY, btree (id)
In that case, the Model is a class where I specify also some ActiveRecord validations:
class Note < ActiveRecord::Base
# connessione a specifico db
establish_connection(ENV['SAR_DB_URL'] || \
'postgres://YOURUSERNAME:YOURPASSWORD@HOSTIPADDRESS/sar')
# set del nome di una tabella, nel caso in cui non sia fatta con convenzione Rails
self.table_name = "notes"
# validazioni ActiveRecord
validates :title, presence: true, length: { minimum: 3 }
validates :body, presence: true
end -
git clone the source code from github
-
install all gems specified in Gemfile, with command:
$ bundle
-
Run the API server in a first terminal
-
set environment variables defining DB URI for both database instances:
$ export ESAMIANATOMIA_DB_URL=\ postgres://your-username:your-password@localhost/esamiAnatomia_development $ export SAR_DB_URL=\ postgres://your-username:your-password@localhost/sar
-
run the API SERVER daemon, by example in development env, with command:
$ shotgun -o localhost
-
-
Run the API CLIENT calls, using
curlin a second terminal (below some examples) -
you can monitor/debug ActiveRecord queries running
tuxin a third terminal
Here below I listed some examples of usage of client-side API calls, using $ curl command line utility.
$ curl localhost:9393/JSON 'pretty printed' reply (is sinatra server is running in developement):
{
"message": "Sinatra API Server Toolbox"
}JSON 'minified' reply (is sinatra server is running in production):
{"message":"Sinatra API Server Toolbox"}$ curl -X POST localhost:9393/login -d '{ "username":"admin", "password":"admin" }'JSON reply:
{
"message": "OK: login passed"
}The example here below show how to manage an Api-key (authorization token).
List of all items of model users, passing an invalid key (an UUID, by example):
$ curl -X GET http://localhost:9393/users -H "key: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"JSON reply:
{
"message": "sorry, you are not authorized."
}List of all items of model users, passing a valid key (let say again an UUID):
$ curl -X GET http://localhost:9393/users -H "key: yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"JSON reply:
[
{
"user": {
"created_at": "2013-08-27T07:44:48+02:00",
"crypted_password": "...",
"login": "Franco Paperone",
"role": "docente",
...
"updated_at": "2013-08-27T07:44:48+02:00"
}
},
{
"user": {
"created_at": "2013-08-27T07:44:48+02:00",
"crypted_password": "...",
"login": "admin",
"role": "admin",
...
"updated_at": "2013-09-25T22:25:38+02:00"
}
}
]$ curl -X POST http://localhost:9393/notes \
-d '{ "title":"prova", "body":"corpo del messaggio di prova!" }'JSON reply:
{
"note": {
"body": "corpo del messaggio di prova!",
"created_at": "2013-10-20T11:43:59+02:00",
"id": 1,
"title": "prova",
"updated_at": "2013-10-20T11:43:59+02:00"
}
}$ curl -X POST http://localhost:9393/notes -d '{ "title":"pr", "body":"" }'
JSON reply (in case of validation errors):
{
"title": [
"is too short (minimum is 3 characters)"
],
"body": [
"can't be blank"
]
}$ curl -i http://localhost:9393/notes/1JSON reply:
{
"note": {
"body": "corpo del messaggio di prova!",
"created_at": "2013-10-20T11:43:59+02:00",
"id": 1,
"title": "prova",
"updated_at": "2013-10-20T11:43:59+02:00"
}
}$ curl -X PUT http://localhost:9393/notes/1 -d '{ "title":"titolo modificato" }' JSON reply:
{
"note": {
"body": "corpo del messaggio di prova!",
"created_at": "2013-10-20T11:43:59+02:00",
"id": 1,
"title": "titolo modificato",
"updated_at": "2013-10-20T11:51:54+02:00"
}
}
$ curl -X DELETE http://localhost:9393/notes/1 $ curl http://localhost:9393/notes JSON reply:
[
{
"note": {
"body": "corpo del messaggio di prova 2!",
"created_at": "2013-10-20T11:55:33+02:00",
"id": 2,
"title": "prova 2",
"updated_at": "2013-10-20T11:55:33+02:00"
}
},
{
"note": {
"body": "corpo del messaggio di prova 3!",
"created_at": "2013-10-20T11:55:46+02:00",
"id": 3,
"title": "prova 3",
"updated_at": "2013-10-20T11:55:46+02:00"
}
}
]Get first page (0), assuming a page contain 10 items, from model Exam:
$ curl http://localhost:9393/exams/paginate/10/0JSON reply:
[
{
"exam": {
"id": 1,
"mail": "selmer.smith@ruelhauck.org",
"matricola": "974146488",
...
"votoanatomia": "",
"votoistologia": "24"
}
},
{
"exam": {
"id": 2,
"mail": "waldo@kub.com",
"matricola": "875086984",
...
"votoanatomia": "",
"votoistologia": ""
}
},
...
{
"exam": {
"id": 10,
"mail": "marilyne.waelchi@kemmer.name",
"matricola": "748212641",
...
"votoanatomia": "",
"votoistologia": "26"
}
}
]If you're using a Rack-based web framework (as Sinatra) you can very easily slim down your responses by compressing them. To achieve this you just have to use the Rack::Deflater middleware (see config.ru). Now any client that sets the Accept-Encoding: gzip,deflate header will benefit from smaller responses.
Let consider the GET /exams where the resource JSON representation amount 1650K bytes:
$ time curl http://192.168.1.33:9393/exams \
--output exams.jsonoutput show an elapsed time of 4.116 seconds:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1650k 100 1650k 0 0 427k 0 0:00:03 0:00:03 --:--:-- 427k
real 0m4.116s
If the client sets the Accept-Encoding: gzip,deflate header, so the rack server gzip slim down to 198K bytes:
$ time curl http://192.168.1.33:9393/exams \
--http1.0 -H "Accept-Encoding: gzip,deflate" \
--output exams.json.gzoutput show an elapsed time of 4.034 seconds:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 198k 0 198k 0 0 53339 0 --:--:-- 0:00:03 --:--:-- 53373
real 0m4.034s
Note: in that case (client and server in localhost) times are similar. Note that data compression (by server) and data decompression (by client) have a cpu cost and there is a time overhead (in comparison to the uncompressed way), But the network payload may decrease so much in case of compression! So the HTTP compression solution could be great in case of low badwith network connections.
Upload file file.txt and store the file in /public directory:
$ curl --upload-file file.txt localhost:9393/upload/Download file file.txt stored in /public directory (/public/file.txt):
$ curl localhost:9393/download/file.txtHere a web demo page: /public/webclient.html. The page allow to test some examples of API methods usage, using jQuery AJAX calls like this one:
$('#notes_post').click(function () {
$.ajax({
type: "POST",
data: JSON.stringify({ title:"nota bla", body: "blablablabla blablablabla" }),
dataType: "json",
context: document.body,
url: url + "/notes",
success:
function (data) { $('#reply_notes_post').html( JSON.stringify(data, null, 4)); }
});
});Here a screenshot of the "runned" webclient page:
run shotgun, basic:
$ shotgun -o localhost == Shotgun/Thin on http://localhost:9393/
setting environment excipitly:
$ shotgun -o localhost -E developmentspecifying environment, host and port, using rackup:
$ rackup -o localhost -p 9393 -E productiontux gem
act as rails console for a Sinatra application! Really useful to query database using ActiveRecord methods.
hirb gem
allow to show (ActiveRecord returned) record data set in pretty print data tables.
Here below some examples using tux interactive console.
Run the tux console from command prompt:
$ tuxUsing tux interactive console with Hirb:
Loading development environment (Rack 1.2)
>> require 'hirb'
=> true
>> Hirb.enable
=> true
>> Note
=> Note(id: integer, title: string, body: text, created_at: datetime, updated_at: datetime)
>> Note.all
+----+---------+---------------------------------+---------------------------+---------------------------+
| id | title | body | created_at | updated_at |
+----+---------+---------------------------------+---------------------------+---------------------------+
| 2 | prova 2 | corpo del messaggio di prova 2! | 2013-10-20 11:55:33 +0200 | 2013-10-20 11:55:33 +0200 |
| 3 | prova 3 | corpo del messaggio di prova 3! | 2013-10-20 11:55:46 +0200 | 2013-10-20 11:55:46 +0200 |
+----+---------+---------------------------------+---------------------------+---------------------------+
2 rows in set
true
>> Exam.select([:id, :cognomenome, :matricola, :updated_at]).order("updated_at DESC").limit(1)
+------+--------------------+-----------+---------------------------+
| id | cognomenome | matricola | updated_at |
+------+--------------------+-----------+---------------------------+
| 1643 | Aaliyah McCullough | 157560384 | 2013-09-25 22:25:38 +0200 |
+------+--------------------+-----------+---------------------------+
1 row in set
true
>> Exam.find_by_sql('SELECT id, cognomenome, matricola FROM exams ORDER BY updated_at DESC LIMIT 5')
+------+------------------------+-----------+
| id | cognomenome | matricola |
+------+------------------------+-----------+
| 1643 | Aaliyah McCullough | 157560384 |
| 1674 | Aaliyah Harvey PhD | 538093496 |
| 5000 | Marisa Gibson | 455246174 |
| 4999 | Wilhelmine Stoltenberg | 403257177 |
| 4998 | Hester Rogahn | 827819282 |
+------+------------------------+-----------+
5 rows in set
true
How to see a table row "vertically" (case of many columns), inside tux, using Hirb:
>> Hirb.enable :output => {"ActiveRecord::Base" => { :options => {:vertical => true}}}
>> Exam.find(11)
********************* 1. row *********************
id: 11
cognomenome: Mr. Oswaldo Willms
matricola: 513845379
votoistologia: 30L
dataistologia: 26-10-2008
votoanatomia:
dataanatomia:
luogonascita: Augustafurt
datanascita: 14-09-1981
cellulare: 539943941
mail: garrison_heel@mcglynngoodwin.org
note:
domande: D1:domanda nr. 1
D2:domandina facile
D3: domanda così così
corsolaurea: Igiene dentale
sede: Lake Leanne
annoaccademico: 2013
docente:
user: Filippo Adinolfi
created_at: 2013-08-27 07:45:16 +0200
updated_at: 2013-08-27 07:45:16 +0200
1 row in set
true
>>
- HTTP Compression: I added the trick using Rack::Deflate feature. I supplied cleient calls examples.
- JSON load/dump speed-up: I substituted JSON Ruby standard implementation with MultiJson gem and super-fast Oj gem.
- Comments translated in English. Better explanantion in README, inserted an image screenshot of Web Client side API calls example using jQuery AJAX.
- Improve authentication key db management.
- Insert an example of managing large amount of data with a super-fast in-memory NOSQL database as Redis!
- Better manage HTTP return codes
- Better manage error handling
- Exceptions handling lack at all.
- Iain Barnett for his answer to my stackoverflow.com question.
- Peter Ohler, for his superb gem Oj (fast JSON parser used in this project behind MultiJson).
Copyright (c) 2014 Giorgio Robino
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
e-mail: giorgio.robino@gmail.com