Tools to play with json-schemas defined APIs.
These tools are based on json-schema draft 3 from http://tools.ietf.org/html/draft-zyp-json-schema-03 Not all features of the schema are supported and probably won't be. Handling of not supported feature varies between the different tools.
All these tools are proofs of concept and work in progress, they need more extensive testing and documentation.
Class to generate random values given a json-schema.
Doesn't support all json-schema monstruousities, only a subset I find useful.
See TODO.md for what is likely to be implemented next.
from datagenerator import DataGenerator
generator = DataGenerator()Generate random values of each basic type using
>>> generator.random_value("string")
'Olzq3LV'
>>> generator.random_value("number")
-6.675904074356879
>>> generator.random_value("integer")
30
>>> generator.random_value("boolean")
Truenumber
>>> generator.random_value({"type":"number", "minimum":30})
32.34295327292445
>>> generator.random_value({"type":"number", "maximum":30})
-35.80704939879546
>>> generator.random_value({"type":"number", "maximum":30, "minimum":12})
16.45747265846327integer supports minimum and maximum like number and more
>>> generator.random_value({"type":"integer", "maximum":30, "divisibleBy":4, "minimum":12})
24
>>> generator.random_value({"type":"integer", "maximum":30, "exclusiveMaximum":True, "minimum":28})
29(same for exclusiveMinimum)
string supports minLength, maxLength, pattern (ignores minLength and maxLength if pattern is used)
>>> generator.random_value({"type":"string", "maxLength":20, "minLength":15})
'VytPCEdAImX11188HU'
>>> generator.random_value({"type":"string", "pattern":"[0-9]{3}[a-zA-Z]{2,5}"})
u'806FoNP'boolean doesn't have any constraints.
Without constraints the array size will be picked the same way as a random integer.
Each item in the array is generated using the default generator for the type given in items.
>>> generator.random_value({"type":"array", "items": {"type":"string"}})
[
'39yxcpvS5tfPf6O',
'sNDk7SlGNQstxxx',
'nPcRSD9yIP7j ',
'PWP7KQfjc1',
'tt6F6Z2YEp'
]minItems, maxItems and uniqueItems are supported
The type of object in items can be anything that the generator knows about, either one of the basic types
or a user defined one available from the generator's schemas store.
from schemasstore import SchemasStore
...
>>> from schemasstore import SchemasStore
>>> store = SchemasStore()
>>> generator.schemas_store = store
>>> store.add_schema({"type":"integer", "name":"small_integer", "minimum":0,"maximum":9})
True
>>> generator.random_value({"type":"array", "uniqueItems":True, "minItems":10, "items":{"type":"small_integer"}})
[0, 7, 2, 5, 3, 6, 1, 4, 8, 9]See datagenerator for other examples.
Objects can be generated the same way as the other types.
Example generating search_result.json
>>> store.load_folder("data/schemas/")
>>> generator.random_value("search_result")
{u'price': 21.980325774975253, u'name': 'wdvfXYrrt', u'reference': 26}Generating arrays of objects is fine as well
>>> generator.random_value({"type":"array", "maxItems":3, "minItems":2, "items":{"type":"search_result"}})
[
{u'price': 20.304440535786522, u'name': 'VUIgjaPbs', u'reference': 40},
{u'price': 28.45387747055259, u'name': 'JTycBU1V78X1S', u'reference': 27}
]Or generating objects with arrays of other objects in them, see search_resuts with an array of search_result
>>> generator.random_value("search_results")
{
u'total_results': 41,
u'total_pages': 26,
u'current_page': 33,
u'items_per_page': 27,
u'results': [
{u'price': 26.218704680177446, u'name': 'B4p1Z1pOFQO', u'reference': 38},
{u'price': 21.205089550441276, u'name': 'FQPHdLds', u'reference': 7},
{u'price': 20.610536930894398, u'name': '8D862p1XVupP', u'reference': 38},
{u'price': 9.543934434058526, u'name': 'PmqBA0e DIWisf', u'reference': 32}
]
}Why not generate random schemas?
>>> r_schema = generator.random_schema()
>>> r_schema
{
'type': 'object',
'properties': {
u'viYXjhu': {'required': False, 'type': 'boolean'},
u'TO': {'required': False, 'type': 'string'},
u'NTSd': {'required': False, 'type': 'string'},
u'WjaL': {'required': False, 'type': 'string'},
u'PtvhZ': {'required': False, 'type': 'boolean'}
},
'name': u'zJllGkKosmocOVO'
}And then generate an array of random values of it
>>> store.add_schema(r_schema)
True
>>> generator.random_value({"type":"array", "minItems":1, "maxItems":3, "items":{"type":"zJllGkKosmocOVO"}})
[
{u'TO': 'jamKFpdwY'},
{u'WjaL': '8LnibWUdsSI', u'PtvhZ': True},
{}
]All the values are generated using the random module, so please don't use the generate values for anything
requiring reliable randomness == don't use it to generate passwords.
To generate the data, the generator has to limit the range of possible values, so the values generated don't
vary too wildly. The ranges are controlled by variables in DataGenerator. Feel free to tweak them, especially
if you need values that don't fall into those ranges without having to set both minimum and maximum on your
properties.
Class to generate links defined in the links section of a json-schema.
Generate links from book.json
Input
...
"isbn" : {
"type":"string",
"required":true,
"pattern":"^\\d{12}(\\d|X)$"
}
},
"links" : [
{
"rel":"self",
"href":"books/{isbn}"
},
{
"rel":"instances",
"href": "books"
}
]
...Output
{
u'instances': [u'books'],
u'self' : [u'books/525259838909X']
}{isbn} got replaced by a random value 525259838909X satisfying the constraints on isbn (matches the regex).
Class to generate invalid data for a given schema
Basically does the opposite of datagenerator. WIP, needs documentation and examples.
Base class to generate models from a schema, nothing too visible on its own, check resourceserver.
Generate SQLAlchemy models to be used with flask-sqlalchemy from a schema. Uses modelgenerator.
Used in resourceserver to store and query items.
Generate models and collections for Backbone.js from a schema.
The models generated use the primary key defined in the rel=self link or id by default.
To be able to use collections, make sure your schema has a rel=instances link or fetch won't work.
$ python backbonemodelgenerator.py -h
Usage: backbonemodelgenerator.py jsonfile1 [jsonfile2]...
Options:
-h, --help show this help message and exit
-t OUTPUT_TYPE, --type=OUTPUT_TYPE
Output type (js|wrapped|html)Outputs only the js code for the models/collections
$ python backbonemodelgenerator.py -t js data/schemas/message.json
App.Models.Message = Backbone.Model.extend({
urlRoot: '/messages',
idAttribute: 'id'
});
App.Collections.Messages = Backbone.Collection.extend({
model : App.Models.Message,
url : "/messages"
});Wraps the js code into $(document).ready()
$ python backbonemodelgenerator.py -t wrapped data/schemas/message.json
$(document).ready(function() {
window.App = { Models : {}, Collections : {} };
App.Models.Message = Backbone.Model.extend({
urlRoot: '/messages',
idAttribute: 'id'
});
App.Collections.Messages = Backbone.Collection.extend({
model : App.Models.Message,
url : "/messages"
});
});Same as wrapped but generate a whole html page including jQuery, Backbone and Underscore to easily test.
You can use it with resource server for example
$ mkdir static
$ python backbonemodelgenerator.py -t html data/schemas/message.json > static/index.html
$ python resourceserver.py data/schemas/message.json
Added message
* Running on http://0.0.0.0:5000/Now open your browser at http://0.0.0.0:5000/static/index.html Open your js console to start playing
var col = new App.Collections.Messages()
col.fetch()You should see backbone talking to the resource server in the server shell
127.0.0.1 - - [20/Nov/2012 01:17:15] "GET /messages HTTP/1.1" 200 -You can inspect the results using
col.modelsUsing fetch() only works if your schema includes a link with rel=instances
var msg = new App.Models.Message({recipient:"01234567890", text:"test message"})
msg.attributesAt that point the message is not saved yet, you can verify by using
msg.isNew()You can save it on the server using
msg.save()You can verify that the message was sent to the server in the server shell
127.0.0.1 - - [20/Nov/2012 01:23:24] "POST /messages HTTP/1.1" 201 -Now you should have an id for the message and it shouldn't be marked as new anymore.
msg.id
msg.isNew()Create a message with the id of the message to fetch
var msg = new App.Models.Message({id: 3})The message is not marked as new as it has an id.
We can then fetch the actual message from the server using
msg.fetch()
msg.attributes()You can see the query in the server shell again
127.0.0.1 - - [20/Nov/2012 01:25:41] "PUT /messages/3 HTTP/1.1" 200 -Once you have a message object, you can update it using save.
> msg.attributes.recipient
"01234567890"
> msg.save({recipient:"00123456789"})
> msg.attributes.recipient
"00123456789"This is done by doing a PUT on the server
127.0.0.1 - - [20/Nov/2012 01:33:35] "PUT /messages/3 HTTP/1.1" 200 -Simply use destroy on the object
msg.destroy()And see the DELETE happening on the server
127.0.0.1 - - [20/Nov/2012 01:34:48] "DELETE /messages/3 HTTP/1.1" 204 -Class to implement the REST api of resources defined in a schema.
Supports creation, update, retrieval, deletion, listing of instances and schema.
Run the server using
$ python resourceserver.py [jsonfile1, jsonfile2, ...]$ python resourceserver.py data/schemas/message.json
Added message
* Running on http://0.0.0.0:5000/$ curl -i -X POST http://0.0.0.0:5000/messages -d "recipient=07771818335&text=nice message"
$ curl -i -X POST http://0.0.0.0:5000/messages -d '{"recipient":"01234567890", "text":"test"}' \
-H "Content-Type: application/json"
HTTP/1.0 201 CREATED
Content-Type: application/json
Content-Length: 13
Location: http://0.0.0.0:5000/messages/2
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:28:56 GMT
{
"id": 2
}$ curl -i -X GET http://0.0.0.0:5000/messages
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 126
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:32:09 GMT
[
{"text": "I </3 ninjas", "recipient": "07771818337", "id": 1},
{"text": "nice message", "recipient": "07771818335", "id": 2}
]$ curl -i -X GET http://0.0.0.0:5000/messages/2
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 71
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:35:42 GMT
{
"text": "nice message",
"recipient": "07771818335",
"id": 2
}$ curl -i -X OPTIONS http://0.0.0.0:5000/messages/2
HTTP/1.0 200 OK
Content-Type: application/json
Content-Length: 590
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:37:06 GMT
{
"description": "Simple message structure",
"type": "object",
"properties": {
"text": {
"required": true,
"type": "string",
"maxLength": 140
},
"recipient": {
"pattern": "0[0-9]{10}",
"required": true,
"type": "string"
},
"id": {
"minimum": 0,
"type": "integer"
}
},
"links": [
{
"href": "/messages",
"rel": "root"
},
{
"href": "{id}",
"rel": "self"
},
{
"rel": "instances"
},
{
"rel": "create"
}
],
"name": "message"
}Supports partial updates
$ curl -i -X PUT http://0.0.0.0:5000/messages/2 -d 'recipient=07771818336'
$ curl -i -X PUT http://0.0.0.0:5000/messages/1 -d '{"text":"foo"}' \
-H "Content-Type: application/json"
HTTP/1.0 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 0
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:38:02 GMT$ curl -i -X DELETE http://0.0.0.0:5000/messages/2
HTTP/1.0 204 NO CONTENT
Content-Type: text/html; charset=utf-8
Content-Length: 0
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:38:38 GMTThe message.json doesn't define an explicit primary key, but defines id as the key in the rel=self link.
Each message then gets an additional id key managed by the server.
Trying to set or update the id results in errors
$ curl -i -X POST http://0.0.0.0:5000/messages -d "recipient=07771818335&text=nice message&id=7"
$ curl -i -X PUT http://0.0.0.0:5000/messages/1 -d "recipient=07771818335&text=nice message&id=3"
HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 43
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:43:48 GMT
{
"error": "id is read only in message"
}$ curl -i -X POST http://0.0.0.0:5000/messages -d "recipient=07771818335&tet=test&haxxy=foo"
$ curl -i -X PUT http://0.0.0.0:5000/messages/1 -d "haxxy=foo"
HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 57
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 19:56:19 GMT
{
"error": "message does not have a 'haxxy' property"
}$ curl -i -X PUT http://0.0.0.0:5000/messages/1 -d "recipient=0notanumber&text=nice message"
$ curl -i -X POST http://0.0.0.0:5000/messages -d "recipient=0notanumber"
HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 86
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 20:03:34 GMT
{
"error": "'0notanumber' is an invalid recipient value: must match u'0[0-9]{10}'"
}$ curl -i -X POST http://0.0.0.0:5000/messages -d "recipient=012345678901"HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 44
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Sun, 18 Nov 2012 20:06:00 GMT
{
"error": "text is required in message"
}$ curl -i -X POST http://0.0.0.0:5000/messages -d '{"recipient":"01234567890", "text":"test}' -H "Content-Type: application/json"
HTTP/1.0 400 BAD REQUEST
Content-Type: application/json
Content-Length: 90
Server: Werkzeug/0.8.3 Python/2.7.3
Date: Tue, 20 Nov 2012 00:23:05 GMT
{
"error": "Invalid data: Unterminated string starting at: line 1 column 35 (char 35)"
}Each model needs a primary key. There are 3 ways to define the primary key of the model:
If there is no rel=self link, an additional id (or appended with as many _ as
necessary to make the name unique) attribute is created. This type of key is called implicit and can
only be set by the server (read only).
If there is a rel=self link and it contains a {variable} part, the variable name is used as the primary key.
- If
variableis the name of an existing property, this property is used as the primary key, and can be updated ( explicit key ) - Otherwise an implicit key is created using the
variablename (stil read-only).
This schema uses isbn as the explicit key. Instances can be created using a specific isbn, and its value
can be updated.
...
"isbn" : {
"type":"string",
"required":true,
"pattern":"^\\d{12}[\\d|X]$"
}
},
"links" : [
{
"rel":"self",
"href":"books/{isbn}"
},
...This schema defines an implicit key order_id (assuming no property is called order_id).
...
"links" : [
{
"rel":"self",
"href":"{order_id}"
},
...Use rstr hosted in a mercurial repo on bitbucket. Run init.sh in dependencies to fetch it.
If you don't have mercurial, apt-get install mercurial should help.
flask-sqlalchemy is required, use flasksqlalchemy-requirements.txt with virtualenv
jinja2 is required, comes with flask if you use the flasksqlalchemy-requirements.txt