collective.jsonify
exports your Plone content to JSON.
Many packages that export data from Plone have complicated dependencies, and so
only work with Plone 3.0 or higher (or not even with 3.0).
collective.jsonify
's only dependency is simplejson. It can be installed in
any Plone version as far back as:
- Plone 2.1 (or probably even Plone 2.0, but not tested)
- Zope 2.6.4 (with CMF rather than Plone)
- Python 2.2
You can use the provided export script by adding it to an external method as described below.
The exported json is a collective.transmogrifier friendly format. Install
collective.jsonify
on a site you want to export from, and setup an import
transmogrifier pipeline on the site you're importing to, using the blueprints in
the collective.jsonmigrator package.
Alternatively you can use collective.exportimport to import that data as described in https://github.com/collective/collective.exportimport#migrate-very-old-plone-versions-with-data-created-by-collective-jsonify
Install collective.jsonify
for your Plone site, so that it is available in
your Plone site's PYTHONPATH
, including the simplejson
package. The
easiest way is to use buildout, as for any other modern Plone project. Other
options include:
- Play with PYTHONPATH manually.
- Use
easy_install collective.jsonify
orpip install collective.jsonify
which will also pullsimplejson
.
Note: if you are working with python 2.2, then you will need to install a `tweaked branch of simplejson <https://github.com/simplejson/simplejson/tree/python2.2>`_.
Then run your Zope instance, go to the Zope root and create the necessary External Methods.
External method for exporting JSON files to the filesystem:
- export_content: - id:
export_content
- module name:collective.jsonify.json_methods
- function name:export_content
External methods for remote access from the importing Plone instance, using
collective.jsonmigrator
:
- get_item - id:
get_item
- module name:collective.jsonify.json_methods
- function name:get_item
- get_children: - id:
get_children
- module name:collective.jsonify.json_methods
- function name:get_children
- get_catalog_results: - id:
get_catalog_results
- module name:collective.jsonify.json_methods
- function name:get_catalog_results
It's true that External Methods are not the nicest to work with and using them
makes the setup a little long. But the nice thing about External Methods is that
they work in Plone 1.0 as well as in Plone 4.0, so you could potentially use
collective.jsonify
to migrate from very old Plone versions.
collective.jsonify
is intended to be used in conjunction with
collective.jsonmigrator
. There you can find an example transmogrifier
pipeline that connects to the Plone site running collective.jsonify
, crawls
it, extracts the content and imports it into the target site.
To see what collective.jsonmigrator
is actually seeing you can issue "json
views" on content you want to explore:
http://localhost:8080/Plone/front-page/get_item http://localhost:8080/Plone/front-page/get_children
The first gets all content out of front-page
; the second lists all content
contained inside this object and returns their ids.
Finally, you can use get_catalog_results
to catalog query results as a list
of paths. To use it, you need to hand your query as a base64'ed Python dict
string. Here's an example of doing this with curl:
curl --data catalog_query=$(echo '{"Type": "Slide"}' | base64 -w0) \ 'http://localhost:8080/Plone/portal_catalog/get_catalog_results
To export your site to a directory of JSON files use the collective.jsonify.export.export_content
function.
This is how to use it with an external method:
Install
collective.jsonify
for your instance.Create a script in your Zope instance
Extensions
directory, e.g. inBUILDOUT_ROOT/parts/instance/Extensions
. Create theExtensions
directory, if it doesn't exist. Create a filejson_methods.py
with the following contents - adapt for your needs:from collective.jsonify.export import export_content as export_content_orig def export_content(self): return export_content_orig( self, basedir='/tmp', # export directory extra_skip_classname=['ATTopic'], # batch_start=5000, # batch_size=5000, # batch_previous_path='/Plone/last/exported/path' # optional, but saves more memory because no item has to be jsonified before continuing... )
Create the "External Method" in the ZMI at the Zope root or Plone root. id: "export_content" module name: "json_methods" function name: "export_content"
For more info on "External Methods" see: https://zope.readthedocs.io/en/latest/zopebook/ScriptingZope.html#using-external-methods
- To start the export, open the url in your browser::
We try to cover the basic Plone types to export useful content out of Plone. We cannot predict all usecases, but if you have custom requirements it's easy to extend functionality.
To control what to export before serialzing is the fastest way to extend..
You can override the export_content
"External Methods" as described above.
Example:
from collective.jsonify.export import export_content as export_content_orig EXPORTED_TYPES = [ "Folder", "Document", "News Item", "Event", "Link", "Topic", "File", "Image", "RichTopic", ] EXTRA_SKIP_PATHS = [ "/Plone/archiv/", "/Plone/do-not-import/", ] # Path from which to continue the export. # The export walks the whole site respecting the order. # It will ignore everything untill this path is reached. PREVIOUS = "" def export_content(self): return export_content_orig( self, basedir="/var/lib/zope/json", skip_callback=skip_item, extra_skip_classname=[], extra_skip_id=[], extra_skip_paths=EXTRA_SKIP_PATHS, batch_start=0, batch_size=10000, batch_previous_path=PREVIOUS or None, ) def skip_item(item): """Return True if the item should be skipped""" portal_type = getattr(item, "portal_type", None) if portal_type not in EXPORTED_TYPES: return True
You can modify or extend the exported data by passing additional wrappers to get_item
or export_content
.
These should to be "External Methods":
http://localhost:8080/Plone/front-page/get_item?additional_wrappers=extend_item
These hooks take the object and the serialized data as arguments.
Example:
def extend_item(obj, item): """Extend to work better well with collective.exportimport""" from Acquisition import aq_parent # Info about parent parent = aq_parent(obj) item["parent"] = { "@id": parent.absolute_url(), "@type": getattr(parent, "portal_type", None), } if getattr(parent.aq_base, "UID", None) is not None: item["parent"]["UID"] = parent.UID() # Review state try: review_state = obj.portal_workflow.getInfoFor(obj, "review_state") except Exception, e: review_state = None item["review_state"] = review_state # Block inheritance of local roles item["_ac_local_roles_block"] = getattr(obj.aq_base, "__ac_local_roles_block__", False) return item