Configuration

Directory layout

All configuration and template information for the registry is stored (by default) in the directory /opt/ldregistry.

Runtime information such as the internal triple store and record of uploaded files are in /var/opt/ldregistry.

Logging information is in /var/log. Most logging information recorded as part of the web app container and so appears in /var/log/tomcat7/catalina.out. There is a separate log file for the velocity template engine which appears in /var/log/ldregistry-velocity.log.

Note that this is a change from the proof-of-concept implementation which used /var/local/registry for everything. That does not follow linux file system layout conventions. Arguably the configuation information should be accessible in /etc/ldregistry, a deployment could use symbolic links to achive this. The use of ldregistry rather than registry is a prefix is to disambiguate against other registry-like software.

When a registry instance first starts up then each area of the configuration directory will be initialized if it does not exist. The only exception to this is that, for technical reasons with the Tomcat context definition, an empty ui/ directory should exist for successful start-up.

The default layout of /opt/ldregistry is summarized below:

Directory or file | Purpose ----|------|---- boot/ | Definitions of bootstrap registers used when starting a registry with no data. boot/system | The bootstrap definition of the system registers. config/ | Configuration and initialization files. config/root-register.ttl | RDF file describing the root register, used during bootstrapping only. config/app.conf | Configuration parameters for the software, defines the set of components that make up the registry and provides configuration parameters for them. config/user.ini | Initialization for the security database, provides OpenIDs and grant information for initial users. Not consulted once the userstore database exists. config/oauth.conf | Configuration of OpenID Connect provider proxy-conf.sh | Script used to trigger the nginx proxy to reconsult configuration information, should be set to be sudo executable by the tomcat user. templates/ | Set of templates defining the built in UI, written in Apache Velocity. ui/ | Resources (CSS, image and javascript files) which should be served from the /ui path of the registry URL.

The default layout of /var/opt/ldregistry is summarized below:

Directory or file | Purpose ----|------|---- index/ | Lucene free text index for the registry content. logstore/ | File copies of all uploaded registration and update payloads. store/ | The RDF data store (an Apache Jena TDB instance). userstore/ | Database holding security information (registrations and grants), an embedded instance of Apache Derby. proxy-registry.conf | Proxy configuration script generated by the registry. Other scripts with the naming pattern proxy-*.conf may be added here to support additional manual proxy rules.

Security configuration

Once the registry is running all security information is held in the database in the userstore directory. Configuration of user permissions is then done through the user interface.

If no userstore database exists then it will be initialized by loading the initial user information from config/user.ini. If no such configuration file exists then a null one will be created.

To set up an administrator for a new installation then the bootstrap userstore should be removed and the bootstrap config/user.ini should be edited to define the administrator.

The user.ini file comprises a set of declarations, one per line (lines beginning with # are comment lines).

User registration entries start with the user keyword and take the form:

user  id  "name"

For example:

user https://me.yahoo.com/a/rb93y8N72dK9HoXHwgNqBKlikg-- "Alice"

Each OpenID or OpenID Connect provider has a different means of obtaining a user-specific OpenID. For example Yahoo! allows creation of an OpenID through http://openid.yahoo.com/ from which a user can determine (or create) an OpenID identity. A variant on this declaration is to specify a password for the user as a final entry on the user line:

user dave@epimorphics.cm "Dave" foobar

In this case the user identity can be any sufficiently unique string, such as an email address, and need not be an OpenID URL (though it can be). N.B. This password in clear text and so should be used for bootstrap only and changed as part of system commissioning.

There is a built in anonymous user with pseudo OpenID of http://localhost/anon. It is convenient to declare that user in the initialization file as well, so as to bind a visible name to that id. For example:

user http://localhost/anon "Any authenticated"

The second type of declaration line is used to grant permissions to a user. These take the form:

openid permission

For example:

https://myopenid      Update:/register/item
https://myotheropenid GrantAdmin

In particular the GrantAdmin permission makes the user an administrator.

Thus a minimal user.ini configuration looks something like:

user https://profiles.google.com/1147194443288764760228 "Alice" changeme
user http://localhost/anon "Any authenticated"
https://profiles.google.com/1147194443288764760228 GrantAdmin

OAuth configuration

To use OpenID Connect the registry instance must be configured with ID credentials for the chosen provider. This is done through the /opt/ldregistry/config/oauth.conf file which takes the form:

ldregistry.client-id = 
ldregistry.client-secret = 
ldregistry.usehttps = true

The usehttps field forces redirection to https for requests after successful authentication and back to https after logout when https is not being used by default for all requests.

Obtaining client credentials is provider specific.

To use Google OAuth and identities provided by Google Plus then the steps are:

Open [https://console.developers.google.com]
Create a "project"
Select APIs and Auth and enable the "Google+" API.
Configure the "consent screen", which determines what users will be told about the registry application when granting permission to use their identity
Under credentials use Create new Client ID. Select Web Application and provide the URIs which your registry instance will be based at (including localhost in these can be useful for development purposes). For the redirect URIs then append /system/security/responseoa to the base URI for your instance, for example http://location.data.gov.uk/registry/system/security/responseoa
Copy the generated client id and client secret to the oauth.conf file

An example Credentials configuration is:

Client ID for web application

Client ID            ...
Email address        ...
Client secret        ...

Redirect URIs        http://localhost:8080/registry/system/security/responseoa
                     http://environment.data.gov.uk/registry/system/security/responseoa

Javascript Origins   http://localhost
                     http://localhost:8080
                     http://environment.data.gov.uk

Bootstrap registers

When a registry instance runs for the first time it needs to create an initial RDF description. There are two parts to this bootstrapping - the root register and all other registers (including the system registers and manager-supplied initial data).

Root register

The root register is a special case because of its different naming convention and because there is no register into which to add it. The root register is bootstrapped by loading an initial RDF file (/config/root-register.ttl) which must contain both the reg:Register declaration and an associated reg:RegisterItem. Each of these must meet minimum constraints for the versioning model (have an owl:versionInfo value and a version:currentVersion link, which can point to itself). The URI of the root register must match the declared base URI for the registry installation (see below).

System and other initial registers

Once the root register is in place then the normal API mechanisms for loading register declarations and registered items can function. In particular use of "relative" addresses is then possible so any such bootstrap files can be independent of the registry base URI.

The additional initial registers are loaded from the area boot/. Each directory in the boot area corresponds to a register and should contain a metadata.ttl defining it. It may also contain sub-directories for sub-registers and other *.ttl files defining items to be included in the register.

Initialization

When a registry first starts up, if the bootstrap files for either of these kinds of register do not exist then a default minimum set will be created. Then if the RDF data store does not exist (store/ and the associated free index in index/) then one will be created and initialized from the bootstrap files.

Thus if you start a blank registry installation that will create both a minimal set of bootstrap files and an initialized store. Normally you will then need to halt the registry, edit the bootstrap files to suit the deployment requirements and delete the initial store/ and index/ areas. Restarting the registry will then rebuild the store based on your customized bootstrap.

Base URI

The registry will create and serve all data relative to an assumed base URI. This needed not be the same as the URL at which the registry installation is hosted - either because this is a staging deployment or because some external proxy will map requests from the real URI to the hosting system.

The base URI can only be set at the time that a registry instance initially starts up. Once the initial store has been created all data in that store will hold absolute URIs which are based on the configured base URI.

There are two places where the base URI must be configured.

Firstly, the registry software itself is configured from the file config/app.conf, see below. In that file change the following line to give a suitable base URI:

registry.baseURI     = http://ukgovld-registry.dnsalias.net/

Secondly, the initial bootstrap RDF for the root register must be consistent with this. This is defined in config/root-register.ttl (see above).

General configuration

The registry software itself is configured through a single file config/app.conf. The file is used to configure a series of software components which together make up the registry. It is possible to provide alternative implementations of some of these components and configure them through this file. The configuration is a series of blocks, one for each component, which take the form:

component       = qualified.name.of.class
component.prop1 = value1
...
component.propN = valueN

In most cases the name of the component is arbitrary. The value may be a string, a long integer, a comma separated array of values or a reference to an earlier component (using the syntax $componentname).

The parameter string {webapp} can be used in configuration values and will be expanded with the location of the running registry application.

In normal use very few of these parameters should need customization, so we will only summarize the key ones.

Note. Prior to release 0.1.1 configuration was through config/services.conf and there are some syntax changes between the two files. These include use of $ for cross referencing components, use of {webapp} instead fo ${webwapp} to reference to files within the war and some differences in the specific java components used. In most cases porting from a prior release to the current release will simply mean taking the sample from src/main/webapp/WEB-INF/app.conf, customising the registry.baseURI setting and deploying this to /opt/ldregistry/config/app.conf.

registry

The registry service itself.

Key property	Meaning
baseURI	The base URI at which the registry is assumed to operate.
pageSize	The number of items that should be returned by list views of collections (i.e. registers).
log	Area used for recording registration/update payloads.
bootSpec	A sequence of possible locations for the bootstrap root register, tried in order.
systemBoot	The location of the boot areas for loading system and other registers.
store, forwarder, userStore	References to the other required components
backupDir	Directory where snapshot backups of the repository dataset will be saved
configExtensions	Optional reference to a set of general configuration options used to customize UI behaviour

velocity

The template rendering system used the generate the default UI.

Key property	Meaning
production	If `true` then run in production mode where macros cannot be updated and templates, once cached in memory, are assumed not to change. If `false` then any change to a template will be detected and cause a change to the UI.
templates	Directory where the template files are located.

basestore

The RDF data store for the registry.

Key property	Meaning
location	The directory where the RDF database is held.
textIndex	Comma-separated list of predicates to include in the text index, can use prefixes from the default prefix set (rdf, rdfs, owl, xsd, dct, foaf, skos, ldp, void, prov, time, reg).
timeout	A global timeout for queries to the store. Set to "first,all" where first is the time until first result and all is the total query time allowed, both in ms. Should be set high enough to allow normal operation but can be useful to block roque ad hoc query requests.

index

The (Lucene) free text index.

Key property	Meaning
location	The directory where the text index is held.

forwarder

Component used to manage forwarding/proxy configurations. This will handle redirects itself and will implement any proxy forwarding by dynamically configuring a separate nginx instance.

Key property	Meaning
proxyConfDir	Directory where nginx proxy scripts will be written.
proxyRestartScript	Shell script which will be used to cause nginx to reconsult the configuration scripts.

General Config

Essentially a hashtable containing abitrary key/value pairs used to control external UI behaviour. It is possible to extend UI behaviour (though modification of the UI Velocity templates) and control those extensions by additions to this block. Additional configuration options can be added here without any change in the registry-core.war, they are simply passed through to the Velocity engine. The options supported by the current templates in registry-config-base are:

Property	Meaning
suppressPasswordLogin	If set to `true` this removes the option to login or register through a password, leaving only the option to register via OAuth/OpenID. If set true this also enables an admin menu item to generate temporary API passwords
suppressLinkView	By default the item display shows two columns of information - item property/values and cross-item links. If this config property is set to `true` the links column is suppressed and the property/value table spans the whole display area.
showRegisterAsDatatable	If set to true then the display of register contents is rendered using a client-side datatable component that supports paging, search and reordering. This is purely client side so to be effective the `pageSize` property of the registry should be set large enough that most registers will fit within one page
registryName	Provides a name for the registry that will be used in page titles. This avoids having to adapt the `main.vm` and `header.vm` templates when implementing a new registry instance.

Logging

Logging of accesses to the registry is done by the nginx front end proxy. See the nginx configuration documentation.

In addition all requests that reach the registry itself will be logged. For requests from authenticated users the user name and their OpenID will be included, for other requests the originating IP address is (should be, when that's fixed!) recorded. This log will also include messages from internal software components. The registry logging is configured by log4j.properties within the registry application itself. Currently this will log to file (log/registry.log) and to standard output (which means it will be captured by the velocity logging).

UI templates

The html styling for an registry pages requested in text/html format is handling by the velocity templating system. The builtin registry UI is implemented using these templates. The templates are all stored in the templates/ directory within the operating area. As usual, if this area does not exist at registry start up time then one will be created containing the default templates.

The key templates for customization are:

Template	Use
`main.vm`	The entry point, the registry will invoke main.vm in response to any request to a registry resource in text/html format.
`header.vm`	Included in all top level templates to provide the HTML header information including page title and loading of style sheets.
`navbar.vm`	Included in all top level templates to render the top level navigation bar.
`footer.vm`	Included in all top level templates to provide the running foot, load all required javascript libraries and close the HTML page.
`root.vm`	Called by the main.vm to render the home page of the registry (root register).

In addition to the calls of main.vm from the registry itself, requests to the /ui area of the registry are handled specially. They can either directly invoke a UI template or are served as files to provide the style sheets, images and javascript libraries needed for the UI.

Request URI	Action
`/ui/template`	If a template called `template.vm` exists then it will be used to render the response, examples include `about.vm` and `login.vm`.
`/ui/*`	All other requests fall through to the underlying Tomcat engine, which in turn serves these files from the `ui/` directory in the operating area.

Configuring the document base from which Tomcat will serve the plain UI files is currently done by setting an alias in a META-INF/context.xml file in the web app itself. The configuration of aliases is Tomcat specific so that switching to a different application server would require some minor modifications to the web app.

Template bindings

When a velocity template is invoked the following environment variables will be bound

Variable	Bound to	Instance of
$uri	The relative path of the resource being requested	String
$registry	The Registry object, access configuration info and invoke API operations from here	com.epimorphics.registry.core.Registry
$subject	The security subject, can test if the user is authenticated and check permissions	org.apache.shiro.subject.Subject
$requestor	Name of the person making the current request	String
$request	The incoming HTTP request	javax.servlet.http.HttpServletRequest
$language	The language code for the user's locale	String

Provide feedback

Saved searches

Use saved searches to filter your results more quickly