Copyright (C) 2019-2020 The Open Library Foundation
This software is distributed under the terms of the Apache License, Version 2.0. See the file "LICENSE" for more information.
The purpose of this edge API is to bridge the gap between self service circulation and patron services stations and FOLIO by allowing these systems to issue requests and receive responses in Standard Interchange Protocol v2 (SIP2). These requests will be serviced by FOLIO APIs and the responses will be based on FOLIO results for all supported (TBD) SIP2 commands.
The edge-sip2 module can be launched via edge-sip2-fat.jar
as follows:
$ java -jar edge-sip2-fat.jar -conf sip2.conf
The -conf option can either specify the filename of the configuration or inline JSON. Here is a sample sip2.conf file:
{
"port": 6443,
"okapiUrl": "https://folio-testing-okapi.dev.folio.org",
"tenantConfigRetrieverOptions": {
"scanPeriod": 300000,
"stores": [{
"type": "file",
"format": "json",
"config": {
"path": "sip2-tenants.conf"
},
"optional": false
}]
}
}
For inline JSON, the format is:
-conf '{"port":1234,"okapiUrl":"https://folio-snapshot-okapi.dev.folio.org".....}'
On Windows, inline JSON configuration is in double quotes and the inner double quotes should be escaped, for example:
-conf "{\"port\":1234,\"okapiUrl\":\"https://folio-snapshot-okapi.dev.folio.org\"....}"
One option is to mount the configuration files to the Docker container and provide
command line arguments to point it to the right path (e.g. -conf /path/to/config
).
Config option | Type | Description |
---|---|---|
port |
int | The port the module will use to bind, typically 1024 < port < 65,535. |
okapiUrl |
string | The URL of the Okapi server used by FOLIO. |
tenantConfigRetrieverOptions |
JSON object | Location for tenant configuration. |
scanPeriod |
int | Frequency in msec that sip2 will check for and reload tenant configuration changes. |
stores |
JSON array | Defines the properties for the tenant configuration stores. Multiple sources of tenant configuration can be loaded and combined together. |
type |
string | The store type. Several supported types include: file, http, github, s3. See: vertx config |
format |
string | Sip2 expects configuration to be in json format. |
config |
string | Store type-specific properties. |
path |
string | Path name of the tenant configuration file for file type stores. |
optional |
boolean | If a failure is caught while loading the tenant configuration from an optional store, the failure is logged, but the processing does not fail. Instead, the tenant configuration will be empty. |
netServerOptions |
JSON object | Configuration options for the server. These are Vertx options and are numerous. See: NetServerOptions. |
Note: edge-sip2 now requires two config files: the main bootstrap sip2.conf and tenant configuration: sip2-tenants.conf. The additional config file is required to support multi-tenants and runtime reloading of tenant configuration without restarting the edge-sip2 module.
Here is a sample sip2-tenants.conf file:
{
"scTenants": [
{
"scSubnet": "11.11.00.00/16",
"tenant": "test_tenant1",
"errorDetectionEnabled": true,
"messageDelimiter": "\r",
"fieldDelimiter": "\|",
"charset": "ISO-8859-1"
},
{
"scSubnet": "22.22.00.00/16",
"tenant": "test_tenant2",
"errorDetectionEnabled": true,
"messageDelimiter": "\r",
"fieldDelimiter": "\|",
"charset": "ISO-8859-1"
}
]
}
Config option | Type | Description |
---|---|---|
scTenants |
JSON array | Array of sip2 tenant configurations. |
scSubnet |
string | IPv4 CIDR of a tenant's self service kiosk. This is used to identify the tenant configuration for an incoming kiosk connection. |
tenant |
string | The FOLIO assigned tenant ID. |
errorDetectionEnabled |
boolean | Indicates whether or not the self service kiosk will be using SIP error detection in messages sent to and from this module. Defaults to "false". |
messageDelimiter |
string | The character sequence that indicates the end of a single SIP message. This is available in case the self check kiosk is not compliant with the SIP specification. The default is "\r" |
fieldDelimiter |
string | The character that the self service kiosk will use when encoding SIP messages. Defaults to "|". |
charset |
string | The character set SIP messages must be encoded with when sent and received by the self service kiosk. The charset must be defined as a "Canonical Name for java.nio API". See: Supported Encodings. Default is "IBM850". |
Edge-sip2 supports various locations for sip2-tenants.conf tenant configuration. Additionally, it supports S3 config. To include vertx-config-s3 libraries when building edge-sip2, include the maven profile command:
mvn -P vertx-config-s3
Here is a sample sip2.conf for storing tenant config in S3:
{
"port": 6443,
"okapiUrl": "https://folio-testing-okapi.dev.folio.org",
"tenantConfigRetrieverOptions": {
"scanPeriod": 300000,
"stores": [{
"type": "s3",
"format": "json",
"config": {
"region": "my-region",
"bucket": "my-bucket",
"key": "sip2/sip2-tenants.conf"
},
"optional": true
}]
}
}
Certain properties are retrieved from FOLIO configuration once a user has logged in via SIP2. There are properties at the tenant level and properties per kiosk, which is defined as a FOLIO service point. Properties are stored as JSON via the configuration
module. Missing configuration properties will lead to edge-sip2 runtime failures. The edge-sip2 properties listed below must be manually created via the POST
/configurations/entries
API using a tool such as curl or postman.
Property | Type | Description |
---|---|---|
statusUpdateOk |
boolean |
Indicates to the kiosk that the SIP service allows patron status updates from the kiosk. |
offlineOk |
boolean |
Indicates to the kiosk that FOLIO supports off-line operations. |
supportedMessages |
object[] |
An array objects that indicate to the kiosk which messages are supported by the edge-sip2 module. |
patronPasswordVerificationRequired |
boolean |
Indicates whether or not SIP commands that supply a patron password will attempt to verify the password by attempting a FOLIO login with these supplied patron credentials. A failed patron login will fail the SIP request. |
Property | Type | Description |
---|---|---|
messageName |
string |
The name of the message. See: Messages |
isSupported |
string |
Y or N to indicate to the kiosk whether or not the message is supported |
{
"module": "edge-sip2",
"configName": "acsTenantConfig",
"enabled": true,
"value": "{\"supportedMessages\": [{\"messageName\": \"PATRON_STATUS_REQUEST\",\"isSupported\": \"N\"},{\"messageName\": \"CHECKOUT\",\"isSupported\": \"Y\"},{\"messageName\": \"CHECKIN\",\"isSupported\": \"Y\"},{\"messageName\": \"BLOCK_PATRON\",\"isSupported\": \"N\"},{\"messageName\": \"SC_ACS_STATUS\",\"isSupported\": \"Y\"},{\"messageName\": \"LOGIN\",\"isSupported\": \"Y\"},{\"messageName\": \"PATRON_INFORMATION\",\"isSupported\": \"Y\"},{\"messageName\": \"END_PATRON_SESSION\",\"isSupported\": \"Y\"},{\"messageName\": \"FEE_PAID\",\"isSupported\": \"N\"},{\"messageName\": \"ITEM_INFORMATION\",\"isSupported\": \"N\"},{\"messageName\": \"ITEM_STATUS_UPDATE\",\"isSupported\": \"N\"},{\"messageName\": \"PATRON_ENABLE\",\"isSupported\": \"N\"},{\"messageName\": \"HOLD\",\"isSupported\": \"N\"},{\"messageName\": \"RENEW\",\"isSupported\": \"N\"},{\"messageName\": \"RENEW_ALL\",\"isSupported\": \"N\"}, {\"messageName\": \"REQUEST_SC_ACS_RESEND\",\"isSupported\": \"Y\"}],\"statusUpdateOk\": false,\"offlineOk\": false,\"patronPasswordVerificationRequired\": true}"
}
Property | Type | Description |
---|---|---|
retriesAllowed |
number |
Indicates to the kiosk the number of retries allowed by FOLIO. This should be a number between 0 and 999, where 999 means that the retry number is unknown. |
timeoutPeriod |
number |
Indicates to the kiosk the period of time before a transaction is aborted by the kiosk. The number should be between 0 and 999, where 0 means that FOLIO is not online and 999 means the time out is unknown. The number is expressed in tenths of a second. |
checkinOk |
boolean |
Indicates whether or not the kiosk is allowed to check in items. |
acsRenewalPolicy |
boolean |
Indicates that the kiosk is allowed by FOLIO to process patron renewal requests. |
checkoutOk |
boolean |
Indicates whether or not the kiosk is allowed to check out items. |
libraryName |
string |
The name of the library where the kiosk is located or whatever makes sense for the tenant. |
terminalLocation |
string |
This could be the location of the kiosk within the library or the UUID of the service point. |
{
"module": "edge-sip2",
"configName": "selfCheckoutConfig.e0ab8c91-2a4a-433d-a3cf-1837053c89a8",
"enabled": true,
"value": "{\"timeoutPeriod\": 5,\"retriesAllowed\": 3,\"checkinOk\": true,\"checkoutOk\": true,\"acsRenewalPolicy\": false,\"libraryName\": \"Datalogisk Institut\",\"terminalLocation\": \"e0ab8c91-2a4a-433d-a3cf-1837053c89a8\"}"
}
Property | Type | Description |
---|---|---|
timezone |
string |
The tenant's time zone as set in FOLIO. |
{
"module": "ORG",
"configName": "localeSettings",
"enabled": true,
"value": "{\"timezone\":\"America/New_York\"}"
}
Currently, edge-sip2 implements select SIP messages. Below, is the list of all implemented messages.
SIP Request | Implemented | Notes |
---|---|---|
Patron Status Request | No | |
Checkout | Yes | Response SIP fields hardcoded: "renewal ok" is set to "N", "magnetic media" is set to "U". SIP field "desensitize" is set to "Y" is the FOLIO check out succeeded and "N" when there is failure. Fee/fines related fields are not implemented. The "due date" format is the same as other SIP date/time format strings: "YYYYMMDDZZZZHHMMSS". |
Checkin | Yes | Response SIP fields hardcoded: "alert" is set to "N", "magnetic media" is set to "U". Most optional SIP fields are not implemented. The "resensitize" field will be set to "Y" if the FOLIO check in succeeded and "N" if there was a failure. |
Block Patron | No | |
SC Status | Yes | |
Request ACS Resend | Yes | |
Login | Yes | The request "location code" should contain the UUID of the service point for the kiosk. |
Patron Information | Yes | Response SIP field "hold items count" currently only refers to FOLIO "Hold" requests and not "Page" requests. Only "Hold" request data is returned in the summary results as well. The "charged items count", "fine items count" and "unavailable holds count" are not supported. However, "unavailable holds count" is not implemented due to an oversight as "hold items count" includes all "Hold" requests in any "Open" state, instead of limited to "Open - Awaiting pickup". Likewise, "unavailable holds count" could contain "Hold" requests that are not "Closed" and not "Open - Awaiting pickup". This will likely need to be corrected. The response "patron status" field is partially supported via FOLIO manual blocks where a "borrowing" block will set all privilege codes to "N", a "renewals" block will set the SIP "renewal privileges defined" code to "N", and a "requests" block will set "hold privileges denied" and "recall privileges denied" to "N". |
End Patron Session | Yes | |
Fee Paid | No | |
Item Information | No | |
Patron Enable | No | |
Hold | No | |
Renew | No | |
Renew All | No |
The SIP protocol does not consider security apart from providing the possibility of SC/ACS negotiated encryption algorithm for the password and user ID, which seems unlikely to be used. All communication over the wire, via TCP, is plain text. This may be fine in an environment that is locked down in some way. However, FOLIO, along with this module, can be hosted in the cloud and plain text communication is not acceptable.
We strongly recommend securing communication from the SC to edge-sip2. To this end, edge-sip2 can be configured to use TLS to encrypt communication. This requires the SC to communicate with TLS as well. It is our understanding that most self service kiosks do not have this ability natively and third party solutions must be employed. One such solution is stunnel
.
A typical stunnel
deployment will involve installing the stunnel
service either on the SC or a machine that is locked down with the SC and provides a port that the SC will be configured to connect to for SIP communication. The SIP commands are then sent to and received from the stunnel
port unencrypted. Communication from the stunnel
to the TLS termination end point will be encrypted. There are several ways to terminate TLS and the advantages/disadvantages of these is out of scope for this document. Here, we will focus on enabling TLS termination via the edge-sip2 module. This is done via simple launch configuration options.
stunnel
can be downloaded here: https://www.stunnel.org/downloads.html
Example FOLIO configuration section for stunnel.conf
:
[FOLIO]
key = stunnel.pem
cert = stunnel.pem
client = yes
accept = 127.0.0.1:5555
connect = sip2.example.com:6443
Example edge-sip2 configuration:
$ java -jar edge-sip2-fat.jar -conf '{"port":1234,"okapiUrl":"https://folio-snapshot-okapi.dev.folio.org","tenant":"diku","netServerOptions":{"ssl":true,"pemKeyCertOptions":{"certPaths":["cert.crt"],"keyPaths":["cert.key"]}}}'
Config option | Type | Description |
---|---|---|
ssl |
boolean | Indicates whether or not to enable SSL (TLS) support for the server |
pemKeyCertOptions |
JSON object | Used when the certificate is in PEM format |
pfxKeyCertOptions |
JSON object | Used when the certificate is in PFX format |
keyStoreOptions |
JSON object | Used when the certificate is in JKS (Java Keystore) format |
pemKeyCertOptions |
type | Description |
---|---|---|
certPath |
string | File system path to a PEM formatted certificate |
certPaths |
JSON array of strings | File system paths to PEM formatted certificates |
keyPath |
string | File system path to PEM formatted key |
keyPaths |
JSON array of strings | File system paths to PEM formatted keys |
pfxKeyCertOptions |
type | Description |
---|---|---|
path |
string | File system path to PFX (PKCS #12) store |
password |
string | The password for the PFX (PKCS #12) store |
keyStoreOptions |
type | Description |
---|---|---|
path |
string | File system path to JKS key store |
password |
string | The password for the JKS key store |
For local development, there is no requirement to encrypt communications from a SIP2 client to edge-sip2. Unencrypted TCP sockets are the default when launching edge-sip2 as described in the Configuration section. Encrypted communication from a SIP2 client is only required when explicitly configured via the above options and is up to the developer to provide that secure connection for edge-sip2.
This module makes use of Micrometer to collect SIP2, Vert.x and JVM metrics. The metrics need to be collected by a monitoring system backed. This is where Micrometer provides flexibility, by allowing the module to code to the Micrometer interface, which is vendor neutral. Once determined, the vendor specific backend binding is provided runtime and can be easily replaced.
By default, metrics are disabled. To enable Vert.x metrics pass the following Java argument:
-Dvertx.metrics.options.enabled=true
With metrics enabled, configuration must be supplied to the verticle. This can be done as follows:
-options '{"metricsOptions":{"labels":["LOCAL","REMOTE","HTTP_PATH","HTTP_METHOD","HTTP_CODE","CLASS_NAME"],"enabled":true,"prometheusOptions":{"enabled":true,"startEmbeddedServer":true,"embeddedServerOptions":{"port":8081}}}}'
The metricsOptions
here indicate that the verticle should collect metrics with the supplied list of labels
. Some of these labels, like REMOTE
, may lead to high cardinality metrics. The default labels
list is "HTTP_METHOD", "HTTP_CODE", "POOL_TYPE" and "EB_SIDE".
Also specified here are prometheusOptions
. In this case, the backend will be Prometheus. Prometheus "scrapes" metrics via HTTP at a specified interval. The options specified here allow Vert.x to create an HTTP server to handle metrics scraping. Prometheus setup is outside the scope of this document. Other bindings could be used, like InfluxDB.
For a list of Vert.x metrics (HTTP Client and Net Server are the primary sources for metrics in this module) see: Vert.x core tools metrics
The following metrics are supplied by this module:
Metric name | Labels | Type | Description |
---|---|---|---|
org_folio_edge_sip2_command_timer |
command |
Timer | SIP2 command execution time |
org_folio_edge_sip2_invalidMessage_errors |
port |
Counter | A count of invalid message errors |
org_folio_edge_sip2_request_errors |
port |
Counter | A count of request errors |
org_folio_edge_sip2_response_errors |
port |
Counter | A count of response errors |
org_folio_edge_sip2_scResend_errors |
port |
Counter | A count of SC resend errors, which occurs when the module fails to send the SC a resend message when the prior received message was not understood |
org_folio_edge_sip2_socket_errors |
port |
Counter | A count of socket errors |
JVM metrics (memory, GC, threads, etc.) are supplied as well.
The Maven pom.xml contains 2 profiles, metrics-prometheus
and metrics-influxdb
. Building with either or both of these profiles active will include the appropriate dependencies required to use metrics with that registry.
$ mvn install -P metrics-prometheus
If metrics need to be enabled, it is probably best to add any required runtime binding jars to the fat jar as part of a build. If this is not possible, the module can still be launched via the community Docker image. N.B., we may find that this approach cumbersome and may need to come up with an alternative approach.
Copy the appropriate runtime binding jars to a directory:
$ cp micrometer-registry-prometheus-1.1.5.jar simpleclient_common-0.5.0.jar simpleclient-0.5.0.jar /my/metrics/libs
Then run a container from the FOLIO docker hub image (either snapshot folioci/edge-sip2
or released folioorg/edge-sip2
):
$ docker run -v /my/metrics/libs:/metrics -p 6443:6443 --expose 8081 -p 8081:8081 -e JAVA_OPTIONS="-Dvertx.metrics.options.enabled=true " -e JAVA_CLASSPATH=/metrics/*:/usr/verticles/edge-sip2-fat.jar -e JAVA_MAIN_CLASS=io.vertx.core.Launcher folioci/edge-sip2 run org.folio.edge.sip2.MainVerticle -conf '{"port":6443,"okapiUrl":"https://folio-okapi.example.com","tenant":"diku","messageDelimiter":"\r","errorDetectionEnabled":true,"charset":"ISO-8859-1"}' -options '{"metricsOptions":{"labels":["LOCAL","REMOTE","HTTP_PATH","HTTP_METHOD","HTTP_CODE","CLASS_NAME"],"enabled":true,"prometheusOptions":{"enabled":true,"startEmbeddedServer":true,"embeddedServerOptions":{"port":8081}}}}'
This example shows how to launch with the Prometheus binding. Since Prometheus needs to scrape the metrics, we need to expose port for the HTTP server.
This log message happens when one or more of the FOLIO configuration key/value maps are missing when retrieved. Ensure that each set of properties is stored in FOLIO configuration
and that the service point UUID for the kiosk configuration matches the "location code" in the SIP "Login" message. Another problem could be that the tenant locale settings may not be saved in the database. On the initial deployment of FOLIO, as of Edelweiss, the locale settings are defaulted by the UI and not stored in the database until the "save" button is pressed. Since the UI defaults to usable settings for many, it may be misleading that these settings are present for backend modules, like edge-sip2, to consume.
See project SIP2 at the FOLIO issue tracker.
Other modules are described, with further FOLIO Developer documentation at dev.folio.org