Persistence

Configuration

The JEM entities persistence must be configured both Hazelcast and jem-env.xml configuration file.

In Hazelcast, it's mandatory to configure the persistence to a relational database. Synchronous persistence must be configured to ensure data consistency.

The following sample is used to configure the input queue (see org.pepstock.jem.node.persistence.InputMapManager class name).

<map-store enabled="true">
    <!-- Name of the class implementing MapLoader and/or MapStore. The class 
         should implement at least of these interfaces and contain no-argument constructor. 
         Note that the inner classes are not supported. -->
    <class-name>org.pepstock.jem.node.persistence.InputMapManager</class-name>
    <!-- Number of seconds to delay to call the MapStore.store(key, value). 
        If the value is zero then it is write-through so MapStore.store(key, value) 
        will be called as soon as the entry is updated. Otherwise it is write-behind 
        so updates will be stored after write-delay-seconds value by calling Hazelcast.storeAll(map). 
        Default value is 0. -->
    <write-delay-seconds>0</write-delay-seconds>
</map-store>

It's mandatory to configure the persistence to a relational database in JEM node configuration file. JEM supports MySQL, IBM DB2 (former UDB) and ORACLE, and it can be extended for any kind of relational database. The database is used to make persistent the following:

• Input queue
• Output queue
• Routing queue
• Roles with users and permissions
• Resources
• Swarm configuration
• User preferences

A database is shared among cluster's nodes. Each node of the cluster must then have the same database configuration. Here is a sample:

<database>
   <driver>com.mysql.jdbc.Driver</driver>
   <url>jdbc:mysql://10.10.10.10:3306/JEM</url>
   <user>jem</user>
   <password>ASKJDPouisadalseioqwe</password>
   <properties>
      <property name="key" value="value">
      </property>
   </properties>
</database>

Attention that the database (or schema) must be created with the capability to be case sensitive. For example, for MySql, use the following sample:

CREATE DATABASE JEM COLLATE utf8_bin 

For Oracle please have a look to Oracle documentation.

For IBM DB2 please have a look to IBM documentation.

Oracle 11g and IBM DB2 10.5 (both for Windows 7, 64bit) has been used for testing.

In case of failure

JEM is able do not crash if it has got a problem with database used to persist objects. If an SQL error occurs to persist a object, for any reason, JEM moves the REDO action in a map (without persistence) of Hazelcast, waiting when the database will be ready again.

It's important do not close the whole JEM cluster having still REDO statements to apply, to avoid loosing data. JEM sets itself in Not Operational special status and drains itself.

No human actions are possible to restart the node. ONLY when the database will be available again, JEM will restart itself automatically.

The list of REDO statements are available to be checked inside of Administration section of web interface.

In case of failure of whole cluster, the jobs stored in RUNNING queue table are moved in output queue setting FATAL error (CC 12).

Currently the maps which can work in REDO mode are the following:

• JCL checking queue
• Input queue
• Output queue
• Routing queue
• Roles with users and permissions
• Resources