SaasSync

A pre-processing task connects to the SaaS instance and retrieves the primary key, update time and creation time from a table in the SaaS. The list of primary keys is stored in the context for a "SyncReader" to use to retrieve the records which need to be added or updated to the local database. The Reader generates complete frames and sent through the pipeline to the "SyncWriter" which determines how to write (insert or update) the frame to the local database. Post-processing tasks save results and records any audit activities which may be required due to irregularities encountered during writing the local data.

Another job which takes the output of the above process and uses configured settings to select a set of records for auditing; a set or remote and local records is selected and compared to ensure data integrity.

The implementation of this use case is currently under development.