Inventory Filter Module

Inventory Filter Module

Use Case

This filter module is used to extract a set of user-defined columns of a certain DB to be stored in the correspondent table file. A csv file is created for each table containing exported records. This files can then be used to, for instance, remove some records from a database.

Implementation

When the filter receives the data structure that contains the database records, they come in three types of cells: Simple Cell, Binary Cell and Null Cell. To extract the data, only the Simple Cell are considered, while the others are marked as "N/A" in the generated csv files. In order to make a perfect combination between the column name, and the data to which it corresponds, the indexes in which they occur in the list of cells are saved, and then print the csv file with the correct information. When the user enters invalid parameters or in the wrong format, exceptions with error messages are thrown.

Result

Produces a CSV file for each table that has annotated columns for export.

For example, for this configuration file, the result obtained is the generation of these two CSV files.

Example config file

schemas:
    sakila:
        tables:
            - name: "actor"
              columns:
              - name: "actor_id"
              - name: "first_name"
                inventory: true
              - name: "last_name"
                inventory: true
            - name: "last_update"
                where: ""
                orderBy: ""
            - name: "category"
              columns:
              - name: "category_id"
              - name: "name"
                inventory: true
              - name: "last_update"
                inventory: true
                where: ""
                orderBy: ""

Files obtained:

sakila.actor.csv

first_name,last_name
PENELOPE,GUINESS
NICK,WAHLBERG
ED,CHASE
JENNIFER,DAVIS
JOHNNY,LOLLOBRIGIDA
BETTE,NICHOLSON

sakila.category.csv

name,last_update
Action,2006-02-15T04:46:27.000000Z
Animation,2006-02-15T04:46:27.000000Z
Children,2006-02-15T04:46:27.000000Z
Classics,2006-02-15T04:46:27.000000Z

How to use

The filter can be invoked without the need to add parameters. By default, the column headers of the CSV files will be printed, the separator will be a comma and the generated files will be saved in the home directory.

dbptk migrate -i <import-module> [args] -e [export-module] [args] -f inventory

Advanced configurations

Change directory to store csv files

dbptk migrate -i <import-module> [args] -e [export-module] [args] -f inventory -f1dir <path>

Set the prefix to add to the csv files names

dbptk migrate -i <import-module> [args] -e [export-module] [args] -f imventory -f1pr <prefix>

Disable print header feature in csv files

dbptk migrate -i <import-module> [args] -e [export-module] [args] -f inventory -f1dh

Set the separator of csv files

dbptk migrate -i <import-module> [args] -e [export-module] [args] -f inventory -f1sp "<separator>"

Integration with the import-config module

By default, all columns are disabled for the extraction of the DB records. It is possible to explicit choose which columns to export. This can be done with the import-config module that can be consulted here.