data-files.md

Working with Data Files

Instant Answers - particularly Goodies - can use simple text files for display or processing. These files can be read once and reused to answer many queries without cluttering up your source code.

The share function gives each Instant Answer access to a subdirectory of the repository's share directory. The subdirectory for your Instant Answer is based on its Perl package name which is transformed from CamelCase to underscore_separated_words.

Usage

The MAC Address Goodie uses the share directory to hold data for processing purposes:

my %oui_db = share("oui_database.txt")->slurp;

Here the share function grabs the oui_database.txt file. The returned object's slurp method is called which pushes each line of the file into an array.

The full code performs some additional parsing on each line of the slurp-ed array:

my %oui_db = map { chomp; my (@f) = split(/\\n/, $_, 2); ($f[0] => $f[1]); } share("oui_database.txt")->slurp;

You may decide to take advantage of particular data formats, such as JSON, as does the Independence Day Goodie:

my $data = share('independence_days.json')->slurp;
$data = decode_json($data);

Another example involves parsing a YAML file, as does the Paper Size Goodie:

use YAML::XS 'LoadFile';

my $sizes = LoadFile(share('sizes.yml'));

UTF-8 Encoding

One further consideration is whether your data file contains non-ASCII characters. If so, you will want to prepare the file with UTF-8 encoding.

my @shortcuts = share('shortcuts.csv')->slurp(iomode => '<:encoding(UTF-8)');

As with the Passphrase Goodie example above, each line becomes an entry in the resulting array. With this iomode set, the UTF-8 characters used to denote system-specific keys will be handled properly throughout the rest of the processing.

Generating Data Files

In some cases, you may need to generate the data files you will slurp from the share directory. If so, please put the required generation scripts in the Goodie's share directory. While shell scripts are preferred, your scripts can be written in the language of your choice.

As an example, the CurrencyIn Goodie uses a combination of Shell and Python scripts to generate its input data, currency.txt.