Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
ferrarodav authored Apr 20, 2021
1 parent 6bb85fe commit 9d10fcc
Showing 1 changed file with 40 additions and 38 deletions.
78 changes: 40 additions & 38 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,8 @@
# Memory and interruption safe map <!-- omit in toc -->
## Alternative for Matlab® `cellfun` <!-- omit in toc -->
### Alternative for Matlab® `cellfun` <!-- omit in toc -->

`safeMap` saves on disk data generated by long and heavy computations so that you don't have to worry about memory problems and unexpected interruptions.

## Table of contents <!-- omit in toc -->


- [Description](#description)
- [Limitations](#limitations)
- [Installation](#installation)
- [Arguments](#arguments)
- [Outputs](#outputs)
- [Examples](#examples)
- [Explore a grid of hyperparameters for a simulation or training a Neural Netork](#explore-a-grid-of-hyperparameters-for-a-simulation-or-training-a-neural-netork)
- [Setting a configuration](#setting-a-configuration)
- [Contributing](#contributing)
- [License](#license)

## Description

The passed function is fed with each input cell and the outputs are saved in a file as they are generated;`safeMap` can either return the file handle or the whole data (with `config.returnData`, which also deletes the whole file after returning the data). If something interrupts the execution, `safeMap` will automatically resume once executed again.

Each output of the function can be saved in a cell matrix with the same structure of the input one, or joined (with `config.joinUniformOutput`). In the latter case, which is the default, each output matrix will be equal to the function outputs stacked along the leading dimensions (e.g. if inputs is of size `[3, 3]` and each output is of size `[4, 2]` the final output will be of size `[3, 3, 4, 2]`).
Expand All @@ -28,22 +12,33 @@ Moreover, the used file paths and the name under which each output variable is s

```matlab
>> output = safeMap(@(x) x^5, repmat({rand(300)}, 10, 10));
Progress: 100 / 100 (100.0\%)
Progress: 100 / 100 (100.0%)
Estimated remaining time: 0s
```

### Limitations

`safeMap` writes data on the disk after every output computed, so it's inherently slower than `cellfun`. Moreover, currently it accepts only one input, though this can be easily overcome like in the example below.

### Table of contents <!-- omit in toc -->

- [Installation](#installation)
- [Arguments](#arguments)
- [Outputs](#outputs)
- [Examples](#examples)
- [Explore a grid of hyperparameters for a simulation or training a Neural Netork](#explore-a-grid-of-hyperparameters-for-a-simulation-or-training-a-neural-netork)
- [Setting a configuration](#setting-a-configuration)
- [Contributing](#contributing)
- [License](#license)


## Installation

Download the [function](https://github.com/download/ferrarodav/safemap/safeMap.m) and put it in your workspace.
Download the [function](https://raw.githubusercontent.com/ferrarodav/safeMap/main/safeMap.m) and put it in your workspace.

Command line installation example:
Or clone the whole repository with git:
```bash
cd matlab_workspace_folder
curl --location --remote-header-name --remote-name https://github.com/download/ferrarodav/safemap/safeMap.m
git clone https://github.com/ferrarodav/safeMap.git
```

## Arguments
Expand All @@ -56,7 +51,7 @@ curl --location --remote-header-name --remote-name https://github.com/download/f
- `filePath`: absolute or relative path of the file where the output data is saved. NB if using a file already existing, make sure the mat version is 7.3 to allow partial writing [default is `output.mat`]
- `tempFilePath`: absolute or relative path of the file where the current state is saved (the index of the cycle) [default is `progress.safemap.mat`]
- `variableName`: the names of the variables under which the output data is saved inside the file specified in filePath. Can be a single string, a cell of strings if multiple outputs. If less names than outputs "data_i" will be used for the missing names [default is `data`]
- `returnData`: true/false. If true, a cell containing all the outputs will be returned and the output file will be deleted. If false the output file handler is returned [default is `false`]
- `returnData`: true/false. If true, a cell containing all the outputs will be returned and **the output file will be deleted**. If false the output file handler is returned [default is `false`]
- `joinUniformOutput`: like `cellfun` uniformOutput option, instead of returning a cell matrix of individual outputs, tries to join them together in a single matrix. Only numeric, logical, cell and struct outputs can be joined [default is `true`]
- `numberOfFunctionOutputs`: number of outputs to expect from the passed function [by default it is inferred from `function_handle`]

Expand Down Expand Up @@ -97,20 +92,27 @@ disp(output_file.data(2,3,2,1)); % 30i
### Setting a configuration

```matlab
data = num2cell(rand(100, 1000), 2);
size(data)
% [100 1]
size(data{1})
% [1 1000]
maxValues, maxIndexs = safeMap(@(x) maxk(x, 2), data, struct('returnData', true, 'variableNames', {'values', 'indexs'}));
whos -file output.mat
% variables: values, indexs
size(maxValues)
% [100 2]
size(maxIndexs)
% [100 2]
>> data = num2cell(rand(100, 1000), 2);
>> size(data)
ans =
100 1
>> size(data{1})
ans =
1 1000
>> maxValues, maxIndexs = safeMap(@(x) maxk(x, 2), data, struct('returnData', true));
>> size(maxValues)
ans =
100 2
>> size(maxIndexs)
ans =
100 2
>> output_file = safeMap(@(x) maxk(x, 2), data, struct('variableNames', 'filePath', 'max.mat', {{'values', 'indexs'}}));
>> who -file max.mat
Your variables are:
values indexs
```

## Contributing
Expand All @@ -128,4 +130,4 @@ To run the test use matlab command:
```

## License
[MIT](https://choosealicense.com/licenses/mit/)
[MIT](https://choosealicense.com/licenses/mit/)

0 comments on commit 9d10fcc

Please sign in to comment.