TwirlFlakev.1.0

Homological Time Series Analysis of Sensor Signals from Power Plants

In this paper, we use topological data analysis techniques to construct a suitable neural network classifier for the task of learning sensor signals of entire power plants according to their reference designation system. We use representations of persistence diagrams to derive necessary preprocessing steps and visualize the large amounts of data. We derive architectures with deep one-dimensional convolutional layers combined with stacked long short-term memories as residual networks suitable for processing the persistence features. We combine three separate sub-networks, obtaining as input the time series itself and a representation of the persistent homology for the zeroth and first dimension. We give a mathematical derivation for most of the used hyper-parameters. For validation, numerical experiments were performed with sensor data from four power plants of the same construction type.

Keywords: Power Plants · Time Series Analysis · Signal processing · Geometric embeddings · Persistent homology · Topological data analysis.

• This is the link to the arxiv article.
• This is the link to the presentation hold at ML4ITS within the ECMLPKDD.
• This is the link to the transcript of the talk given at ML4ITS.

Citation

@inproceedings{ML4ITS/MelodiaL21,
  author    = {Luciano Melodia and
               Richard Lenz},
  editor    = {Kamp M. et al.},
  title     = {Homological Time Series Analysis of Sensor Signals from Power Plants},
  booktitle = {Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2021},
  series    = {Communications in Computer and Information Science},
  volume    = {1524},
  pages     = {283--299},
  publisher = {Springer},
  year      = {2021},
  url       = {https://doi.org/10.1007/978-3-030-93736-2_22},
  doi       = {10.1007/978-3-030-93736-2_22},
}

Contents

1. Time Series Helper timeSeriesHelper.py
   • zip_to_csv
   • zip_to_npy
   • sql_to_csv
   • sql_to_npy
   • csv_to_sql
   • csv_to_npy
   • npy_to_sql
   • gen_GAF
   • gen_GAF_exec
   • exit
   • switchoption
   • checkpath
   • split_csv_file
   • create_datasets
2. Time Series Converter timeSeriesConverter.py
   • persistence_giotto_to_matplotlib
3. Time Series Generator timeSeriesGenerator.py
   • chunkIt
   • numpy_to_img
   • create_timeseries_dataset_persistence_images
   • create_timeseries_dataset_ts_betti_curves
4. Time Series Embedding timeSeriesEmbedding.py
   • list_files_in_dir
   • read_csv_data
   • fit_embedder
   • get_single_signal
   • get_sliding_window_embedding
   • get_periodic_embedding
   • count_dimension_distribution
5. Time Series Homology timeSeriesHomology.py
   • compute_persistence_representations
6. Time Series Visualisation timeSeriesVisualisation.py
   • export_cmap_to_cpt
   • plot_embedding3D
   • mean_heatkernel2D
   • massive_surface_plot3D
   • massive_line_plot3D
7. Time Series Models timeSeriesModels.py
   • convolution_layer
   • lstm_layer
   • create_lstm
   • create_subnet
   • create_three_nets
8. Time Series Classifier timeSeriesClassifier.py
   • recall
   • precision
   • f1
   • check_files
   • train_neuralnet_images

timeSeriesHelper

zip_to_csv

zip_to_csv(path: str)

Converts a packed sql file to a csv file.

This function unpacks the specified zip file at its location and calls the specified sql_to_csv
function on the sql file, with the same name as the zip file.

• param path: as an absolute path to the zip file with the sql in it, type str.

zip_to_npy

zip_to_npy(path: str)

Converts a packed sql file to an npy file.

This function unpacks the specified zip file at its location and calls the specified sql_to_npy
function on the sql file, with the same name as the zip file.

• param path: as an absolute path to the zip file containing sql, type str.

sql_to_csv

sql_to_csv(path: str, delimiter: str = "\n")

Converts a set of INSERT statements to csv format.

Extracts the data from a set of INSERT statements stored in a sql file, this function
converts the data into a csv file, where each non INSERT line is stored in a separate pickle file
file, and the data of the INSERT statements is stored line by line, with the specified delimiter
at the end of each line.

• param path: as absolute path to the sql file, type str.
• param delimiter: as delimiter at the end of each line, type str.

sql_to_npy

sql_to_npy(path: str, delimiter: str = ",")

Converts a set of INSERT statements into a numpy array.

Similar to the csv function, this function also stores unused data in a pickle file and creates
a brand new file with the extracted data, this time in npy format, but this time the
the delimiter must be the delimiter used in the sql file, plus an additional
missing_values string used to represent missing data.

• param path: as the absolute path to the sql file, type str.
• param delimiter: as the string used in the sql file to separate the data, type str.
• param missing_values: the string used for missing data, type str.

csv_to_sql

csv_to_sql(path: str, delimiter: str = "\n")

Converts an csv file into a set of INSERT statements.

This function converts each set of data separated by the specified separator character
of a csv file into an INSERT statement. It also inserts data
stored in a pickle file which has the same name as the csv file,
as a comment at the beginning, so as not to interfere with functionality.

• param path: as absolute path to the csv file, type str.
• param delimiter: as string to recognize the different records, type str.

csv_to_npy

csv_to_npy(path: str, delimiter: str = ",")

Converts a csv file to a Numpy array representation.

This function converts a csv file into a 2-dimensional Numpy representation,
where each record separated by the specified delimiter is interpreted as a new line.

• param path: as absolute path to the csv file, type str.
• param delimiter: the string used to determine the rows of the numpy array, type str.
• param missing_values: as the string used to represent missing data, type str.

npy_to_sql

npy_to_sql(path: str)

Converts an npy file into a series of INSERT statements.

This function is the reverse of sql_to_npy and if you use it in conjunction with
you will have the same file at the end as at the beginning.

• param path: as absolute path to the npy file, type str.

gen_GAF

gen_GAF(path: str)

Generate a gramian angle field with user input.

This function receives user input via the console to generate
either a Gramian Angular Summation Field or a Gramian Angular Difference Field
from the data of a Numpy array using the function gen_GAF_exec.

• param path: as absolute path to the npy file, type str.

gen_GAF_exec

gen_GAF_exec(
    data: list,
    sample_range: None or tuple = (-1, 1),
    method: str = "summation",
    null_value: str = "0",
)

Generate a Gramian Angular Field.

This is the actual function when it comes to generating a Gramian Angular Field
from the data of a Numpy array. This function takes several variables to determine
how the field should be scaled, what size it should have
and whether it is a sum or difference field.

• param data: as the contents of an npy file , type list.
  param size: this is the size of the square output image, type int or float.
• param sample_range: as the range to scale the data to, type None or tuple.
• param method: as the type of field to scale to, type sum or difference, type str.
• param null_value: as the number to use instead of NULL, type str.

false_input

false_input(path: str)

Output error information and return to main.

This function prints an error message to the console and invokes main.

exit

exit()

Print message and exit program.

This function prints a message on the console and terminates the program.

switchoption

switchoption(n: int, path: str)
...