Releases: karhunenloeve/TwirlFlake
TwirlFlakev.1.0
Homological Time Series Analysis of Sensor Signals from Power Plants
In this paper, we use topological data analysis techniques to construct a suitable neural network classifier for the task of learning sensor signals of entire power plants according to their reference designation system. We use representations of persistence diagrams to derive necessary preprocessing steps and visualize the large amounts of data. We derive architectures with deep one-dimensional convolutional layers combined with stacked long short-term memories as residual networks suitable for processing the persistence features. We combine three separate sub-networks, obtaining as input the time series itself and a representation of the persistent homology for the zeroth and first dimension. We give a mathematical derivation for most of the used hyper-parameters. For validation, numerical experiments were performed with sensor data from four power plants of the same construction type.
Keywords: Power Plants · Time Series Analysis · Signal processing · Geometric embeddings · Persistent homology · Topological data analysis.
- This is the link to the arxiv article.
- This is the link to the presentation hold at ML4ITS within the ECMLPKDD.
- This is the link to the transcript of the talk given at ML4ITS.
Citation
@inproceedings{ML4ITS/MelodiaL21,
author = {Luciano Melodia and
Richard Lenz},
editor = {Kamp M. et al.},
title = {Homological Time Series Analysis of Sensor Signals from Power Plants},
booktitle = {Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2021},
series = {Communications in Computer and Information Science},
volume = {1524},
pages = {283--299},
publisher = {Springer},
year = {2021},
url = {https://doi.org/10.1007/978-3-030-93736-2_22},
doi = {10.1007/978-3-030-93736-2_22},
}
Contents
- Time Series Helper
timeSeriesHelper.py
- Time Series Converter
timeSeriesConverter.py
- Time Series Generator
timeSeriesGenerator.py
- Time Series Embedding
timeSeriesEmbedding.py
- Time Series Homology
timeSeriesHomology.py
- Time Series Visualisation
timeSeriesVisualisation.py
- Time Series Models
timeSeriesModels.py
- Time Series Classifier
timeSeriesClassifier.py
timeSeriesHelper
zip_to_csv
zip_to_csv(path: str)
Converts a packed sql
file to a csv
file.
This function unpacks the specified zip
file at its location and calls the specified sql_to_csv
function on the sql
file, with the same name as the zip file.
- param path: as an absolute path to the
zip
file with thesql
in it, typestr
.
zip_to_npy
zip_to_npy(path: str)
Converts a packed sql
file to an npy file.
This function unpacks the specified zip file at its location and calls the specified sql_to_npy
function on the sql
file, with the same name as the zip file.
- param path: as an absolute path to the zip file containing
sql
, typestr
.
sql_to_csv
sql_to_csv(path: str, delimiter: str = "\n")
Converts a set of INSERT
statements to csv
format.
Extracts the data from a set of INSERT
statements stored in a sql
file, this function
converts the data into a csv
file, where each non INSERT
line is stored in a separate pickle file
file, and the data of the INSERT
statements is stored line by line, with the specified delimiter
at the end of each line.
- param path: as absolute path to the
sql
file, typestr
. - param delimiter: as delimiter at the end of each line, type
str
.
sql_to_npy
sql_to_npy(path: str, delimiter: str = ",")
Converts a set of INSERT
statements into a numpy array.
Similar to the csv function, this function also stores unused data in a pickle file and creates
a brand new file with the extracted data, this time in npy format, but this time the
the delimiter must be the delimiter used in the sql
file, plus an additional
missing_values string used to represent missing data.
- param path: as the absolute path to the
sql
file, typestr
. - param delimiter: as the string used in the
sql
file to separate the data, typestr
. - param missing_values: the string used for missing data, type
str
.
csv_to_sql
csv_to_sql(path: str, delimiter: str = "\n")
Converts an csv
file into a set of INSERT
statements.
This function converts each set of data separated by the specified separator character
of a csv
file into an INSERT
statement. It also inserts data
stored in a pickle file which has the same name as the csv
file,
as a comment at the beginning, so as not to interfere with functionality.
- param path: as absolute path to the
csv
file, typestr
. - param delimiter: as string to recognize the different records, type
str
.
csv_to_npy
csv_to_npy(path: str, delimiter: str = ",")
Converts a csv
file to a Numpy array representation.
This function converts a csv
file into a 2-dimensional Numpy representation,
where each record separated by the specified delimiter is interpreted as a new line.
- param path: as absolute path to the
csv
file, typestr
. - param delimiter: the string used to determine the rows of the numpy array, type
str
. - param missing_values: as the string used to represent missing data, type
str
.
npy_to_sql
npy_to_sql(path: str)
Converts an npy file into a series of INSERT
statements.
This function is the reverse of sql_to_npy and if you use it in conjunction with
you will have the same file at the end as at the beginning.
- param path: as absolute path to the npy file, type
str
.
gen_GAF
gen_GAF(path: str)
Generate a gramian angle field with user input.
This function receives user input via the console to generate
either a Gramian Angular Summation Field or a Gramian Angular Difference Field
from the data of a Numpy array using the function gen_GAF_exec.
- param path: as absolute path to the npy file, type
str
.
gen_GAF_exec
gen_GAF_exec(
data: list,
sample_range: None or tuple = (-1, 1),
method: str = "summation",
null_value: str = "0",
)
Generate a Gramian Angular Field.
This is the actual function when it comes to generating a Gramian Angular Field
from the data of a Numpy array. This function takes several variables to determine
how the field should be scaled, what size it should have
and whether it is a sum or difference field.
- param data: as the contents of an npy file , type
list
.
param size: this is the size of the square output image, type int orfloat
. - param sample_range: as the range to scale the data to, type None or
tuple
. - param method: as the type of field to scale to, type
sum
ordifference
, typestr
. - param null_value: as the number to use instead of NULL, type
str
.
false_input
false_input(path: str)
Output error information and return to main.
This function prints an error message to the console and invokes main.
exit
exit()
Print message and exit program.
This function prints a message on the console and terminates the program.
switchoption
switchoption(n: int, path: str)
...