TFSpectralGate is a TensorFlow-based implementation of Spectral Gating, an algorithm for denoising audio signals. It is designed to inherit from the tf.keras.Model class, which allows it to be used either as a standalone module or as part of a larger neural network architecture.
The algorithm was originally proposed by Sainburg et al [1] and was previously implemented in a GitHub repository [2]. The current implementation was developed in TensorFlow to improve computational efficiency and reduce run time.
[1] Sainburg, Tim, and Timothy Q. Gentner. “Toward a Computational Neuroethology of Vocal Communication: From Bioacoustics to Neurophysiology, Emerging Tools and Future Directions.” Frontiers in Behavioral Neuroscience, vol. 15, 2021. Frontiers, https://www.frontiersin.org/articles/10.3389/fnbeh.2021.811737.
[2] Sainburg, T. (2019). noise-reduction. GitHub. Retrieved from https://github.com/timsainb/noisereduce.
pip install tfgating
Tested on:
Python 3.10
matplotlib==3.7.1
numpy==1.24.2
soundfile==0.11.0
tensorflow==2.15.0
Please note that TensorFlowSpectralGating may work on other versions of the above dependencies, but these are the versions that were tested.
Class for performing parallel spectral gating.
import tensorflow
from tfgating import TFGating as TG
gpus = tf.config.experimental.list_physical_devices('GPU')
device = "/GPU:0" if gpus else "/CPU:0"
# Create TensorFlowGating instance
tg = TG(sr=8000, nonstationary=True)
# Apply Spectral Gate to noisy speech signal
noisy_speech = tf.random.normal(shape=(3, 32000), dtype=tf.float32, seed=42)
with tf.device(device):
enhanced_speech = tg(noisy_speech).numpy()
| Parameter | Description |
|---|---|
| sr | Sample rate of the input signal. |
| n_fft | The size of the FFT. |
| hop_length | The number of samples between adjacent STFT columns. |
| win_length | The window size for the STFT. If None, defaults to n_fft. |
| freq_mask_smooth_hz | The frequency smoothing width in Hz for the masking filter. If None, no frequency masking is applied. |
| time_mask_smooth_ms | The time smoothing width in milliseconds for the masking filter. If None, no time masking is applied. |
| n_std_thresh_stationary | The number of standard deviations above the noise mean to consider as signal for stationary noise. |
| nonstationary | Whether to use non-stationary noise masking. |
| n_movemean_nonstationary | The number of frames to use for the moving average in the non-stationary noise mask. |
| n_thresh_nonstationary | The multiplier to apply to the sigmoid function in the non-stationary noise mask. |
| temp_coeff_nonstationary | The temperature coefficient to apply to the sigmoid function in the non-stationary noise mask. |
| prop_decrease | The proportion of decrease to apply to the mask. |
The "run.py" script provides a command-line interface for applying the SpectralGate algorithm to audio files. The program will apply the SpectralGate algorithm to all audio files in the input directory, or to the single audio file specified by 'input', and save the processed files in the output directory. If the --graphs option is enabled, the program will also display input and output spectrograms as plots.
Here is an example of how to use the command line interface:
tfgating <input_path> --output <output_path> --nonstationary --verbose --norm --graphs --subdirs
The script takes the following arguments:
- input: Path to a directory containing audio files or to a single audio file.
- --output: Path to a directory to save the output files (default: 'output').
- --nonstationary: Whether to use non-stationary or stationary masking (default: False).
- --verbose: Flag indicating whether verbose mode is enabled (default: False).
- --cpu: Flag indicating whether to run the algorithm on CPU instead of GPU (default: False).
- --subdirs: Whether to create a subdirectory for stationary or non-stationary outputs (default: False).
- --norm: Whether to normalize the signals (default: False).
- --graphs: Flag indicating whether to display input and output spectrograms as plots (default: False).
- --figsize: Figure size for the spectrogram plots in inches (default: (10, 6)).
- --figformat: If figformat is set, it determines the output format (default: png).
- --vmin: Minimum value for the color scale in dB (default: -80).
- --vmax: Maximum value for the color scale in dB (default: None).
- --cmap: Name of the colormap to use for the spectrogram plots (default: 'magma').
TensorFlowSpectralGate supports both stationary and non-stationary noise reduction. To enable parallel computation, a few modifications were made to the original algorithm. In the non-stationary spectral gating, an FIR filter was implemented instead of an IIR filter.
TF-Mask can be estimated using stationary and non-stationary methods.
)
TBD
For the evaluation, a speech utterance was taken from the NOIZEUS database [3], a repository of noisy speech corpus.
The sentence 'sp09.wav' was degraded with car noise. This was done through the addition of interfering signals at signal-to-noise ratios ranging from 0 to 15 dB, using method B of the ITU-T P.56.
[3] Hu, Y. and Loizou, P. (2007). “Subjective evaluation and comparison of speech enhancement algorithms,” Speech Communication, 49, 588-601.
