Skip to content

Commit

Permalink
Documentation update for new BirdNET-Lite functions
Browse files Browse the repository at this point in the history
  • Loading branch information
@sprestrelski
sprestrelski committed May 16, 2022
1 parent 92a31a2 commit e6190c6
Show file tree
Hide file tree
Showing 3 changed files with 138 additions and 36 deletions.
86 changes: 63 additions & 23 deletions PyHa/IsoAutio.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@
from .birdnet_lite.analyze import analyze
from copy import deepcopy

def build_isolation_parameters(
def build_isolation_parameters_microfaune(
technique,
threshold_type,
threshold_const,
threshold_min=0,
window_size=1.0,
chunk_size=2.0):
"""
Wrapper function for all of the audio isolation techniques (Steinberg,
Simple, Stack, Chunk). Will call the respective function of
each technique based on isolation_parameters "technique" key.
Wrapper function for all of Microfaune's audio isolation techniques
(Steinberg, Simple, Stack, Chunk). Will call the respective function
of each technique based on isolation_parameters "technique" key.
Args:
technique (string)
Expand Down Expand Up @@ -53,7 +53,7 @@ def build_isolation_parameters(
"""
isolation_parameters = {
"technique": technique,
"treshold_type": threshold_type,
"threshold_type": threshold_type,
"threshold_const": threshold_const,
"threshold_min": threshold_min,
"window_size": window_size,
Expand All @@ -80,9 +80,9 @@ def isolate(
manual_id="bird",
normalize_local_scores=False):
"""
Wrapper function for all of the audio isolation techniques (Steinberg,
Simple, Stack, Chunk). Will call the respective function of
each technique based on isolation_parameters "technique" key.
Wrapper function for all of Microfaune's audio isolation techniques
(Steinberg, Simple, Stack, Chunk). Will call the respective function of each technique based on
isolation_parameters "technique" key.
Args:
local_scores (list of floats)
Expand Down Expand Up @@ -536,7 +536,7 @@ def stack_isolate(
# increasing this stack counter will be referred to as "pushing"
stack_counter = stack_counter + 1

# when a score is below the treshold
# when a score is below the threshold
else:
# the case where it is the end of an annotation
if stack_counter == 0 and annotation_start == 1:
Expand Down Expand Up @@ -651,7 +651,7 @@ def chunk_isolate(
chunk_end = min((ndx + 1) * local_scores_per_chunk, len(local_scores))
# breaking up the local_score array into a chunk.
chunk = local_scores[int(chunk_start):int(chunk_end)]
# comparing the largest local score value to the treshold.
# comparing the largest local score value to the threshold.
# the case for if we label the chunk as an annotation
if max(chunk) >= thresh and max(
chunk) >= isolation_parameters["threshold_min"]:
Expand Down Expand Up @@ -785,7 +785,7 @@ def generate_automated_labels_microfaune(

def generate_automated_labels_birdnet(audio_dir, isolation_parameters):
"""
Function that generated the bird labels for an audio file or across a
Function that generates the bird labels for an audio file or across a
folder using the BirdNet-Lite model
Args:
Expand All @@ -796,17 +796,54 @@ def generate_automated_labels_birdnet(audio_dir, isolation_parameters):
isolation_parameters (dict)
- Python Dictionary that controls the various label creation
techniques. The keys it accepts are :
- output_path
- lat
- lon
- week
- overlap
- sensitivity
- min_conf
- custom_list
- filetype
- num_predictions
- write_to_csv
- output_path (string)
- Path to output folder. By default results are written into
the input folder
- default: None
- lat (float)
- Recording location latitude
- default: -1 (ignore)
- lon (float)
- Recording location longitude
- default: -1 (ignore)
- week (int)
- Week of the year when the recording was made
- Values in [1, 48] (4 weeks per month)
- default: -1 (ignore)
- overlap (float)
- Overlap in seconds between extracted spectrograms
- Values in [0.5, 1.5]
- default: 0.0
- sensitivity (float)
- Detection sensitivity. Higher values result in higher sensitivity
- Values in [0.5, 1.5]
- default: 1.0
- min_conf (float)
- Minimum confidence threshold
- Values in [0.01, 0.99]
- default: 0.1
- custom_list (string)
- Path to text file containing a list of species
- default: '' (not used if not provided)
- filetype (string)
- Filetype of soundscape recordings
- default: 'wav'
- num_predictions (int)
- Defines maximum number of written predictions in a given 3s segment
- default: 10
- write_to_csv (boolean)
- Set whether or not to write output to CSV
- default: False
Returns:
Dataframe of automated labels for the audio clip(s) in audio_dir.
Expand All @@ -822,7 +859,7 @@ def generate_automated_labels(
Normalized_Sample_Rate=44100,
normalize_local_scores=False):
"""
Function that generated the bird labels across a folder of audio clips
Function that generates the bird labels across a folder of audio clips
given the isolation_parameters
Args:
Expand All @@ -843,6 +880,9 @@ def generate_automated_labels(
Normalized_Sample_Rate (int)
- Sampling rate that the audio files should all be normalized to.
Used only for the Microfaune model.
normalize_local_scores (boolean)
- Set whether or not to normalize the local scores.
Returns:
Dataframe of automated labels for the audio clips in audio_dir.
Expand Down
2 changes: 1 addition & 1 deletion PyHa/birdnet_lite/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ python3 analyze.py --i 'example/XC563936 - Soundscape.mp3' --lat 47.6766 --lon -

Note: Please make sure to provide lat, lon, and week. BirdNET will work without these values, but the results might be less reliable.

The results of the anlysis will be stored in a result file in CSV format. All confidence values are raw prediction scores and should be post-processed to eliminate occasional false-positive results.
The results of the analysis will be stored in a result file in CSV format. All confidence values are raw prediction scores and should be post-processed to eliminate occasional false-positive results.

# Contact us

Expand Down
86 changes: 74 additions & 12 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,13 @@ PyHa = Python + Piha (referring to a bird species of our interest known as the s

Many of the functions take in the `isolation_parameters` argument, and as such it will be defined globally here.

The `isolation_parameters` dictionary is as follows:
The `isolation_parameters` dictionary definition depends on the model used. The currently supported models are Microfaune and BirdNET-Lite.

The Microfaune `isolation_parameters` dictionary is as follows:

``` python
isolation_parameters = {
"model" : "microfaune",
"technique" : "",
"threshold_type" : "",
"threshold_const" : 0.0,
Expand All @@ -43,10 +46,32 @@ isolation_parameters = {
"chunk_size" : 0.0,
}
```

The `technique` parameter can be: Simple, Stack, Steinberg, and Chunk. This input must be a string in all lowercase.
The `threshold_type` parameter can be: median, mean, average, standard deviation, or pure. This input must be a string in all lowercase.

The remaining parameters are floats representing their respective values.
The remaining parameters are floats representing their respective values.

<br>

The BirdNET-Lite `isolation_parameters` dictionary is as follows:

``` python
isolation_parameters = {
"model" : "birdnet",
"output_path" : "",
"lat" : 0.0,
"lon" : 0.0,
"week" : 0,
"overlap" : 0.0,
"sensitivity" : 0.0,
"min_conf" : 0.0,
"custom_list" : "",
"filetype" : "",
"num_predictions" : 0,
"write_to_csv" : False
}
```

<!-- IsoAutio.py file -->

Expand All @@ -56,7 +81,7 @@ The remaining parameters are floats representing their respective values.
### [`isolate`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

This function is the wrapper function for all the audio isolation techniques, and will call the respective function based on its parameters.
This function is the wrapper function for all of Microfaune's audio isolation techniques, and will call the respective function based on `isolation_parameters` "technique" key.

| Parameter | Type | Description |
| --- | --- | --- |
Expand Down Expand Up @@ -127,7 +152,7 @@ Usage: `simple_isolate(local_scores, SIGNAL, SAMPLE_RATE, audio_dir, filename,is
### [`stack_isolate`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

This function uses a technique created by Jacob Ayers. Attempts to produce automated annotations of an audio clip baseon local score array outputs from a neural network. It is called by the `isolate` function when `isolation_parameters['technique'] == stack`.
This function uses a technique created by Jacob Ayers. Attempts to produce automated annotations of an audio clip based on local score array outputs from a neural network. It is called by the `isolate` function when `isolation_parameters['technique'] == stack`.

| Parameter | Type | Description |
| --- | --- | --- |
Expand All @@ -146,7 +171,7 @@ Usage: `stack_isolate(local_scores, SIGNAL, SAMPLE_RATE, audio_dir, filename,iso
### [`chunk_isolate`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

This function uses a technique created by Jacob Ayers. Attempts to produce automated annotations of an audio clip baseon local score array outputs from a neural network. It is called by the `isolate` function when `isolation_parameters['technique'] == chunk`.
This function uses a technique created by Jacob Ayers. Attempts to produce automated annotations of an audio clip based on local score array outputs from a neural network. It is called by the `isolate` function when `isolation_parameters['technique'] == chunk`.

| Parameter | Type | Description |
| --- | --- | --- |
Expand All @@ -165,7 +190,7 @@ Usage: `chunk_isolate(local_scores, SIGNAL, SAMPLE_RATE, audio_dir, filename,iso
### [`generate_automated_labels`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

This function applies the isolation technique determined by the `isolation_parameters` dictionary accross a whole folder of audio clips.
This function generates labels across a folder of audio clips determined by the model and other parameters specified in the `isolation_parameters` dictionary.

| Parameter | Type | Description |
| --- | --- | --- |
Expand All @@ -180,6 +205,41 @@ This function returns a dataframe of automated labels for the audio clips in aud

Usage: `generate_automated_labels(audio_dir, isolation_parameters, manual_id, weight_path, Normalized_Sample_Rate, normalize_local_scores)`


### [`generate_automated_labels_microfaune`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

This function is called by `generate_automated_labels` if `isolation_parameters["model"]` is set to `microfaune`. It applies the isolation technique determined by the `isolation_parameters` dictionary across a whole folder of audio clips.

| Parameter | Type | Description |
| --- | --- | --- |
| `audio_dir` | string | Directory with wav audio files |
| `isolation_parameters` | dict | Python Dictionary that controls the various label creation techniques. |
| `manual_id` | string | controls the name of the class written to the pandas dataframe |
| `weight_path` | string | File path of weights to be used by the RNNDetector for determining presence of bird sounds.
| `Normalized_Sample_Rate` | int | Sampling rate that the audio files should all be normalized to.
| `normalize_local_scores` | boolean | Set whether or not to normalize the local scores.

This function returns a dataframe of automated labels for the audio clips in audio_dir.

Usage: `generate_automated_labels_microfaune(audio_dir, isolation_parameters, manual_id, weight_path, Normalized_Sample_Rate, normalize_local_scores)`


### [`generate_automated_labels_birdnet`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

This function is called by `generate_automated_labels` if `isolation_parameters["model"]` is set to `birdnet`. It generates bird labels across a folder of audio clips using BirdNET-Lite given the isolation parameters.

| Parameter | Type | Description |
| --- | --- | --- |
| `audio_dir` | string | Directory with wav audio files |
| `isolation_parameters` | dict | Python Dictionary that controls the various label creation techniques. |

This function returns a dataframe of automated labels for the audio clips in audio_dir.

Usage: `generate_automated_labels_birdnet(audio_dir, isolation_parameters)`


### [`kaleidoscope_conversion`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)
*Found in [`IsoAutio.py`](https://github.com/UCSD-E4E/PyHa/blob/main/PyHa/IsoAutio.py)*

Expand Down Expand Up @@ -259,7 +319,7 @@ This function takes in a dataframe of efficiency statistics for multiple clips a
| --- | --- | --- |
| `statistics_df` | Dataframe | Dataframe of statistics value for multiple audio clips as returned by the function automated_labelling_statistics. |

This function returns a dataframe of global statistics for the multiple audio clips' labelling..
This function returns a dataframe of global statistics for the multiple audio clips' labelling.

Usage: `global_dataset_statistics(statistics_df)`

Expand Down Expand Up @@ -340,15 +400,15 @@ Usage: `dataset_Catch(automated_df, manual_df)`
| --- | --- | --- |
| `automated_df` | Dataframe | Dataframe of automated labels for multiple classes. |
| `human_df` | Dataframe | Dataframe of human labels for multiple classes. |
| `stats_type` | String | String that determines which statistis are of interest. |
| `stats_type` | String | String that determines which statistics are of interest. |
| `threshold` | float | Defines a threshold for certain types of statistics. |

This function returns a dataframe with clip overlap statistics comparing automated and human labeling for multiple classes

The `stats_type` parameter can be set as follows:
| Name | Description |
| --- | --- |
|`"IoU"`| Default. Compares the intersection over union of automated annotationswith respect to manual annotations for individual clips. |
|`"IoU"`| Default. Compares the intersection over union of automated annotations with respect to manual annotations for individual clips. |
|`"general"` | Consolidates all automated annotations and compares them to all of the manual annotations that have been consolidated across a clip. |

Usage: `clip_statistics(automated_df, manual_df, stats_type, threshold)`
Expand Down Expand Up @@ -432,10 +492,12 @@ Usage: `plot_bird_label_scores(automated_df,human_df,save_fig)`

All files in the `microfaune_package` directory are from the [microfaune repository](https://github.com/microfaune/microfaune), and their associated documentation can be found there.

All files in the `birdnet_lite` directory are from a [modified version](https://github.com/UCSD-E4E/BirdNET-Lite) of the [BirdNET Lite repository](https://github.com/kahst/BirdNET-Lite), and their associated documentation can be found there.

## Examples
*These examples were created on an Ubuntu 16.04 machine. Results may vary between different Operating Systems and Tensorflow versions.*

Examples were created using this dictionary for the `isolation_parameters`:
Examples using Microfaune were created using this dictionary for the `isolation_parameters`:

```json
isolation_parameters = {
Expand Down Expand Up @@ -538,7 +600,7 @@ statistics_df = automated_labeling_statistics(automated_df,manual_df,stats_type
![image](https://user-images.githubusercontent.com/44332326/127575467-cb9a8637-531e-4ed7-a15e-5b5b611ba92c.png)


### Function that takes the statistical ouput of all of the clips and gets the equivalent global scores
### Function that takes the statistical output of all of the clips and gets the equivalent global scores
```python
global_dataset_statistics(statistics_df)
```
Expand All @@ -564,7 +626,7 @@ stats_df = automated_labeling_statistics(automated_df,manual_df,stats_type = "Io
```
![image](https://user-images.githubusercontent.com/44332326/127575771-9866f288-61cf-47c5-b9de-041b49e583d1.png)

### Function that takes the output of dataset_IoU Statistics and ouputs a global count of true positives and false positives, as well as computing common metrics across the dataset
### Function that takes the output of dataset_IoU Statistics and outputs a global count of true positives and false positives, as well as computing common metrics across the dataset
```python
global_stats_df = global_statistics(stats_df)
```
Expand Down

1 comment on commit e6190c6

@JacobGlennAyers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me!

Please sign in to comment.