deep-listening

Deep learning experiments for audio classification

A full write-up, including technical explanations and design decisions, as well as a summary of results achieved can be found within the associated Project Report.

This project consists of several Jupyter notebooks that implement deep learning audio classifiers.

1-us8k-ffn-extract-explore.ipynb

• this notebook contains code to extract and visualise audio files from the UrbanSound8K data set
• the feature extraction process uses audio processing metrics from the librosa library, which reduces each recording to 193 data points
• as the audio information is highly abstracted, (we can not process successive frames using a receptive field), these features are intended to be fed into a feed-forward neural network (FFN)

2-us8k-ffn-train-predict.ipynb

• this notebook contains the code to load previously extracted features and feed them into a 3-layer FFN, implemented using Tensorflow and Keras
• also included is some code to evaluate model performance, and to generate predictions from individual samples, demonstrating how a trained model would be used to identify the nature of live recordings

3-us8k-cnn-extract-train.ipynb

• this notebook extracts audio features suitable for input into a classic 2-layer Convolutional Neural Network (CNN)
• much more of the audio data is preserved in this approach, as the saved numpy feature data is over 2GB I haven't included it with this repository, but you can use the code in this notebook to extract it from the original UrbanSound8K data set

4-us8k-cnn-salamon.ipynb

• this notebook implements an alternative CNN, similar to one described by Salamon and Bello

5-ffbird-cnn.ipynb

• this notebook uses the Salamon and Bello CNN to process the FreeField1010 data set of field recordings, with the goal of recognising the presence of birdsong.
• the data set is not part of this repository, so if you want to run this code you'll need to download the data yourself (see instructions in the notebook)

7-us8k-rnn-extract-train.ipynb

• this uses a Recurrent Neural Network to classify Mel-frequency cepstral coefficients (MFCC) features.

Do get in touch if you've any questions, (me @ jaroncollis . com)