7.4. The Neural Networks

The code used in the implementation of the Multi-Layer Perceptron has the advantage of being completely flexible in structure, with the added option of recurrency. This has meant that swapping between diverse network structures has not been a difficult task.

7.4.1. Backpropagation

During the training of the some of the neural networks, overfitting was observed. However, it is conceivable that the degree of overfitting has been kept to a minimum due to the way the backpropagation algorithm has been implemented. A backpropagation pass is only performed on patterns that cannot be correctly identified, this having the advantage of speeding up backpropagation as the network converges.
The type of noise inherent in the subject patterns is usually the product of some undesirable sound source present in the original recording and will therefore not be strictly random. It will also be the same for every identical training pattern although different across similar types. Section 3.5.1 suggests the introduction of further randomness in the form of Gaussianly distributed noise presented at the inputs. This technique could yield further improvements in error.
In all, the time taken to perform backpropagation is not a factor of consideration in the finished application.
This is because the recognition software would be supplied fully trained (it is specific to one task). Online training is not advised due to the fact that input data can not be validated and that users could easily make errors when identifying their training set.

7.4.2. Recurrent networks

It appears that the best result is in the use a feed forward multi-layer perceptron of 60:25:5 structure.
One would have thought that by splitting the recognition segment into two and introducing a temporal element, the recognition rate would have been improved further. However, the shortcomings of the recurrent network could just be a product of the surrounding implementation. With hindsight a better method could have been to split the two recognition segments in a different way. A shorter first segment could serve the purpose of observing the initial transient of the drum and make distinctions between stick hits (in a drum kit) and hand hits (in congas) whilst a second much longer segment serves a purpose similar to that of the static feed forward network.