A Modular Neural Network Architecture
with Additional Generalization Abilities
for High Dimensional Input Vectors
A thesis submitted to the Manchester Metropolitan University in partial fulfillment of the requirements for the degree of Master of Science in Computing by:
Manchester Metropolitan University,
Department of Computing,
I owe a debt of gratitude to my supervisor, Dr Zuhair A. Bandar from whom I have learned much about neural networks. I wish to acknowledge his support and guidance throughout the project.
I am greatly thankful to Dave Shield for his efforts in introducing me to the mysteries of the English language while proof-reading this thesis.
Special thanks to my dear friend Petra Dollinger for her encouragement over the last six months.
I would also like to express thanks to my parents for their support during my stay in Manchester.
In this project a new modular neural network is proposed. The basic building blocks of the architecture are small multilayer feedforward networks, trained using the Backpropagation algorithm.
The structure of the modular system is similar to architectures known from logical neural networks. The new network is not fully connected and therefore the number of weight connections is much less than in a monolithic multilayer Perceptron.
The suggested training algorithm works in two stages and is easy to implement in parallel. Due to the used modular structure the training is very quick for large input vectors.
The modular architecture is designed to combine two different approaches of generalization known from connectionist and logical neural networks; this enhances the generalization ability, which is especially significant for a high dimensional input space.
An object-oriented implementation of the proposed model was written to simulate the behaviour.
The evaluation using different real world data sets showed that the new architecture is very useful for high dimensional input vectors. For certain domains the learning speed as well as the generalization performance in the modular system is significantly better than in a monolithic multilayer feedforward network.