next up previous contents
Next: Further Work Up: No Title Previous: Experiments on the Fault


The proposed architecture proved to be useful for many different real world data sets. The experiments showed that the model performed satisfactory for all of the tested application domains. In some of the experiments the MLP or LNN was superior to the suggested model; but in these cases the performance of the modular network was close to that of the better performing network.

In most data sets with a medium input dimension (12-320 variables) the new model showed similar memorization and generalization performance as a Backpropagation trained monolithic multilayer feedforward network.

For larger input dimensions the modular architecture proved to be superior over the monolithic MLFFN, both for memorization as well as generalization. In particular the generalization ability on large continuous input vectors was significantly better. This can be explained by the combined generalization mechanism in the new model.

The theoretical limitation for learning statistically neutral data sets did not appear in the test cases used. This problem seems to be unlikely in practice, particularly when dealing with a large number of inputs.

An important improvement is the speed of training in the proposed network. The training process was much faster due to the smaller number of weight connections and the splitting of the input vector in smaller parts. This effect was particularly significant in problems with high dimensional input spaces.

The training of the decision network proved to be very easy; often very few cycles were needed to reach a sufficiently small error value. This can be explained by the resulting training set for the decision network. The output of each module in the input layer is the class number of the input vector and the task of the decision network is reduced to a very simple mapping. For some experiments the decision module converged in less than 20 cycles.

Throughout the experiments it appeared that nearly any configuration of the modular neural network could be trained to convergence. The individual input modules proved very robust, due to their small size. The modular approach delivered a significant advantage for problems with large input vectors, where it was very difficult to find a parameter set for a monolithic network.

All modules in the input layer are mutually independent, which makes parallel training of the input layer straight forward. Since there is no communication during the learning process, this also allows a distributed implementation without additional problems.

Drawing on the modular nature of the human nervous system, an attempt to explore modularity was made in this project. The results which are far from comprehensive are very promising. The findings during the experiments are encouraging to search for new artificial neural networks based on parallelism and modularity.

next up previous contents
Next: Further Work Up: No Title Previous: Experiments on the Fault

Albrecht Schmidt
Mit Okt 4 16:45:34 CEST 2000