Further Work

  The project showed a lot of ways into the world of artificial modular neural networks. The time for experiments was very limited and many possible approaches were left unexplored. In this section some ideas for further work are given.

The aim of this project was to answer the general question: Is the architecture useful?. The next step should be to investigate further issues:

• How do the following parameters influence the performance of the network:
  • the number of inputs per module
  • the number of training steps in each module
  • the learning parameters
  • the number of neurons in the modules
  • the intermediate representation
  This test should be made using a single data set. Because of the huge number of possibilities using a genetic algorithm might be appropriate.
• The mapping between the input variables and the modules can be explored; The experiments assigned this mapping randomly. Making connections based additional information on the domain could improve the performance. Statistical methods could be used to calculate a mapping from the inputs to the modules that the class information is maximal for all modules. This could also be used to avoid training modules with statistically neutral data sets.
• A more general architecture, such as a pyramidal structure, can be investigated; it is also to consider if a different intermediate representation is more appropriate.
• Different training algorithms can be explored. The effect of training the decision network with noise in the input vectors can be investigated.
• The behaviour of the modular architecture for data sets with a larger number of classes may be analyzed.
• In the experiment it was shown that the decision network can be retrained very quickly. This feature could be used to adapt the network to a changing environment. To investigate this the whole network should be trained using the original data; if the data set changes only the decision network is retrained. The performance on the new data set is then measured.
• A theoretical investigation in the generalization behaviour of the network can give additional evidence for a suitable application domain.

   
  Figure 8.1:  A Suggested Architecture for Further Experiments.
  

• Using a more flexible structure could improve the performance. To explore this it is suggested to use modules of different size in the input layer and connect each input to two modules, see Figure 8.1. The resulting number of possible network architectures is huge therefore a genetic algorithm seems to be a reasonable method to find a good structure for a particular training set.

  To avoid training of all networks in a population to calculate the fitness the statistical knowledge of the input variables could be used instead. To define a measure for the class information carried by a subset of input variables is probably a way to create a fitness function that can be evaluated quickly.

• If the system will be used for real applications a parallel implementation would certainly be desirable.
• To make the usage of the modular neural network easier, especially for users with a non-computing background, a graphical user interface should be provided.