Next: Conference Paper
The project showed a lot of ways into the world of artificial modular
neural networks. The time for experiments was
very limited and many possible approaches were
left unexplored. In this section some ideas for further work are given.
The aim of this project was to answer the general question:
Is the architecture useful?.
The next step should be to investigate further issues:
- How do the following parameters influence the performance of the
This test should be made using a single data set.
Because of the huge number of possibilities using a
genetic algorithm might be appropriate.
- the number of inputs per module
- the number of training steps in each module
- the learning parameters
- the number of neurons in the modules
- the intermediate representation
- The mapping between the input variables and the modules can be
experiments assigned this mapping randomly. Making connections based
additional information on the domain could improve the performance.
Statistical methods could be used to calculate a mapping from
the inputs to the modules that the class information is maximal
for all modules. This could also be used to avoid training
modules with statistically neutral data sets.
- A more general architecture, such as a pyramidal structure, can be
investigated; it is also to consider if a different intermediate
representation is more appropriate.
- Different training algorithms can be explored. The effect of
training the decision network with noise in the input
vectors can be investigated.
- The behaviour of the modular architecture for data sets
with a larger number of classes may be analyzed.
- In the experiment it was shown that the decision network can be
retrained very quickly. This feature could be used to adapt the
network to a changing environment. To investigate this the whole
network should be trained using the original data; if the data
set changes only the decision network is retrained.
The performance on the new data set is then measured.
- A theoretical investigation in the generalization behaviour of the
network can give additional evidence for a suitable application domain.
Figure 8.1: A Suggested Architecture for Further Experiments.
- Using a more flexible structure could improve the performance. To explore
this it is suggested to use modules of different size in the input layer
and connect each input to two modules, see Figure 8.1.
The resulting number of possible network architectures
is huge therefore a genetic algorithm seems to be a reasonable
method to find a good structure for a particular training set.
To avoid training of all networks in a population to calculate the fitness
the statistical knowledge of the input variables could
be used instead. To define a measure for the class information carried by a
subset of input variables is probably a way to create a fitness function
that can be evaluated quickly.
- If the system will be used for real applications a parallel
implementation would certainly be desirable.
- To make the usage of the modular neural network easier, especially
for users with a non-computing background, a graphical user interface
should be provided.
Next: Conference Paper
Mit Okt 4 16:45:34 CEST 2000