next up previous contents
Next: Multiple and Modular Artificial Up: Modularity in ANN Previous: Modularity in ANN

Motivation: 4-Bit-Parity Problem

 figure457
Figure 4.4:  A Modular Solution for the 4-Bit-Parity Problem.

This is a small example to investigate the advantages of a modular structure over a monolithic network. A monolithic multilayer ANN can learn the 4-bit-Parity Problem. Due to the structure of this data set the learning will take rather long.

In Figure 4.4 a modular design and the adjusted training set is depicted. In this example it is obvious that it is much easier to train a single module (with 6 weights and 4 data tuples) and replicate this three times than to train the bigger monolithic MLP (with 26 weights and 16 data tuples). The task performed by the trained networks will be the same. The truth-table for the modular solution it is shown in Table 4.1.

 

x1 x2 x3 x4 y1 y2 y3 = ydesired
0 0 0 0 1 1 1
0 0 0 1 1 0 0
0 0 1 0 1 0 0
0 0 1 1 1 1 1
0 1 0 0 0 1 0
0 1 0 1 0 0 1
0 1 1 0 0 0 1
0 1 1 1 0 1 0
1 0 0 0 0 1 0
1 0 0 1 0 0 1
1 0 1 0 0 0 1
1 0 1 1 0 1 0
1 1 0 0 1 1 1
1 1 0 1 1 0 0
1 1 1 0 1 0 0
1 1 1 1 1 1 1
Table 4.1:  The 4-Bit-Parity Problem: The Truth-Table.

In this example it can be seen that the modular approach is superior to a monolithic network.

The main problem remains: how to chose modules, how to structure the problem? In the example above human design knowledge was used to restructure the problem and the data set for the modular solution. This approach is only applicable for a very limited domain of problems.

In [boer92] a genetic algorithm was used to find an optimal network architecture. The problem is that the number of possible network structures is huge even for very simple problems. With no structural limitations to the network architecture the complexity is too great. Therefore the problem can not be solved with reasonable computational power.

The use of a genetic algorithm offers enormous potential if the search space could be limited. This approach is biologically very plausible. It is very likely that connections in the brain have developed as a result of the evolutionary process.


next up previous contents
Next: Multiple and Modular Artificial Up: Modularity in ANN Previous: Modularity in ANN

Albrecht Schmidt
Mit Okt 4 16:45:34 CEST 2000