The experiment of the fault tolerance of the system is another investigation in possible advantages of a modular architecture. For this test the Diabetes data set described in section 7.3.1 was used.
Figure 7.10: Tests on the Fault Tolerance of the System.
The network was used. 250 training steps were performed on the input layer of the modular neural network; this took 33 minutes and resulted in a root mean square error of the input layer of 0.184. Then 200 training steps in the decision network were made (10 minutes, root mean square error for the decision network of 0.238). The achieved performance on the training set was and the generalization performance on the test set was .
One of the five modules in the input layer was reset to random weights while the other four modules remained in the trained state. The performance of the partly damaged network was tested ( for the performance on the training set and for the performance on the test set).
Than the decision network was trained another 50 steps; this took in average about three minutes. Than the performance of the repaired network ( and ) was calculated.
This was repeated for all modules in the first layer, in each test only one module was reseted randomly, this is equal to about 18.5% of the weights.
The results are visualized in Figure 7.10. This result is very astonishing. The loss of about 10% performance can be retrained to nearly the original values with very little effort.
It is interesting that a similar effect can be observed in human neuro psychology. Patients with a partly damaged brain lose some of their abilities but most of them are able to relearn the things in a very short time [kala92].