next up previous contents
Next: Binary Pattern Recognition Up: Experimental Evaluation Previous: Experimental Evaluation

Test Methodology

The test sets used are either obtained from neural network benchmark data bases [prob96], [ucim96], or [nnbc96], or were developed from raw data.

The following methodology was used for the tests:

Two data sets were prepared: a training set and a test set. Some data sets were split randomly in two sets of equal size (plus/minus one if the number of instances was odd). For other tests selected instances were used for training and the others for measuring the performance.

The networks first learnt the training set and then the recognition or prediction performance was calculated on both sets. The result of the training set is denoted by PM and it is a measure for the memorization capabilities. The performance on the test set is given as PG and indicates the generalization abilities of the trained network.

Depending on the test, different network structures, various numbers of hidden units and network layers, or alternative network parameters were investigated. Furthermore the influence of the number of training steps and of the error value as stop condition were examined.

The time specified in the tests is measured on the same system for one experiment. Due to the hardware availability different experiments were carried out on different systems, therefore a direct comparison between different experiments is not valid. The notation x'y'' stays for x minutes and y seconds. No parallel training was used.

In the experiment to investigate the fault tolerance of the modular neural network architecture a different methodology was used, this is described in section 7.5.


next up previous contents
Next: Binary Pattern Recognition Up: Experimental Evaluation Previous: Experimental Evaluation

Albrecht Schmidt
Mit Okt 4 16:45:34 CEST 2000