Figure 3.1: An Adaptive Node.
An adaptive node has the ability to memorize the desired output for certain input patterns. All inputs and outputs are binary; the states are denoted with `0' and `1'. An AN has a number of data inputs, a mode input, a teaching input, and an output, see Figure 3.1 according to [alek95, p2,].
The mode input determines whether the AN is learning or working. In the learning mode the node memorizes the input pattern (connected to the data inputs) together with the desired output (connected to the teaching input). In the working mode the response for a given input pattern (connected to the data input) is calculated and delivered to the output.
During the learning phase all the training pairs are presented to the AN; the node will memorize all pairs which are unambiguous. The response in the working phase will match the desired output for all correctly recognized training patterns.
For ambiguous and new patterns the AN will give the response `0' or `1' with equal probability (denoted here as `0/1'). Pairs are considered as ambiguous if the desired output can not be determined from the training set; e.g. one input pattern is given with different desired outputs. So far a single AN has only the function of a memory cell.
The search for a `similar' pattern is implemented by a firing rule; this provides the ability to generalize on the input space. The response for an unseen input pattern is the same as the desired output for the memorized pattern with the minimal Hamming distance (HD) to the applied input pattern. If the minimal HD is equal to both a 0-taught pattern and a 1-taught pattern then the response is `0/1'.
To illustrate the firing rule of an AN consider a training set with two 0-taught vector (0 0 0) and (1 1 1); and a 1-taught vector (0 1 0). In the following table the responses to all possible 3-bit input patterns are shown.
|0 0 0||0||0|
|0 0 1||0/1||0|
|0 1 0||1||1|
|0 1 1||0/1||0/1|
|1 0 0||0/1||0|
|1 0 1||0/1||0|
|1 1 0||0/1||0/1|
|1 1 1||0||0|
The task performed by a single adaptive node with firing rule is simple template matching. The patterns are classified according to the Hamming distance.
Figure 3.2: A Network of Adaptive Nodes.
A larger network consisting of ANs has additional generalization abilities. One possible network structure is shown in Figure 3.2. In the first layer the Hamming distance on small parts of the input pattern are calculated; in the second layer the results of the ANs of the first layer are combined to provide an overall result.
To calculate the HD of the whole input vector is often not appropriate as a large number of input samples would be necessary to enable generalization. If the Hamming distance is measured on smaller parts of the input vector it is much easier to obtain meaningful generalization from a small data set. It is more likely that subparts of a pattern are similar for the same class than that the whole vector is similar, see [alek95, p11,].