![]() |
Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications
by Lakhmi C. Jain; N.M. Martin CRC Press, CRC Press LLC ISBN: 0849398045 Pub Date: 11/01/98 |
Previous | Table of Contents | Next |
Neural Net Learning
The equivalent error, dkout, at the output layer is
where ok is the output of the output neuron k, tk is the desired output of the output neuron k, and f ′(netk) is the derivative which is unity for layers 2 and 3 neurons as mentioned above.
The weight modification equation for the weights between the hidden layer and output layer, Wjk, is
where ε is the learning rate, Wjk is the weight between hidden layer neuron j and output layer neuron k, and oj is the output from the hidden layer neuron j.
The general equation for the equivalent error in the hidden layer is
However, for the fuzzification layer in Figure 4, the equivalent error is different, as for the middle layer, the netpj is
where oi is the output of the input layer neuron i.
Thus, for the input layer (Figure 3), the equivalent error expression becomes
where both i and k are indices in the input layer and j is the index in the hidden layer; the summation is over j and product is over k.
The generated fuzzy rules and membership functions are processed using the neural net based nonheuristic fuzzy logic algorithm as described below.
Antecedent Processing
We are using multiplication of the membership functions, as opposed to minimum of the membership functions, in processing the antecedents. Thus the equation to combine two antecedents is
where υc is the membership function of the combinations of the membership functions υa and υb. Use of multiplication as dictated by the neural net significantly improves the results [9].
Rule Format
The rule format is
IF Input 1 is Low AND Input 2 is Low THEN the output is X
where X is a number. Thus, instead of using an output membership function, we are using singletons.
Defuzzification / Rule Evaluation
Consider the following equation for the defuzzification:
where Output is the final defuzzified output which includes the contribution from all the rules as represented by layer 2 and y represents the output of layer 2 neurons.
Clearly, we get a defuzzification which exactly matches the neural net behavior. This defuzzification is also simpler, as it does not use the division operation used in the conventional COG (Center of Gravity) defuzzification algorithm. Another point to note is that the proposed defuzzification is actually a rule evaluation. Since the output of the rule is singleton, we actually do not need a defuzzification.
Figure 5 shows a recurrent neural network based neural fuzzy system. This is based on the NeuFuz architecture (Figure 3) by adding recurrent connections in the hidden layer (i.e., rule base).
Figure 5a Recurrent neural net using the architecture of NeuFuz shown in Figure 3. Recurrent connections are added in the hidden layers. D represents a unit delay.
The recurrent connections in the hidden layer (Figure 5) use multiple delays. These recurrent connections are used both during learning and recall phases. In calculating the output of a neuron in layer 2, outputs of layer 1 neurons connected to layer 2 neurons are multiplied with the feedback signals of layer 2 neuron with appropriate delays. Recurrency can also be added to the input neurons to exploit more information about context.
Figure 5b Hidden layer of Figure 5a is redrawn showing all possible recurrent path delays and weights. Only one neuron is shown for simplicity. D represents a unit delay. Weight WRj,n represents feedback weight of j-th neuron in layer 2 with n delays. The recurrent weights are modified during the learning process using back propagation as described in Section 5.2.1. Recurrent weights are also used in the recall mode calculations.
Using similar analysis as we did for NeuFuz in Section 5.1.2, we derive the following equations:
The netpjhidden(t), at the hidden layer, is
where | netpjhidden(t - m) = netpjhidden with delay m with respect to current time t, |
netpjhidden(t) = layer 2 (hidden layer) output at time t, and | |
Oi = output of the i-th neuron in layer 1. |
We have used linear neurons with a slope of unity for the hidden (layer 2) and output (layer 3) layer neurons. Thus, the equivalent error at the output layer is
where ok is the output of the output neuron k
The general equation for the equivalent error at the hidden layer neurons using Back Propagation model is
However, for the fuzzification layer in Figure 5a, the equivalent error is different, as for the hidden layer we have product (instead of sum) neuron.
Thus, for the input layer in Figure 5a, the equivalent error expression (after incorporating the effect of the recurrent paths) becomes ([11, 12, 13])
where both i and k are indices in the input layer and j and m are the indices in the hidden layer. Also, the value of k is different from the value of i.
The weight update equation for any layer (except recurrent path in layer 2) is
For the feedback connections (layer 2), oi is denoted by oirecurrent and is the netpjhidden(t-m) multiplied by WRj,m (which corresponds to the weight in the path with m delays). We add a bias term of unity to oirecurrent to help convergence.
Thus, oirecurrent is given by
and dj is the equivalent error in the hidden layer.
Previous | Table of Contents | Next |