![]() |
Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications
by Lakhmi C. Jain; N.M. Martin CRC Press, CRC Press LLC ISBN: 0849398045 Pub Date: 11/01/98 |
Previous | Table of Contents | Next |
Weight Update for Recurrent Path
The weight update equation for the recurrent path is somewhat different. For simplicity, consider unit delay (i.e., m = 1). For better clarification, the neuron in Figure 5b is redrawn with recurrent connection showing 2 parts: P1 and P2 (Figure 6). The output of P1 is In and the output of P2 is netpjhidden as described before. Thus, we have
Let fn = netpjhidden. Then, the partial derivative of fn with respect to WRj,1 becomes
Using standard gradient descent and chain rules, it can easily be shown that
Figure 6 A neuron from layer 2 (Figure 5) is redrawn showing 2 parts: P1 and P2. The output of P1 is In and the output of P2 is netpjhidden as used in Figure 5a.
It is to be noted that the temporal information is accumulated by the recurrent path. Hence updates need to be done in batch mode. Thus, the above weight changes are calculated and stored for each pattern, but weight update is done only after all patterns are applied and by summing all ΔWij for each pattern. This process is then repeated until convergence is obtained. Thus, we follow schemes similar to real-time learning as reported in [7]. Accumulation of information also takes place in the recall mode, and it has corresponding impact on the fuzzy rules as described below. It should be noted here that the net only remembers the recent past, i.e., it gradually forgets the old context.
Like NeuFuz, the first layer neurons (Figure 5a) include the fuzzification process whose task is to match the values of the input variables against the labels used in the fuzzy control rule. The first layer neurons and the weights between layer I and layer 2 are also used to define the input membership functions. The layer 2 neurons represent fuzzy rules. However, considering the recurrent connection (Figure 5a), the fuzzy rule format is accordingly changed to incorporate the recurrency. Thus, the recurrent fuzzy rule format is
IF input 1 is Low AND input 2 is Low AND previous output is Y1 | ||
THEN the new output is X | ||
[recurrent fuzzy rule with one delay] |
considering recurrent connection with unit delay. Y1 is a singleton like X. However, it can have two forms:
Clearly, the previous output information is incorporated in the rules antecedent and is represented either by weights WRj,1 or by (WRj,1·netpjhidden(t-1)), although the latter makes more sense from temporal information point of view. Thus, fuzzy processing is different for these two cases, although both would essentially yield the same results.
For recurrent connection with n-delays, the recurrent fuzzy rule becomes,
IF input 1 is Low AND input 2 is Low | |||
AND last output is Y1 | |||
AND 2nd last output is Y2 | |||
AND 3rd last output is Y3 | |||
... | |||
... | |||
AND n-th last output is Yn | |||
THEN the new output is X | |||
[recurrent fuzzy rule with n delays] |
To ensure that we do not lose any information (or accuracy) in mapping neural net to fuzzy logic, a one-to-one mapping is used (for both versions of Y singletons). This approach has the key advantage of generating fuzzy logic rules and membership functions that meet a pre-specified accuracy. To ensure one-to-one mapping between the neural net and the fuzzy logic, neural network based fuzzy logic algorithms are used, as described for the NeuFuz.
Recurrency can also be used in the inputs. A typical fuzzy logic rule considering single delay would look like
IF input 1 is Low AND previous Input 1 is Medium | |||
AND input 2 is Low AND Previous input 2 is Low | |||
AND last output is Y1 | |||
AND 2nd last output is Y2 | |||
...y | |||
... | |||
AND n-th last output is Yn | |||
THEN the new output is X | |||
[recurrent fuzzy rule with delays at inputs and outputs] |
Extension to multiple delay case is straightforward.
Use of too many feedback delays complicates fuzzy rules, uses more memory, and does not necessarily improve the solution. This is because the context information is useful only when the signals in the near vicinity belong to the same membership function (i.e., when autocorrelation function does not decay rapidly). With a large number of delays in the feedback path, all distant signals will not belong to the current membership class and so the recurrent information will not be useful. Typical values for number of delays are from 1 to 5. As shown in Table 1, context information with more than one delay does not help much. Also, note that the recurrency significantly improves the convergence time.
Accuracy | Number of cycles to converge | ||||
---|---|---|---|---|---|
1D | 2D | 3D | 4D | NeuFuz (no recurrency) | |
0.01 | 9 | 8 | 8 | 7 | 1036 |
0.001 | 172 | 160 | 160 | 158 | 5438 |
Previous | Table of Contents | Next |