|
Otto-von-Guericke University Magdeburg | |||
|
School of Computer Science | |||
|
Institute of Knowledge Processing and Language Engineering | |||
|
Information Retrieval Group | |||
Over the last few decades, neural networks and fuzzy systems have established their reputation as alternative approaches to information processing. Both have certain advantages over classical methods, especially when vague data or prior knowledge is involved. However, their applicability suffered from several weaknesses of the individual models. Therefore, combinations of neural networks with fuzzy systems have been proposed, where both models complement each other. These so-called neural fuzzy or neuro-fuzzy systems allow to overcome some of the individual weaknesses and offer some appealing features.
Using fuzzy set theory it is easily to model the ‘fuzzy’ boundaries of linguistic terms by introducing gradual memberships. In contrast to classical set theory, in which an object or a case either is a member of a given set (defined, e.g., by some property) or not, fuzzy set theory makes it possible that an object or a case belongs to a set only to a certain degree [1]. Interpretations of membership degrees include similarity, preference, and uncertainty [2]. They can state how similar an object or case is to a prototypical one, they can indicate preferences between sub optimal solutions to a problem, or they can model uncertainty about the true situation, if this situation is described in imprecise terms. In general, due to their closeness to human reasoning, solutions obtained using fuzzy approaches are easy to understand and to apply. Due to these strengths, fuzzy systems are the method of choice, if linguistic, vague, or imprecise information has to be modeled [3].
The fuzzy systems - as used here - are based on if-then rules. The antecedent of a rule consists of fuzzy descriptions of input values, and the consequent defines a - possibly fuzzy - output value for the given input. The benefits of these fuzzy systems lie in the suitable knowledge representation. But problems may arise when fuzzy concepts have to be represented by concrete membership degrees, which guarantee that a fuzzy system works as expected.
A fuzzy system can be used to solve a problem if knowledge about the solution is available in the form of linguistic if-then rules. By defining suitable fuzzy sets to represent linguistic terms used within the rules, a fuzzy system can be created from these rules. There is no formal model of the problem of interest and no training data required.
Neural networks are systems that try to make use of some of the known or expected organizing principles of the human brain. They consist of a number of independent, simple processors - the neurons. These neurons communicate with each other via weighted connections. At first, research in this area was driven by neurobiological interests. The modeling of single neurons and the so-called „learning rules“ for modifying synaptic weights were the initial research topics. Modern research in neural networks considers the development of architectures and learning algorithms, and examines the applicability of these models to information processing tasks. Although, there are still many researchers who are modeling biological neural networks by artificial neural networks to learn more about the structure of the human brain and the way it works, biological plausibility is usually neglected and only the problem of information processing with artificial neural networks is considered. These models have in common that they are based on rather simple processing units or neurons exchanging information via weighted connections.
Different types of neural networks can solve different problems, like pattern recognition, pattern completion, determining similarities between patterns or data - also in terms of interpolation or extrapolation - and automatic classification (see, e.g. [4, 5]). Learning in neural networks means to determine a mapping from an input to an output space by using example patterns. If the same or similar input patterns are presented to the network after learning, it should produce an appropriate output pattern.
Neural networks can be used if training data is available. It is not necessary to have a mathematical model of the problem of interest, and there is no need to provide any form of prior knowledge. On the other hand the solution obtained from the learning process usually cannot be interpreted. Although there are some approaches to extract rules from neural networks, see e.g. [6], most neural network architectures are black boxes. They cannot be checked whether their solution is plausible, i.e. their final state cannot be interpreted in terms of rules. This also means that a neural network usually cannot be initialized with prior knowledge if it is available, and thus the network must learn from scratch. The learning process itself can take very long, and there is usually no guarantee of success.
The basic idea of combining fuzzy systems and neural networks is to design an architecture that uses a fuzzy system to represent knowledge in an interpretable manner and the learning ability of a neural network to optimize its parameters. The drawbacks of both of the individual approaches - the black box behavior of neural networks, and the problems of finding suitable membership values for fuzzy systems - could thus be avoided. A combination can constitute an interpretable model that is capable of learning and can use problem-specific prior knowledge. Therefore, neuro-fuzzy methods are especially suited for applications, where user interaction in model design or interpretation is desired. Discussions and an overview of current approaches can be found, e.g. in [7, 8, 9].
Despite of the research that has already been done in the area of neuro-fuzzy systems the recurrent variants of this architecture are still rarely studied, although the - most likely - first paper on recurrent fuzzy systems had already been published 1994 [10]. In contrast to pure feed-forward architectures, that have a static input-output behavior, recurrent models are able to store information of the past (e.g. prior system states) and are thus more appropriate for the analysis of dynamic systems. If pure feed-forward architectures are applied to these types of problems (e.g. prediction of time series data or physical systems), the obtained system data usually has to be preprocessed or restructured to map the dynamic information appropriately. Meanwhile, some recurrent neuro-fuzzy systems have been proposed. In [11] we have presented some results of our work on this subject.
[1] L. A. Zadeh, Fuzzy Sets, Information and Control, 8, pp. 338-353, 1965.
[2] D. Dubois, H. Prade, and R. R. Yager, Information Engineering and Fuzzy Logic, In: Proc. 5th IEEE International Conference on Fuzzy Systems (FUZZ-IEEE`96, New Orleans, LA, USA), pp. 1525-1531, IEEE Press, Piscataway, NJ, USA, 1996.
[3] R. Kruse, C. Borgelt, and D. Nauck, Fuzzy Data Analysis: Challenges and Perspectives, In: Proc. IEEE Int. Conf. on Fuzzy Systems 1999 (FUZZIEEE99), pp. 1211-1216, Seoul, 1999.
[4] S. Haykin, Neural Networks, Prentice Hall, New Jersey, 1994.
[5] R. Rojas, Neural Networks - A Systematic Introduction, Springer-Verlag, Berlin, New York, et al., 1993.
[6] A. B. Tickle, F. Maire, G. Bologna, R. Andrews and J. Diederich, Lessons from Past, Current Issues and Future Research Directions in Extracting the Knowledge Embedded in Artificial Neural Networks, In: S. Wermter and R. Sun (eds.), Hybrid Neural Systems, pp. 226 - 239, Springer, 2000.[7] C.-T. Lin, and C.-C. Lee, Neural Fuzzy Systems. A Neuro-Fuzzy Synergism to Intelligent Systems, Prentice Hall, New York, 1996.
[8] D. Nauck, F. Klawonn, and R. Kruse, Foundations of Neuro-Fuzzy Systems, Wiley, Chichester, 1997.
[9] A. Klose, A. Nürnberger, D. Nauck, and R. Kruse. Data Mining with Neuro-Fuzzy Models, In: A. Kandel, H. Bunke, M. Last (Eds.), Data Mining and Computational Intelligence, pp. 1-36, Physica-Verlag, 2001.
[10] V. Gorrini, and H. Bersini, Recurrent Fuzzy Systems, In: Proceedings of the 3rd Conference on Fuzzy Systems (FUZZ-IEEE 94), IEEE, Orlando, 1994.
[11] A. Nürnberger, A Hierarchical Recurrent Neuro-Fuzzy System, In Proc. of Joint 9th IFSA World Congress and 20th NAFIPS International Conference, pp. 1407-1412, IEEE, 2001.
© 2001, Andreas Nürnberger
