Analysis and Modeling of Learning Processes for Anticipatory Behavioral Control

Wolfgang Stolzmann & Joachim Hoffmann

The research project " Analysis and Modeling of Learning Processes for Anticipatory Behavioral Control" is founded by the German research foundation "Deutsche Forschungsgemeinschaft" (DFG). In this project we work on a learning mechanism, called Anticipatory Behavioral Control, that was introduced in Cognitive Psychology by Hoffmann in 1993. Our aim is to analyze and model this learning mechanism, i.e. to develop Anticipatory Behavioral Control into a learning algorithm. A basic learning algorithm was developed by Stolzmann (1997) and is called Anticipatory Classifier Systems (ACS). The goals of the research project are

to further develop the ACS,
to solve well known tasks in Machine Learning with an ACS,
to simulate experiments about learning in animals with an ACS,
to develop learning experiments that allow a comparison of human learning with learning in an ACS and
to apply ACS in learning robots.

Many observations in psychology have led to the learning theory of anticipatory behavioral control. The first principle is that learning can only occur if a need is present that wants to be satisfied. The observation of an operational drive in humans, additional to the known drive of satisfaction, showed that learning cannot rely solely on the satisfaction of the basic needs such as food, water, or sex. Rather, another type of need must exist in humans. In animal learning it was shown that rats are able to learn without any direct reinforcement (i.e. learn latently) (Blodgett, 1929, Tolman, 1932, Seward, 1949, Croake , 1971, ...). This and many other experiments showed that the need of accurate anticipations must exist in higher mammels. The theory of anticipatory behavioral control evolved out of this observation. It can be outlied as follows: First, a behavior R (=response) is always accomponied by the anticipated consequences E (=effect) and the actual given situation abstracted to a condition S (=stimulus). Second, a continuous comparison takes place between the anticipations and the successive perceptions. If the comparison was valid or invalid, it leads to an increase or decrease of the relation between the anticipation and the according stimulus-response relation, respectively. Finally, inaccurate anticipations lead to further differentiations of the conditions. In order to realize this learning theory in an artificial system, the basic necessity is that the anticipations must be represented in some form. This could be done by a recurrent neural network or by explicitly inking rules to possible effects represented by other rules. The most straight-forward approach, though, is to build S-R-E rules directly. This is done by the ACS. A detailed introduction to ACS is given by Stolzmann (2000).

Publications

Wolfgang Stolzmann (1998): Untersuchungen zur Adäquatheit des Postulats einer antizipativen Verhaltenssteuerung zur Erklärung von Verhalten mit ACSs. In : W. Krause & U. Kotkamp (Hrsg.), Intelligente Informationsverarbeitung. Wiesbaden: Deutscher Universitäts Verlag.
Word-File (525 KB), ps.gz-File (277 KB)
Wolfgang Stolzmann (1998): Anticipatory Classifier Systems. In Koza, John R., Banzhaf, Wolfgang, Chellapilla, Kumar, Deb, Kalyanmoym Dorigo, Marco, Fogel, David B., Garzon, Max H., Goldberg, David E., Iba, Hitoshi, and Riolo, Rick. (editors). Genetic Programming 1998: Proceedings of the Third Annual Conference, July 22-25, 1998, University of Wisconsin, Madison, Wisconsin, 658-664. San Francisco, CA: Morgan Kaufmann.
ps.gz-File (407 KB)
Wolfgang Stolzmann (1999): Latent Learning in Khepera Robots with Anticipatory Classifier Systems. In A.S. Wu (Ed.), Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Programm, pp. 290-297.
PS-file, PDF-file
Martin Butz & Wolfgang Stolzmann (1999): Action-Planning in Anticipatory Classifier Systems. In A.S. Wu (Ed.), Proceedings of the 1999 Genetic and Evolutionary Computation Conference Workshop Programm, pp. 242-249.
PS-file, PDF-file
Martin Butz, David E. Goldberg, & Wolfgang Stolzmann (1999): New Challenges for an Anticipatory Classifier System: Hard Problems and Possible Solutions. IlliGAL Report No. 99019. Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign.
Abstact and PS-file
Wolfgang Stolzmann (2000): An Introduction to Anticipatory Classifier Systems. In Lanzi, Stolzmann, & Wilson (Eds.), Learning Classifier Systems - An introduction to contemporary research. LNAI 1813, Heidelberg: Springer-Verlag.
Abstact
Wolfgang Stolzmann & Martin Butz (2000): Latent Learning and Action-Planning in Robots with Anticipatory Classifier Systems. In Lanzi, Stolzmann, & Wilson (Eds.), Learning Classifier Systems - An introduction to contemporary research. LNAI 1813, Heidelberg: Springer-Verlag.
Abstact
Butz, M., Goldberg, D.E., Stolzmann, W. (2000): Introducing a Genetic Generalization Pressure to the Anticipatory Classifier System Part 1: Theoretical Approach IlliGAL Report No. 2000005 . Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign.
Abstact and PS-file
Butz, M., Goldberg, D.E., Stolzmann, W. (2000): Introducing a Genetic Generalization Pressure to the Anticipatory Classifier System Part 2: Performance Analysis IlliGAL Report No. 2000006 . Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign.
Abstact and PS-file
Stolzmann, W., Butz, M.V., Hoffman, J., Goldberg, D.E. (2000) First Cognitive Capabilities in the Anticipatory Classifier System IlliGAL Report No. 2000008 . Illinois Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign.
Abstact and PS-file

generated: 24 February 2000; last update: 24 February 2000 / WST