4   Theories of classical conditioning and habituation

‘It is evident from the description given in the present lecture that we must distinguish in animals an elementary, from a higher type of analysis and synthesis’

Pavlov (1927;  p. 148)
 

Having briefly introduced modern cognitive theories of conditioning in the previous chapter, it is now possible to point to their limitations. The main one is that these cognitive theories, just as much as the theories which treat only with reflexes, or even only with synapses, ignore the factor of perceptual range, cognitive complexity, or whatever we wish to call it that distinguishes the psychology of a ganglion which once belonged to a slug from the capacities of the average dog. In many cases the cognitive theories are testable by reference to experiments with laboratory pigeons or rats, but it is not part of the theory that the explanation for the behaviour of Aplysia should be in any way different from the explanation of the behaviour of higher vertebrates. In some cases it is explicitly part of the theory that there is no important difference (Fantino and Logan, 1979; Macphail, 1982; Hall, 1983), while in others the question is ignored (Mackintosh, 1983). Part of the reason for this is that the experimental data indicate that there are a great many similarities in the overall form of behaviour observed under conditioning procedures with many different responses in widely varying species. This could mean that it is in fact theoretically correct to give the same explanation in all instances, but, alternatively, it

93

could mean that the superficially similar patterns of behaviour that is, gradual acquisition of a conditioned response, extinction of it when reinforcement ceases, spontaneous recovery after a rest period, and so on may be exhibited by biological systems of vastly different internal complexity and mode of operation. It is the latter view, the reader will have observed, which is taken in this text. This is not to deny the observed similarities in behaviour, but to argue for the possibility of different levels of association within and between the nervous systems of different grades of animal, which can be experimentally assessed provided differences in behaviour are counted up as well as similarities.

Levels of representation in habituation and classical conditioning

As a very similar point was made at the end of the chapter about habituation, it is perhaps better to discuss this issue in terms of the level of representation of stimuli, rather than the level of any association between them; it is not clear that any associations at all are necessarily involved in the habituation of response to a single stimulus. The term representation is now commonly used to refer to items of information that are involved in both human and animal learning, but without any clear agreed definition (e.g. Roitblat, 1982). I can offer no hard and fast definitions here, but the kinds of distinctions between levels of representation that could conceivably be supported by empirical evidence are illustrated, I hope, by Figure 4.1.

There should be little cause for confusion in saying that stimuli which are represented in the isolated spinal cord of a frog (Thompson and Glanzman, 1976), or the isolated abdominal ganglion of a slug (Carew and Kandel, 1973), are represented at a different level from stimuli which occupy the full attention of an intact and awake mammal. This would serve at least to differentiate level 1 from level 6 in Figure 4. 1. All the other distinctions therein are problematical to some degree, since there may be considerable overlap between, for instance, emotional, subcortical and cognitive

94

Figure 4.1 Levels of representation in habituation and classical conditioning              

Type of representation . Indicative phenomena in habituation Indicative phenomena in classical conditioning
gif
1  Synaptic stimulus-response connections Spinal or ganglionic habituation (Thompson and Spencer, 1966; Kandel, 1976) Anticipatory shift of spinal or ganglionic responses (Beggs et al., 1983; Ince et al., 1978; Carew et al. 1981, 1983)
gif
2  Autonomic nervous system and metabolic responses Many forms of physiological adaptation and fatigue; also systemic tolerance to drugs (Eikelboom and Stewart, 1982) Conditioned drug tolerance effects (Siegel, 1976; Stewart et al., 1984)
gif
3  Central emotional and motivational states Habituation to emotionally significant stimuli (Berlyne. 1960; Eysenck, 1976) Associative shifts of emotional response. forward and backward (Ohman et al.. 1975a; Miller. 1948: Keith-Lucas and Guttman. 1975)
gif
4  Peripheral perceptual systems Sensory adaptation (Hinde. 1970): some perceptual learning (e.g. Gibson and Walk, 1956) Perceptual shifts between or within stimulus modalities due to contiguity (e.g. McCullough. 1965, see p. 82)
gif
5  Central but subcortical mechanisms Habituation in decorticate mammals and lower vertebrates (e.g. Thompson and Gtanzman. 1976) Classical conditioning in decorticate mammals with simple stimuli (Pavlov. 1927: Oakley. 1979b)
gif
6  Central cognitive representations Human perceptual habituation and similar results in vertebrates (Sokolov, 1963. 1975); some perceptual learning by exposure and exploration (e.g. Gibson and Walk, 1956) Results with complex stimuli in intact mammals (Pavlov, 1927): similar results in people (Ohman et al., l975a; Levey and Martin. 1975); second-order conditioning and more elaborate results in higher vertebrates (e.g. Rescorla, 1978)
gif
(Note: Types of representation are not mutually exclusive; thus, for instance, human emotional conditioning may link specific autonomic responses and more general emotional evaluations to complex perceptions — 1, 3 and 6.)

95

levels of representation, but on the other hand it would probably be difficult to obtain widespread consent for the position that all the phenomena listed down the levels are identical. The main point of adopting a classificatory scheme of the kind shown in Figure 4. 1, which is clearly extremely provisional, is that it makes it easier to ask questions about any theoretical analysis of classical conditioning that may be put forward. In particular, of course, it prompts the general question of whether a theory which is derived from experiments at one level of representation of stimuli necessarily applies to all the other levels. The usual assumption is that it does, because some essential features of the behavioural phenomena of Pavlovian conditioning, or habituation, must have been observed for the level of representation to be in the table at all. But this does not mean that there could not be extra and additional behavioural tests which might allow us to distinguish the functioning of one level of representation from another. The main extra criterion of this kind is simply the nature of the stimuli which the system can respond to, which taps both perceptual complexity and the specializations or restrictions imposed by species membership and/or neuroanatomical factors. Figure 4.1Levels of representation in habituation and classical conditioning

If conditioning at one level of representation were only to be observable if both conditioned and unconditioned stimuli took the form of direct electrical stimulation of sensory nerves, then this would provide a test of performance which would distinguish this level from another at which, for instance, a conditioned stimulus could be a particular human face. Slightly more indirect and theoretical criteria which arise from consideration of stimulus range concern the degree of involvement of attentional and motivational mechanisms in the conditioning process. A very rough-and-ready rule of thumb would suggest that the lower the level of representation of stimuli, the less attention and motivation need to be considered. For instance, using standard procedures with intact mammals, the parameters of conditioning will vary with the motivational significance of the unconditioned stimulus — in salivary conditioning performance will be directly related to the animal’s degree of hunger, and whether it has any appetite or taste for the particular food signalled.

96

No doubt this is itself partly because of the effect of motivation on attention, but other attentional variables may also have a profound influence on conditioning most crudely, it is necessary for the animal to be awake (with only very rare or physiologically abnormal exceptions: Weinberger et al., 1984). However, there is no immediately obvious way in which wakefulness or emotional involvement can be attributed to the spinal cord, or to individual synapses. And this is not just a matter of all-or-none questions, which can be detached from the conditioning process itself, since overall vigilance in scanning the environment for remote stimuli, and very subtle differences in the amount of attention commanded by particular stimuli, are both likely to be involved in theories which apply to standard laboratory animals, and of course to human subjects. It is possible that, in the case of conditioning experiments with people (see Davey, 1986), Figure 4.1 ought really to have another, higher level added on, in which verbal reformulations of causal relationships between stimuli, and many other exclusively human attributional processes. are given more explicit acknowledgment; these may be considered to be subsumed in category 6.

Because of the normal influence of motivation on standard conditioning procedures, it is arguable that a/I conditioning must involve changes in motivational or emotional states, where these terms are applicable. There are a number of results, however, that suggest that while degree of motivation, or the motivational significance of the stimuli, are very important variables, it would be too restrictive to suppose that all associative learning required motivation significance as a necessary condition. The spinal level of representation would account for many of these, the human knee-jerk reflex being one of the earliest and most celebrated examples. In his pioneering study, Twitmeyer (1902/1974) noticed that one of the peculiarities of the conditioned knee-jerk response was that it appeared to be independent of motivation, since his subjects’ voluntary attempts to inhibit their responses were wholly unsuccessful. The example serves also to illustrate, . however, that there must be many cases where levels of stimulus representation and/or levels of association are mixed. Though the knee jerk may be regarded as a spinal reflex, and

97

the unconditioned stimulus of a hammer blow to the patellar tendon would hardly qualify for a high score on a scale of perceptual complexity, the conditioned stimuli, at least as used by Twitmeyer, were not completely straightforward, since although a bell was sounded as a special signal, the operation of the special hammers almost certainly was also involved — on test trials the hammers swung as usual hut were halted only just before contact with the knee was made. The conditioned kneejerks were observed after over 150 trials of conditioning in this apparatus, and this lengthy procedure suggests that familiarity with the whole context of the experiment may have been part of the learning process.

It is also possible to demonstrate conditioned associations which are relatively remote from stimuli of motivational significance within conventional paradigms in the animal laboratory. Rescorla and Durlach, (1981) , for instance, suggest that rats allowed to drink sweet and quinine-flavoured water may associate the sweet and bitter flavours, even though these associations are not revealed in behaviour until more powerful motivating circumstances are imposed on the animals, in the taste- aversion procedure (see pp. 84 and 232). More generally, ‘behaviourally silent’ or ‘latent’ learning (Dickinson, 1980, see chapter 5), implies that associations may be formed in the absence of high levels of motivation, only to be revealed at a later date motivation affects performance more strongly than it affects learning. Therefore although associations which are formed with emotionally arousing events are presumably of the greatest ecological significance, as well as being more likely to be amenable to laboratory study, it is necessary to allow for a category of relatively bloodless associations, as well as more highly charged ‘hedonic shifts’ in the value attached to real or experimental stimuli (Garcia et al., l977a).

The stimulus-response theory of classical conditioning

The stimulus-response theory of classical conditioning, as put forward by Hull (1943) and Spence (1956), was once widely held, but is now generally in disrepute (Mackintosh, 1974, 1983; Dickinson, 1980). It contains two parts. First is the assumption that the structure of the association in learning

98

takes the form of a link between stimulus input and response output, and not for instance as a link between two stimuli, nor, in its strictest form, would a link between a stimulus and an inner emotional state be acceptable in stimulus-response theory. The second part of the stimulus- response theory of conditioning is the hypothesis that the crucial condition which leads to the formation of an association is the conjunction in time of the conditioned stimulus and the unconditioned response. This hypothesis may be readily disconfirmed at higher levels of stimulus representation. For example, with salivary conditioning in normal dogs, salivation may be blocked by use of the drug atropine, during pairings of a CS with acid — when tested with the CS alone when the drug has worn off the conditioned response is normal (Finch, 1938). Using pigeons in the autoshaping apparatus, the birds may be allowed simply to observe the sequence of the lighting-up of a key being followed by presentation of food, without being able to peck at the key, or actually get to the food, but after observing this conjunction of events without responding, they will peck at the key as soon as they have access to it (Browne, 1976). It would be possible for stimulus-response theorists to argue in these cases that the internal neural motor instructions for response output had been elicited, even if not implemented, making their theory less testable but more plausible. However, this modification does not help for the slightly less robust but nevertheless real phenomenon of sensory preconditioning (Mackintosh, 1974, 1983). An example of this was given above (p. 97) in the course of discussing conditioning without motivational significance — rats testing sweet and quinine- flavoured water (or almond- and banana-flavoured water) associate the two flavours together, before any definite ‘unconditioned response’ has been imposed on them there can be no question of any small-scale version of the response, or motor instructions for it being performed, since the rats do not yet know what it is. Since the eventual motivationally significant event in this experimental procedure is intestinal distress, it is not clear that there is ever a specifiable unconditioned response. The traditional comeback for stimulus- response diehards in these cases is to argue that the fairly small effects measured should be inter-

99

preted as being due to unconditioned observing reactions, or something of that sort. Lip-smacking of some kind would have to be appealed to for associations between flavours —earlier experiments on sensory preconditioning which used the conventional tones and lights (Brogden, 1939; Rizley and Rescorla, 1972) had to be explained in terms of pricking up of ears and eye and head movements.

It is usually felt nowadays, however, that this type of sensory preconditioning experiment, along with others, negates the stimulus-response interpretation of classical conditioning in intact mammals and birds. However, it remains possible that some or other form of stimulus- response theory is applicable to classical conditioning when it is observed in the lesser systems discussed earlier, of mammalian spinal preparations or invertebrates. Certainly, when conditioning is attributed to synapses between particular identified sensory and motor neurons (Hawkins a al., 1983; Hawkins and Kandel, 1984), it seems fair to invoke the first assumption of stimulus-response theory, that the nature of the association in this case is a link between stimulus and response. However, it is interesting to see that the theory put forward by Hawkins et al., (1983) does not correspond to a cellular version of the second part of traditional stimulus-response theory. They do not say that the condition for learning is the conjunction of sensory input activity with motor nerve activity: instead they propose that it is the conjunction of CS input with the activity of a presynaptic facilitatory neuron. Thus their explanatory theory is of the stimulus pairing or ‘S-S’ form (see below), even though the result of this pairing can only be described as an increase in the ability of the CS neuron to fire the motor CR neuron. Nevertheless, this is still only a theory, and it is still only a theory about certain ganglia in Aplysia californica. It is not impossible that the animal world contains some neural systems or subsystems which correspond to what Hawkins et al., (1983) refer to as the Hebb synapse, after Hebb (1949), in which the condition of modification is precisely that .proposed in stimulus-response theory — the conjunction of some particular sensory input with a previously unconnected response motor output. Be that as it may, we already know

100

that the animal world contains neural subsystems in which associations, however formed, are accurately described as stimulus-response connections, despite the fact that stimulus-response theory is inadequate when called upon to explain the majority of results describable as classical conditioning.

Stimulus-substitution theory of classical conditioning

Pavlov’s own assumption about classical conditioning was that ‘the neutral stimulus readily acquires the property of eliciting the same reaction in the animal as would food itself’ (1927, p.26); workers directly in the Pavlovian tradition accepted and elaborated this view (Asratyan, 1965; Konorski, 1948, 1967), and various versions ofthe stimulus-substitution or ‘S-S’ theory have now replaced stimulus-response theory in the West (Jenkins and Moore, 1973; Boakes et al., 1978; Mackintosh, 1974, 1983). As a generality, it is fairly accurate to say that, in classical conditioning, the conditioned stimulus becomes a substitute for the unconditioned stimulus, but, for particular cases, there is an almost infinite number of possibilities as to exactly what is substituted for exactly what. We may first get in our ritual celebration of the variousness of stimulus representations. When one stimulus is able to substitute for another, or acquires the properties of another, it is clearly not the stimuli outside that are changing but some internal representation of the stimulus inside the animal. At the lowest levels of representation which figures in Figure 4. 1, the internal representation of an external event is simply activity in one particular sensory neuron in the spinal cord. Some physiological theorists (e.g. Barlow, 1972) might want to say that all other representations are activities in other individual neurons, at different places in the central nervous system, but by the criteria of range of stimuli and perceptual and psychological complexity, we are entitled to claim that some kinds of representation are richer and more elaborate than others.

Whether or not this claim is accepted, the behavioural results from many experiment& make it clear that it is quite impossible to predict all the phenomena of conditioning by saying that the representation of the conditioned stimulus

101

activates the representation ofthe unconditioned stimulus. In some cases, it is true, the behaviours elicited by the conditioned stimulus . appear to be almost identical to the behaviours elicited by the UCS, thus tempting us to the conclusion that the animal is undergoing a complete hallucination of this particular unconditioned stimulus. Pavlov quoted the complex sequence of physiological reactions to morphine, all elicited by the sight of a syringe. A modern argument for stimulus substitution relies on evidence that, when a pigeon pecks at a key because this signals a reinforcer, slow-motion photographic analysis suggests that if the reinforcer is food they peck at the key as if they were pecking at grain, but if it is water that is signalled they peck at the signal as if it were water (Jenkins and Moore, 1973). But in other cases there are very large differences between the set of responses elicited by the conditioned stimulus and that normally given to the reinforcer.

Although Pavlov pointed to the principle of stimulus substitution for physiological responses, he also took a very broad view of the natural function of conditioning as a signalling process, and suggested that great adaptive importance attached to the way in which auditory and visual signals for food elicited the response of seeking for the food (1927, p. 14). Similarly, there are behaviours aroused by conditioned stimuli which can be put under the heading of investigation, or exploration. Pavlov called this the ‘what is it?’ reflex, and said that its biological significance was obvious (1927, p. 12). Specific searches for food, or more general investigation and inquisitiveness, are in some ways the opposite ofthe responses elicited by food itself, and therefore are at variance with the stimulus substitution idea. Fortunately Mackintosh (1983, pp. 56—7), has recently restored order to the confused ranks of stimulus-substitution theorists by arguing that the unconditioned stimulus, or reinforcer, should not be taken as a single unit, to be substituted or not, but as a large collection of attributes, any sample of which may or may not be transferred to the conditioned stimulus or signal. The attributes are not a random and unorganized collection, and Mackintosh (1983) emphasizes that a broad distinction can be drawn between the emotional and the sensory attributes of any rein-

102

forcing event, and also between ‘preparatory conditioning’ which may be diffuse emotional restlessness or excitement, and ‘consummatory conditioning’, which involves the transfer of more precisely definable responses, normally elicited by the reinforcer, such as salivation or blinking, to its signal. The later distinction, between preparatory and consummatory conditioning, was also made earlier by Konorski (1967). Further, for both emotional and hedonic, and more specific sensory and physiological, responses to a signalled event, there may be contrasting reactions, the ‘opponent processes’ of Solomon and Corbit (1974), with an initial process (the ‘a-process’) which is the first reaction, and then a compensatory or antagonistic process (the ‘b.process’) which has behavioural effects in the opposite direction.

This is a very much richer and more complete version of the stimulus-substitution theory than the usual straw man, and it can account for the varying results listed under different levels of representation in Figure 4. 1 . Thus emotional state conditioning would be distinguished from purely sensory or perceptual effects (levels 3 and 4) and the conditioning of compensatory reactions or opponent processes in the cases of morphine tolerance (level 2) or colour after effects (level 4) is taken into account. But all these cases are based on separating out attributes of the signalled event (or unconditioned stimulus). A further modification to stimulus-substitution theory, included by Mackintosh (1983) and stressed by others such as Boakes (1977), is that the behavioural effects of stimulus pairings will also depend on the attributes of the first event, the conditioned stimulus, which does the signalling. In many cases this is obviously bound up with attentional processes or orienting reflexes specific to a stimulus modality — animals may prick up their ears to an auditory but not to a visual signalling event. Holland (1977), in a very systematic study, found that rats given the stimulus of a light to signal the impending arrival of food pellets in a magazine reacted to the light predominantly by rearing up on their hind legs. This behaviour was more frequent than the next most popular, and readily explicable, response of standing motion-less in front of the food magazine with the nose positioned at the spot where food pellets would shortly fall. His rats that

103

were given the signal of a tone instead of a light made anticipatory responses of this kind, but never reared up, as the light signal animals did, but instead gave a snap of the head or a jerk of the hind quarters, suggesting components of the startle response given to much louder non-signalling noises.

Although Mackintosh (1983) himself resists it, it would seem natural to blend the instinctive investigatory reactions of animals to the sensory attributes of signalling events into more complex reactions to these conditioned stimuli. For instance, if, in an experimental procedure very like that used by Holland (1977), the signalling event is not the presence of a light or tone but rather the presence of another rat, dropped into the experimental cage, reactions to the new and social stimulus reflect its food signalling properties, but only in social ways. Another rat dropped into the cage as a signal for food is greeted more enthusiastically , by pawing and grooming of the normal rat kind, than a conspecific similarly dropped but without a history of bearing gifts (Timberlake and Grant, 1975).

With dogs, explicitly social reactions are observed even when the conditioned stimulus itself supplies no social prompt. Jenkins et al., (1978) used a speaker above a protruding lamp to provide a localized compound signal for 10 seconds before small pieces of hot-dog were dropped into a tray a metre or so away. The enthusiastic anticipation of these titbits by the dogs was indicated not merely by alertness or salivation, but by the fact that they approached the signal source and showed individual patterns of social behaviour. Some dogs devoted the time when the signal was on to prancing in front of the food tray, but others nuzzled or stood by the signal, and wagged their tails, This can all be incorporated into the modified stimulus-substitution theory advanced by Mackintosh (1983), as it is clearly the shift of emotional associations of hot-dogs to the signal source which arouses tail-wagging directed at it. However, when the attributes of the signalling event elicits social behaviour, as with Timberlake and Grant’s experiment above, or if, as we may imagine, smells or sounds remotely associated with game elicit tracking or co-operative hunting in canines, then the release of instinctive behaviours characteristic of particular species

104

will have been subsumed in a large measure under the stimulus-substitution heading (Pavlov, 1927, p. 14).

Modified stimulus-substitution theory — conclusion

That psychological effects due to one stimulus come to be elicited by another is the essence of Pavlovian conditioning, and putting this in terms of the substitution of the signalling for the signalled event is usually satisfactory. it is clear, however, that the real meat of the experimental data, if one can put it that way, is not in this truism, but in the great variety of psychological effects to which it applies. If Pavlovian conditioning applied only to salivation, then not even Pavlov could have made very much of it. It is only because results obtained with salivation could be put in the contexts of general theories about signalling stimuli, and the action of the cerebral hemispheres, that the results acquired theoretical importance. Thus the question within stimulus-substitution theory is now not whether it is happening, but exactly what is being substituted for what. Is it a diffuse emotional substitution or a precise perceptual one? And is the psychological or physiological effect which is transferred to a signalling stimulus an instinctive general preparation, a premature jumping of the gun with a precise response characteristic of those given to the goal, or actually a conditioning of internal responses which will resist and oppose the final event? The variety of results discussed above demonstrates that these do not always boil down to the same thing. In any case stimulus-substitution is not really a theory at all, but a glorified description — it does not say why or how the changes take place or what other neural or psychological mechanisms are at work. The detailed explanations of why stimulus-substitution appears to take place will almost certainly depend on which attributes of the signalled event are involved, and on what I have called the level of representation of the stimuli whether the behaviour concerns only spinal reflexes, metabolic reactions to injected drugs, or the attentive perceptual and motivational resources of the whole animal.

105

Discrepancy and expectancy theories of stimulus repetition and stimulus pairing

The most active area of theoretical development in the study of classical conditioning for more than the last decade has been in the test of assumptions originally put forward by Rescorla and Wagner (1972). One of the reasons for the fertility of these ideas is that they are capable of precise mathematical expression, but this means that the ideas may not be very accessible to an audience who are not specialists in the area. Therefore I shall attempt here to discuss the theoretical questions with a minimum of mathematical exact-ness. Readers who wish to follow up the original references might consult also Wagner (1976, 1978, 1979, 1981), Mackintosh (1975), Pearce and Hall (1980) and Grossberg (1982), but excellent detailed accounts of this work are also given in Dickinson (1980) and Mackintosh (1983). Despite the great fertility of this area, one of the difficulties of relating it to the present context is that the mathematical assumptions are not usually clearly tied down to biological and physiological realities. In fact, most of the testing of the theories takes place with the standard laboratory procedures discussed on pp. 83—90, which use intact mammals, and therefore I shall interpret the theories in terms of the perceptual and attentional capacities that might be expected to be present in these instances. The usefulness of the mathematical equations lies partly in the fact that they might apply equally well to the behaviour of very different systems, in some cases to the synaptic and stimulus-response level of stimulus representation (Sahley et al., 1981), but the biological mechanisms responsible for the success ofthe equations might presumably be different in different systems.

The basic idea put forward by Rescorla and Wagner (1972) may be interpreted in terms ofthe discrepancy between actual and expected events in conditioning procedures. Their assumption was that a change in the properties of a conditioned stimulus only occurs when something happens which is surprising or unexpected. Thus a dog receiving food for the first time after hearing a buzzer will be pleasantly surprised, and a large increase in the conditioned effects of

106

the buzzer should therefore take place. However, on receiving food after the buzzer for the umpteenth time, there is no surprise or unexpectedness, and therefore little change should accrue to the properties of the conditioning stimulus. This can be and was put in terms of attentiveness and expectations; and is experimentally testable in the phenomena of blocking in classical conditioning. If the dog is already conditioned to salivate to the buzzer, and a light is then added to the buzzer as a signal, subsequent testing reveals little salivation to the light (Pavlov, 1927; see Kamin, 1969), even though if the light had been used by itself to start with for the same number of trials, it might have been a most effective conditioned stimulus. This result of blocking of attention to the light is readily explained by saying that signalled events which are already expected contribute little to further conditioning. The more mathematically precise way of expressing this was in the equation below:

 

gif

 

This refers to a conditioned stimulus A which precedes a reinforcer or unconditioned stimulus R. gif is the increment to the conditioned properties of A. This depends on two constants, one of which is gif, the salience or associability of A, which has recently been a point of theoretical controversy, the other, gif, the intensity or emotional significance of the reinforcer, being of unquestioned but little investigated practical importance. The originally vital part of the equation is gif, which represents the upper limit of all conditioned properties for the reinforcer R, minus the overall value of conditioned properties that are already present on a given trial covered by the equation. The equation would work perfectly well whatever these conditioned properties happened to be, and in terms of gradual acquisition of a conditioned response, as its strength approaches an upper limit, it would apply to the spinal leg flexions of Beggs et al., (1983), and for the reduced response to a second conditioned stimulus which is added to one which is already functioning, it would apply to the behaviour of the terrestrial slug (Sahley et al., 1981). It is conventional to interpret the equation,

107

and others of the same form, in terms of expectations and representations of the conditioned and unconditioned stimuli, but it should be borne in mind that in some cases the internal representation of outer events will be accomplished by the activity of very rudimentary neural systems.

Following the convention, and ignoring the degree of significance of the signalled event, the equation can be interpreted as follows: the increase in the predictive value of a signal is proportional to a measure of its associability times the discrepancy between the obtained experience of the unconditioned stimulus and the overall level of expectation. In Rescorla and Wagner’s treatment, the associability of the signal was a fixed property to do with physical things such as the intensity of a tone, and thus the explanation of conditioning phenomena rested entirely on reactions to the reinforcer. If the reinforcer was surprising, then extra conditioning could happen, but if the reinforcer was already expected, no further changes were necessary. What is missed out here, both in the equation and informally, is any other way of representing the attention given to signals.

Only expectations of the reinforcer change in the original Rescorla and Wagner (1972) theory, but a brief look at chapter 2 would be sufficient to indicate that the idea of expectations which change with repeated experience was used by Sokolov (1963) as a theory of response to stimuli which are repeated by themselves, without signalling anything of powerful motivational significance. Thus, if a stimulus is presented on its own repeatedly, a theory of habituation supposes that certain important things happen: in Sokolov’s theory, it will be recalled, attention to that stimulus declines as knowledge of the stimulus, embodied in a neuronal model, increases. What, then, would Sokolov (or Pavlov) predict if a stimulus is first thoroughly habituated, and then used as a signal for food, or some other standard unconditioned stimulus? For two separate reasons, it could be predicted that subsequent conditioning will be somewhat delayed. First, the initial attention given to an habituated stimulus is reduced (the ‘orienting reflex’ has extinguished); and second, if the neuronal model of the repeated stimulus is as comprehensive in terms of temporal relationships as Sokolov (1963, 1975)

108

supposed, then the neuronal model will include the fact that nothing else normally happens at the end of the signal stimulus, and it may take some time to overlay this bias with the new knowledge that the habituated stimulus now usually precedes a more powerfully motivating stimulus.

For whatever reason, it is a well- established experimental phenomenon that prior habituation of a stimulus to be used as a signal in Pavlovian conditioning will delay the process (Baker and Mackintosh, 1977; see Mackintosh, 1983). Frequently this phenomenon is referred to as ‘latent inhibition’, on the assumption that some inhibitory process is building up during habituation, but habituation does not necessarily involve the inhibitory processes which are appealed to to account for the suppression of responding by non-reinforcement (Mackintosh, 1983). There are both theoretical and factual reasons therefore to elaborate the theory of Rescorla and Wagner (1972) to incorporate habituation, and more generally the amount of attention devoted to conditioned stimuli, usually discussed in terms of the associability of those stimuli with subsequent signalled events. There is not yet complete agreement about the exact form such an elaboration should take. Pearce and Hall (1980) give a very thorough and formal account of the complexities and subtleties of the theoretical problems, but supply some ideas which may be usefully paraphrased, and link most directly with Sokolov’s theory of habituation. They retain the central hypothesis of Rescorla and Wagner that the essence of conditioning is surprise, and that equations must therefore incorporate expressions which compare obtained experience with expectations. But instead of assuming that the unconditioned stimulus (US) loses effectiveness, they propose that the factor of associability of the conditioned stimulus, a separate entry in the equation, will depend on the confirmation of expectations, since they argue that any conditioned stimulus loses associability when its consequences are accurately predicted. Metaphorically, at least, they suggest that this is equivalent to the shift from controlled to automatic processing observed by Schneider and Shiffrin (1977) in studies of vigilance in human subjects. If we were to suppose that the sequence of conditioned stimulus followed by uncon-

109

ditioned stimulus was a complex single whole in Sokolov’s model, we should similarly expect that a very often repeated sequence would lead to some reduction in attention and activation (see Figure 2.1). In Pavlov’s laboratories, dogs were very often given several different positive conditioned stimuli, simply because some of them became so inattentive of repeated sequences that they were overcome by sleep. On grounds of general plausibility therefore, the familiarity of a signalling sequence of stimuli might be thought to result in some change in processing. Several particular experimental results also support the theory put forward by Pearce and Hall (1980). Most directly, Hall and Pearce (1979) used a tone as a signal for a weak shock, in a conditioned suppression procedure, (see pp. 83—4). Then this tone was used as a signal for a stronger shock, and the rapidity of the behavioural effects of this compared in control groups which had previously received the tone with no shock at all, or the weak shock signalled by a light. The results (see Figure 4. 1) showed that having experienced the tone-weak shock sequence appeared to delay the effects of the tone-strong shock procedure. This is evidence against the idea that the initial pairing of the tone with weak shock should enhance subsequent conditioning using very similar stimuli. However it is not clear from this data whether the previous tone-shock experience had reduced attention to the tone, or built up a tone-weak shock association which delayed the tone- strong shock learning by a process analogous to proactive interference.

The simplest encapsulation of the Pearce- Hall (1980) theory is that a stimulus is more actively processed if there is uncertainty about its consequences (Mackintosh, 1983, p. 231). This certainly covers ordinary habituation, since there is little scope for uncertainty in the sense that there are no consequences, or, in Sokolov’s model, the consequences are built into the model in terms ofthe temporal relationships between successive single stimuli.

An elaboration of the Rescorla-Wagner theory which at first sight is in conflict with the Hall-Pearce model is that presented by Mackintosh in 1975, in which he suggested that stimuli which are already good predictors of other events will

110

Figure 4.2 An experiment on stimulus familiarity using tones and lights.

gif

The effects of pairing the same tone with a strong shock, for 3 groups of rats with varying experience of the tone. The lower a point on the graphs, the stronger the effects of the pairing. Thus the group with no previous experience of the tone but with experience of a light signalling a weak shock showed the fastest conditioned suppression and the group which had experienced the tone but no previous shocks showed the slowest effect of the pairing. In between was a group which had previously experienced the same tone signalling a weaker shock. After HaIl and Pearce (1979).

 

be more actively attended to, and therefore more readily associable with a new unconditioned stimulus. There is ample evidence in favour of this view from rather different experimental procedures, which will be discussed in chapter 8. Most obviously, if an animal merely hears a pure tone, uncorrelated with motivating events, it is unlikely to continue to attend to it for very long; but if the same tone is a signal for food or shock, then attention to the tone will be increased. This is confirmed by the comparison in the experiment of Hall and Pearce (1979: see Fig. 4.1). One resolution of this conflict would be to assume that the loss of associability indicated by the comparison between the tone- shock and light-shock groups in that experiment was not due to lack of

111

attention to the tone, but because attention given to the tone was combined with an expectation of weak shock, which expectation had to be altered. In general it is very difficult to be sure from such experiments exactly what form changes in ‘associability’ might be taking. However, it is possible to include, in the notion that uncertainty increases processing, the possibility that in the early stages of conditioning attention to a stimulus will be increased, only to wane again when the whole procedure becomes routine. This again is roughly similar to what happens in the development of habituation to novel stimuli.

Finally, Wagner himself has proposed a number of elaborations to the Rescorla and Wagner (1972) theory, which are explicitly intended to apply to habituation (Wagner, 1976), which make use of the previous theories of both Sokolov (1963) and Konorski (1967), and which are extended to link the phenomena of classical conditioning with some aspects of automatic processing in human memory (Wagner, 1981). Animals are assumed to represent stimuli in both short- term and long-term stores (see Figure 4.2 from Wagner, 1978). If such representations are already primed, either by recent presentation of the stimulus itself (in habituation) or by presentation of another signal previously associated with the stimulus (in classical conditioning), then response decrements of some kind are theoretically predicted (Wagner, 1976, p. 1 24) . The former process (in habituation) is referred to as ‘self-generated priming’, and the later (in classical conditioning) as ‘retrieval-generated priming’ (ibid., p. 124). The storage of information in classical conditioning can be regarded as the ‘course of activity in an individual memorial node under different circumstances of stimulation’ (Wagner, 1981, p. 17), and equations specifying in detail such changes can be mapped on to precise experimental data. The most relevant data are of course those from experiments on habituation before conditioning, or latent inhibition. An important contribution of Wagner’s theory (1976, 1981) is the suggestion that habituation to repeated stimuli should be context-specific (1976, p. 120) . Just as it is possible to view repeated stimulus pairings as requiring some degree of habituation to the pairing interpreted as a single complex stimulus, so it is possible to

112

Figure 4.3 Habituation as a form of priming.

gif

A theory of stimulus-response habituation in terms of short- term and long-term memory stores. It is assumed that, if a representation of a stimulus is already primed in the short- term store (STM) when the stimulus arrives at the sensory register, then response output is reduced. At (a) the receipt of a stimulus leads to a representation of the stimulus being moved from long-term to short-term memory, where it may be maintained by the loop (‘rehearsal’). At (h) a representation may be retrieved without the stimulus itself being present, as in classical conditioning. At (c) the external stimulus directly accesses its already present representation in STM. After Wagner (1978).

 

think of a simple habituation experiment as the formation of an association between the context in which an experimental stimulus is presented and the stimulus itself. If a human subject listens to repetitions of a tone in a given room then habituation to the tone will probably involve associations between that tone and that room. Detailed studies of habituation and conditioning in animal experiments suggest that habituation (tested as the latent inhibition phenomenon of delayed subsequent conditioning) is certainly context- specific in this way, though a particular CS-US association is much less influenced by such contextual cues as background odours (Lovibond et al., 1984). Clearly one would not want an infinite regress of all stimuli being regarded as forming associations

113

with prior contexts, but the notion that repetitions of a stimulus will be coded in terms of surrounding cues is certainly consistent with Sokolov’s treatment of the ‘neuronal model’, and with the common experimental result of dishabituation when background cues are changed. Wagner’s theories should therefore be regarded as strong support for the strategy of treating habituation and classical conditioning as alternative experimental procedures applied to the same subjects and therefore as requiring theories about the general nature of information processing in those subjects.

Levels of representation in discrepancy theories of conditioning

As yet, it is a feature of the expectancy and discrepancy that few predictions can be made from them about the phenomena to be expected at the different levels of stimulus representation shown in Figure 4.1 and discussed earlier in this chapter. It may be that analogous neural mechanisms in very different biological systems mean that no difference in predictions is necessary. For instance, the ‘Standard Operating Procedures’ of automatic memory processing which are discussed at length by Wagner (1981) are mainly based on the experimental phenomenon of ‘conditioned diminution of the UR’ in the nictitating membrane response of restrained rabbits, where the main stimulus is electric shock to the eye region. When the shock is applied, there is normally an eyeblink whose amplitude can be measured, and ‘gross body movements’ which similarly may be quantified. If a signal of a 1—second tone is given before all shocks, then some blinking and movement occurs to the signal, but also, the blinking and movement then given to the shocks themselves are reduced (Wagner, 1981, p. 27). The theory of retrieval-generated priming, by the signal, of a representation of the unconditioned stimulus may be the best way to account for this result, but it is easy to see that not wholly dissimilar data might be obtained, possibly from decorticate rabbits (Oakley and Russell, I 976, see pp. 72—3) or possible from conditioning with Aplysia (e.g. Carew et al., 1983; Hawkins and Kandel,

114

1984), in which the reduction in response to the unconditioned stimulus was a result of relatively straightforward short-term fatigue in the appropriate muscle systems.

Therefore it is not necessary to assume that short-term and long-term memory stores, and processes of exchange between them, always take precisely the same form at all levels of stimulus representation. In particular, it would be expected that active processes of attention, which are to be increased when certain stimuli are recognized as good predictors, as proposed by Mackintosh (1975), should not be as obvious at lower levels of representation and/or in simpler neural systems. Mackintosh himself has proposed that this may be one of the dimensions of the difference between the behavioural capacities of the various species of vertebrate animals, from fish to mammal (Mackintosh, 1969).

Habituation and conditioning — conclusions

In the first few years of this century, it was a view held by some that the conditioning of the response of salivation to a signal, as was then being studied by Pavlov, could not occur in dogs with certain parts of the cortex of their cerebral hemispheres removed. Pavlov himself demolished this hypothesis by sitting in front of two such dogs, in someone else’s laboratory, and demonstrating the conditioning of the salivary reflex on the spot. A main trend in the study of conditioning ever since has been the discovery of anticipatory shifts of simple reflexes to new signals in neural systems progressively further removed from that possessed by a fully equipped dog, culminating most recently with some physiologically important work on the gastropod mollusc Aplysia californica. It was certainly not Pavlov’s point that the cerebral cortex was irrelevant for classical conditioning; on the contrary he felt, erroneously as it turns out, that conditioned reflex techniques would provide the key to uncovering its mysteries. There are signs, however, that the more complex and cognitive aspects of associative learning in higher organisms, as well as the physiological basis of reflex connections in lower ones, are becoming better understood. In new versions of the stimulus- substitution theory, it is fully recognized that

115

a signalled event may be made up of a multiplicity of attributes, some in the form of internal perceptual representations of various degrees of elaboration, some in the form of emotional states, others being best seen as a delicate balance between opposed metabolic processes (Mackintosh, 1983). In conditioning, some sample of these attributes is transferred to a signalling event, the mere size of the sample, but also in particular the degree of elaboration of the perceptual representations transferred, providing some measure of the complexity of the associative processes involved. Theories of precisely how these associative processes operate now cluster around the notion of the ability to detect a discrepancy between expected and observed events. (Rescorla and Wagner, 1972; Rescorla, 1978; Wagner, 1976, 1981; Pearce and Hall, 1980). This provides a useful bridge to the theory of habituation (Sokolov, 1963, 1975) . Animals may be viewed as engaged in a continuous process of updating an internal model of how external reality should be. Anticipation is an adaptive virtue, and the functional end of both habituation and classical conditioning is that there are no surprises: when a surprising event occurs the animal system changes so as to predict it in the future on the basis of time, place, or cause.

Detailed experimental results along these lines can be obtained with laboratory mammals and birds, which allow the testing of some precisely formulated theories, but the essential feature of anticipatory reflexes, if not the mechanisms for cognitive expectation, can be studied in simpler preparations. Here again it would appear that internal organization reflects the procedural commonalities of the repetition of a single stimulus (habituation) or of a stimulus pairing in classical conditioning, since changes in the responsiveness of simple neural systems to either procedure is thought to be based on some or other kind of pre-synaptic facilitation, that is, changes in the interactions of sensory neurons which are independent of the characteristics of the motor or output neuron (Thompson and Glanzman, 1976; Kandel, 1976; Hawkins et al, 1983). This might be regarded as a form of prediction for the future at the simplest possible level of sensory representation of external events. Be that as it may, the descriptive features of classical conditioning, as the

116

transfer of psychological effects from one stimulus to another, are common to very many kinds of learning from experience, and several different theoretical issues. Therefore the phenomena of classical conditioning, and the various attempts to explain them, will come up frequently in subsequent chapters.

 

End of Chaper 4 | Start of Chapter 5 | Contents