School of Psychology, Birkbeck College

Course PSYC044U (Psychobiology II) WEEK 6
March 1st 2007

gifThis is just 6 of the pages of the much longer paper handout. Web versions of the other pages in the paper handout are accessible from the side index. If you need to print out the handout, then all the pages are in this 'pdf' file, but this is quite large and may be difficult to download over a telephone modem.




  • Instrumental conditioning differs, at least at first sight, from Pavlovian conditioning in that i) new response patterns emerge and ii) responses are “goal- directed”.

  • An early experimental and theoretical analysis of instrumental learning was given by Thorndike (1898, 1901: 1898 excerpt reprinted 1998) who studied cats escaping from “problem boxes” in which it was necessary for them to pull a loop of string, press a lever (or make some other artificial response) in order to get out.

  • Thorndike explained the results by the “Law of Effect” which supposes that the positive motivational consequences of behaviours influence learning, but only by “stamping-in” connections between successful responses and the circumstances in which they were performed.

  • Thus, although goal achievement affects learning, the resulting behaviours are not goal directed in the sense of being modified by any kind of internal anticipation of their outcomes.

  • Mechanisms of this kind undoubtedly exist in many species, including mammals (Brembs 2003; Eisenstein & Carlson, 1994; Packard and Knowlton, 2002; Segal and Wolf, 1994; Hawkins et al., 2006).

  • An even more reductionist “stimulus-response” theory of instrumental learning was put forward by Hull (1943, 1952)

  • Elaborate procedures for controlling the outcome of instrumental procedures were devised by Skinner (1938; Ferster and Skinner, 1957) without further elaboration of Thorndike's “Law of Effect” explanation.

  • A more cognitive interpretation of instrumental learning phenomena is due to Tolman (1932, 1948), and this is broadly supported by more recent analyses (Dickinson, 1985; Mackintosh, 1983; Rescorla, 1991; Lieberman, 2000; Brown & Giumetti, 2006; Raby et al., 2006).

  • Tolman's view was that the animals in his experiments made use of “cognitive maps” of the environment and its significant features, and formed “expectancies” with regard to specific goals.

  • Two main kinds of experimental evidence can be used to argue against the Thorndikean “Law of Effect” and in favour of more purposive theories.

1. Spatial Learning. Most of Tolman's own experiments were on maze learning, where results such as the use of novel short-cuts, and the apparent utilization of spatial knowledge for differing modes of locomotion (swimming round mazes previous learned when dry: Macfarlane, 1930) are difficult to explain in terms of stimulus- response habits.

The term "cognitive map" is no longer popular, because the evidence suggests that the representation of spatial relationships in insects (Giufra and Capaldi, 1999), rats (Hogarth et al., 2000) and indeed humans (Wang and Spelke, 2000; Foo et al., 2005) is not as rich as the term "cognitive map" would suggest. However, Tolman's basic points, that there are internal representations of space of some kind, and internal representations of rewards, as opposed to blind habits alone, are no longer in dispute (Jacobs, 2003; Jacobs and Schenck, 2003; Menzel et al., 2005). Although this supports a cognitive interpretion of the effects of incentives, much current work on "reward-circuits" emphasises that these circuits are facilititated by the effects of addictive drugs. All addictive drugs have in common that they enhance (some directly, some indirectly) dopaminergic function at crucial dopamine pleasure/reward synapses in the nucleus accumbens. (Berridge, 1996; Cardinal and Everitt, 2004; Everitt & Robbins, 2005; Davis et al., 2004; Ikemoto and Panksepp, 1999; Rodd et al., 2004; Vanderschuren & Everitt 2004; Wise, 1996).


[bottom of page 1 of handout]
2. Reward re-evaluation.

  • In Tolmanian accounts of instrumental learning, responses are produced as means to ends (and are thus “actions” rather than “habits”).

  • Such accounts can be tested by direct alterations of the reward-value of goals.

  • Early experiments on “latent learning” (Tolman and Honzik, 1930) demonstrated that in a familiar maze rats will return immediately to a location in which they have once previously found food.

  • In the “radial maze” (Olton and Samuelson, 1976) rats appear to use a working memory of places where they have already finished all available food on a given day.

  • Extensive work on the devaluation of specific rewards for specific responses also suggests that there are associative links between learned responses and their customary outcomes (Crespi, 1942 see page 11; Dickinson, 1985; Rescorla, 1991; Percoraro et al., 1999; Cowill and Triola, 2002; Colwill, 2006; Yin et al., 2004; Corbit and Balleine, 2005; Balleine, 2005; Wang et al., 2005).



[bottom of page 1 of handout]

Sample Essay

To what extent was Thorndike correct in assuming that instrumental learning can be fully accounted for by the strengthening of connections between preceding stimuli and successful responses?






[top of page 2 of handout]

“Classical conditioning can be regarded as a rather primitive, automatic consequence of certain associations: instrumental conditioning requires that the organism utilize the information embodied in certain associations in order to reach certain conclusions.”
Mackintosh (1983), p. 112


“It seems likely that there are at least two kinds of human long-term learning, one of which is open to conscious reflection (perhaps based on instrumental conditioning), and influenced by the capacity to verbalise a strategy, while the other is implicit and independent of verbalization (perhaps based on Pavlovian conditioning?)”
Baddeley (1990) p. 206.
















Although there have been attempts to explain both Pavlovian and Instrumental conditioning in terms of a single underlying associative principle, there are a number of behavioural and physiological features which suggest differentiation between Pavlovian and instrumental learning (see the table attached). For instance, instrumental procedures appear not to be directly effective in modifying the responsiveness of the autonomic nervous system (Dworkin and Miller, 1986) while Pavlovian procedures are not suitable for producing substantial changes in repertoires of motor skills. A critical theoretical distinction is that Pavlovian conditioning is not sensitive to motivational pay-offs, whereas instrumental learning, by definition, is.

An experimental test derived from this is the use of an 'omission' procedure to check if supposedly Pavlovian responses are sensitive to changes in pay-offs (e.g. Sheffield, 1965; Williams and Williams, 1969: see Lieberman, 2000 pp443-444; or 1993, pp.345-346).

[middle of page 2 of handout]
Neuroscientific evidence tends to support separate brain mechanisms for Pavlovian and instrumental conditioning (Dickenson et al., 2000; Ikemoto and Pankseep, 1999; Leszczuk and Flaherty, 2000; Balleine et al., 2003; Baldwin et al., 2002; Baxter & Byrne, 2006; Shultz, 2006; Spicer et al., 2007 – informally 'reward-seeking' can be different from 'reward-anticipation' and 'wanting' can be different from 'liking' - Berridge and Robinson, 2003; Wilson et al., 2006).

However, many natural behaviours and laboratory tests are likely to be influenced by both Pavlovian and instrumental processes. A particular instance is the procedure of "autoshaping" in pigeons, in which the species-specific response of pecking can be used in stimulus-pairing procedures, where it is resistant to loss of pay-offs (Williams and Williams, 1969), but at the same time can be shown to be highly sensitive to instrumental contingencies in other contexts (Ferster and Skinner, 1957; Lieberman, 1993, pp 215-225 or 2000, pp. 220-229).

Apart from comparisons between Pavlovian and instrumental learning (see these tables) conceived of as learning processes which can apply very generally across species, and across behaviours in any species, a second question is the degree to which any kind of learning is either species- or domain-specific: that is, how far it should be regarded as a specialized solution to a particular biological problem, as opposed to a generally applicable principle. This issue was raised in the context of "imprinting" (Week 2), and another important test case is "taste-aversion learning" (Garcia and Koelling, 1966; Garcia, 1981; already discussed in Weeks 3 and 4), which at first sight appears to be a good candidate for a highly specialized learning process. It is however very general across species, and it is arguable that it is a specialized form of classical conditioning, rather that a different learning process altogether (Revusky, 1985; Garcia et al, 1977; Davey, 1989). Similarly, in many species instrumental learning as studied in the laboratory can be viewed as an adaptation to aid optimal foraging for food (Lea, 1979, 1984; Baum et al. 1999). The assessment of experienced pay-offs, suggested above as one of the defining features of instrumental learning, would clearly be important in many kinds of foraging, but may have more general applicability (Brems, 2003; Kringelback & Rolls, 2004; Lorenzetti et al., 2006; Baxter & Byrne, 2006; Lea & Webley, 2006).


[bottom of page 2 of handout]

Sample essay

Compare and contrast classical and instrumental conditioning.









Main Sources

Lieberman, D. (1993/2000) Learning: Behavior and Cognition. Belmont: Wadsworth.(in 1993: pp 57-65, 166-185. 215-231, 325-9 & 343-356/ in 2000: pp. 41-2, 163-167, 192-234 & 442-449)

Walker, S.F. (1985) Animal Thought. Routledge & Kegan Paul: London. (pp. 61-94).

Walker, S.F. (1987) Animal Learning: An Introduction. Routledge & Kegan Paul: London. (CHAPTER 5, 'Instrumental Learning' and Chapter 6 'Biological bases of classical and instrumental conditioning', pp. 165-202.)

There is also basic material on these topics to be found in the Gleitman et al texts, as follows:

Page reference in Gleitman et al 2004 Page reference in Gleitman et al 1999 Page reference in Gleitman 1995 Textbook Heading
132-3 128-9 115-6 “Instrumental Learning”
143, 147 140, 144 126, 143 “Cognitive Learning/Cognitive Maps”
118 111 99-100 “The Biology of Reward”

Further Reading

Davey. G. (1989) Ecological Learning Theory. London: Routledge. CHAPTER 3. and (Ch 6, "The Biological Function of Classical Conditioning" and pp 225-254 for optimal foraging)

Carlson, N. (1998) Physiology of Behavior. 6th edn Allyn & Bacon: Needham Heights. pp. "Reinforcement", pp 441-449; "Common features of addiction", pp565-584. (6 copies, 3-1wk & 3-3wk loan in BK library)

Baddeley, A.D. (1990) Human Memory. Hove, Lea: Chapter 9, "Acquiring Habits", pp. 201-232.

Other References (not normally needed for further reading)

Adams, C.D. (1982) Variations in the sensitivity of instrumental responding to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 34B, 77-98.

Amiez, C., Joseph, J. P., & Procyk, E. (2006). Reward encoding in the monkey anterior cingulate cortex. Cerebral Cortex, 16(7), 1040-1055.

Anderson, K. G., & Woolverton, W. L. (2000). Concurrent variable-interval drug self-administration and the generalized matching law: A drug-class comparison. Behavioural Pharmacology, 11(5), 413-420.

Aron, A., Fisher, H., Mashek, D. J., Strong, G., Li, H. F., & Brown, L. L. (2005). Reward, motivation, and emotion systems associated with early- stage intense romantic love. Journal of Neurophysiology, 94(1), 327-337.

Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O'Reilly, R. C. (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience, 10(1), 126-131.

Baldwin, A. E., Sadeghian, K., Holahan, M. R., & Kelley, A. E. (2002). Appetitive instrumental learning is impaired by inhibition of cAMP-dependent protein kinase within the nucleus accumbens. Neurobiology of Learning and Memory, 77(1), 44-62.

Balleine, B. W. (2005). Neural bases of food-seeking: Affect, arousal and reward in corticostriatolimbic circuits. Physiology & Behavior, 86(5), 717-730.

Balleine, B. W., Killcross, A. S., & Dickinson, A. (2003). The effect of lesions of the basolateral amygdala on instrumental conditioning. Journal of Neuroscience, 23(2), 666-675.

Balleine, BW and Dickinson, A (1998) The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety. Animal Learning & Behavior, Vol.26, No.1, Pp.46-59.

Bartels, A., & Zeki, S. (2004). The neural correlates of maternal and romantic love. Neuroimage, 21(3), 1155-1166.

Bateson, M & Kacelnik, A (1995) Accuracy of memory for amount in the foraging starling, Sturnus-vulgaris. Animal Behaviour, Vol.50, No.Pt2, Pp.431-443.

Baum, W. M., & Kraft, J. R. (1998). Group choice: Competition, travel, and the ideal free distribution. Journal of the Experimental Analysis of Behavior, 69(3), 227-245.

Baum, W. M., Schwendiman, J. W., & Bell, K. E. (1999). Choice, contingency discrimination, and foraging theory. Journal of the Experimental Analysis of Behavior, 71(3), 355-373.

Baxter, D. A., & Byrne, J. H. (2006). Feeding behavior of Aplysia: A model system for comparing cellular mechanisms of classical and operant conditioning. Learning & Memory, 13(6), 669-680.

Belke, T. W. (2000). Studies of wheel-running reinforcement: Parameters of Herrnstein's (1970) response-strength equation vary with schedule order. Journal of the Experimental Analysis of Behavior, 73(3), 319-331.

Berridge, K. C. (1996). Food reward: Brain substrates of wanting and liking. Neuroscience and Biobehavioral Reviews, 20(1), 1-25.

Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: Hedonic impact, reward learning, or incentive salience? Brain Research Reviews, 28(3), 309-369.

Berridge, K. C., & Robinson, T. E. (2003). Parsing reward. Trends in Neurosciences, 26(9), 507-513.

Blaisdell, A. P., & Cook, R. G. (2005). Integration of spatial maps in pigeons. Animal Cognition, 8(1), 7-16.

Blundell, P., Hall, G., & Killcross, S. (2001). Lesions of the basolateral amygdala disrupt selective aspects of reinforcer representation in rats. Journal of Neuroscience, 21(22), 9018-9026.

Braida, D., Pozzi, M., Cavallini, R., & Sala, M. (2002). 3,4 methylenedioxymethamphetamine (ecstasy) impairs eight-arm radial maze performance and arm entry pattern in rats. Behavioral Neuroscience, 116(2), 298-304.

Breland, K. and Breland, M. (1961) The misbehaviour of organisims. American Psychologist, 16, 681-4.

A complete on-line version of this is provided in the “Classics in History of Psychology Series” (U of York, Canada)

Brembs, B. (2003). Operant conditioning in invertebrates. Current Opinion in Neurobiology, 13(6), 710-717.

Brembs, J., Lorenzetti, F. D., Reyes, F. D., Baxter, D. A., & Byrne, J. H. (2002). Operant reward learning in Aplysia: Neuronal correlates and mechanisms. Science, 296(5573), 1706-1709.

Brown, M. F., & Giumetti, G. W. (2006). Spatial pattern learning in the radial arm maze. Learning & Behavior, 34(1), 102-108.

Brown, P.L. and Jenkins, H.M. (1968) Autoshaping of the pigeon's key peck. Journal of the Experimental Analysis of Behaviour, 11, 1-8.

Caldwell, A. B. (2001). What do the MMPI scales fundamentally measure? Some hypotheses. Journal of Personality Assessment, 76(1), 1-17.

Cardinal, R. N., & Everitt, B. J. (2004). Neural and psychological mechanisms underlying appetitive learning: links to drug addiction. Current Opinion in Neurobiology, 14(2), 156-162.

Colwill, R. M. (2006). Passage of time leaves response-outcome associations intact. Behavioural Processes, 72(1), 91-99.

Colwill, R. M., & Triola, S. M. (2002). Instrumental responding remains under the control of the consequent outcome after extended training. Behavioural Processes, 57(1), 51-64.

Colwill, R.M. and Rescorla, R.A. (1985) Post-conditioning devaluation of a reinforcer affects instrumental responding. Journal of Experimental Psychology: Animal Behaviour Processes, 11, 120-132.

Colwill, R.M. and Rescorla, R.A. (1990) Effect of reinforcer devaluation on discriminative control of instrumental behaviour. Journal of Experimental Psychology: Animal Behaviour Processes, 16. 40-47.

Corbit, L. H., & Balleine, B. W. (2003). The role of prelimbic cortex in instrumental conditioning. Behavioural Brain Research, 146(1-2), 145-157.

Corbit, L. H., & Balleine, B. W. (2005). Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of pavlovian-instrumental transfer. Journal of Neuroscience, 25(4), 962-970.

Corbit, L. H., Muir, J. L., & Balleine, B. W. (2001). The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell. Journal of Neuroscience, 21(9), 3251-3260.

Crespi, L.P. (1942) Quantitative variation in incentive and performance in the white rat. American Journal of Psychology, 55, 467-517.

Davis, C., Strachan, S., & Berkson, M. (2004). Sensitivity to reward: implications for overeating and overweight. Appetite, 42(2), 131-138.

Di Ciano, P., & Everitt, B. J. (2004). Conditioned reinforcing properties of stimuli paired with self-administered cocaine, heroin or sucrose: implications for the persistence of addictive behaviour. Neuropharmacology, 47, 202-213.

de Wit, S., Niry, D., Wariyar, R., Aitken, M. R. F., & Dickinson, A. (2007). Stimulus-outcome interactions during instrumental discrimination learning by rats and humans. Journal of Experimental Psychology-Animal Behavior Processes, 33(1), 1-11.

Dickinson, A (1985) Actions and habits: the development of behavioural autonomy. Proceedings of the Royal Society, B, 308, 67-78.

Dickinson, A., Smith, J., & Mirenowicz, J. (2000). Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behavioral Neuroscience, 114(3), 468-483.

Dragoi, V., & Staddon, J. E. R. (1999). The dynamics of operant conditioning. Psychological Review, 106(1), 20-61.

Dworkin, B, and Miller, N.E. (1986) Failure to replicate visceral learning in the acute curarized rat preparation. Behavioural Neuroscience, 100, 299-314.

Eisenstein, EM, Carlson, AD. (1994) Leg position learning in the cockroach nerve cord using an analog technique. Physiology & Behavior, Vol.56, No.4, Pp.687-691.

Everitt, B. J., & Robbins, T. W. (2005). Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nature Neuroscience, 8(11), 1481-1489.

Everitt, B. J., Dickinson, A., & Robbins, T. W. (2001). The neuropsychological basis of addictive behaviour. Brain Research Reviews, 36(2-3), 129-138.

Ferster, C.B. and Skinner, B.F. (1957) Schedules of Reinforcement. Appleton-Century-Crofts: New York.

Foo, P., Warren, W. H., Duchon, A., & Tarr, M. J. (2005). Do humans integrate routes into a cognitive map? Map- versus landmark-based navigation of novel shortcuts. J. Exp. Psy. -Learning Memory and Cognition, 31(2), 195-215.

Gallistel, C.R. (1990) The Organization of Learning. Cambridge, Mass.: MIT Press

Garcia, J. (1981) Tilting at the paper mills of academe. American Psychologist, 36, 149-58.

Garcia, J., Ervin, F., Yorke, C. and Koelling, R. (1967) Conditioning with delayed vitamin injections. Science, 155, 716-8.

Garcia, J., Hankins, W. and Rusiniak, K. (1974) Behavioural regulation of the milieu interne in man and rat. Science, 185, 824-31.

Gibson, B. M., & Kamil, A. C. (2001). Tests for cognitive mapping in Clark's nutcrackers (Nucifraga columbiana). Journal of Comparative Psychology, 115(4), 403-417.

Goss-Custard, J.D. (1977) Optimal foraging and the size selection of worms by redshank, Tringa totanus, in the field. Animal Behaviour, 25, 10-29.

Harper, D.G.C. (1982) Competitive foraging in mallards: 'ideal free' ducks. Animal Behaviour, 30, 575-584.

Hawkins, R. D., Clark, G. A., & Kandel, E. R. (2006). Operant conditioning of gill withdrawal in Aplysia. Journal of Neuroscience, 26(9), 2443-2448.

Herrnstein, R.J. (1970) On the law of effect. Journal of the Experimental Analysis of Behaviour, 13, 243-66.

Herrnstein, R.J. (1977) The evolution of behaviourism. American Psychologist, 32, 593-613.

Hogarth, L. A., Roberts, W. A., Roberts, S., & Abroms, B. (2000). Spatial localization of a goal: Beacon homing and landmark piloting by rats on a radial maze. Animal Learning and Behavior, 28(1), 43-58.

Hogarth, L., Dickinson, A., Hutton, S. B., Elbers, N., & Duka, T. (2006). Drug expectancy is necessary for stimulus control of human attention, instrumental drug-seeking behaviour and subjective pleasure. Psychopharmacology, 185(4), 495-504.

Holland, P. C., & Petrovich, G. D. (2005). A neural systems analysis of the potentiation of feeding by conditioned stimuli. Physiology & Behavior, 86(5), 747-761.

Hull, C.L. (1943) Principles of Behaviour. Appleton-Century-Crofts: New York.

Hull, C.L. (1952) A Behaviour System. Yale University Press: New Haven.

Hyman, S. E., Malenka, R. C., & Nestler, E. J. (2006). Neural mechanisms of addiction: The role of reward-related learning and memory. Annual Review of Neuroscience, 29, 565-598.

Ikemoto, S., & Panksepp, J. (1999). The role of nucleus accumbens dopamine in motivated behavior: A unifying interpretation with special reference to reward-seeking. Brain Research Reviews, 31(1), 6-41.

Ikemoto, S., & Wise, R. A. (2004). Mapping of chemical trigger zones for reward. Neuropharmacology, 47(Supplement 1), 190-201.

Inglis, IR, Forkman, B and Lazarus, J (1997) Free food or earned food? A review and fuzzy model of contrafreeloading. Animal Behaviour, Vol.53, No.Pt6, Pp.1171-1191.

Jacobs, L. F. (2003). The evolution of the cognitive map. Brain Behavior and Evolution, 62(2), 128-139.

Jacobs, L. F., & Schenk, F. (2003). Unpacking the cognitive map: The parallel map theory of hippocampal function. Psychological Review, 110(2), 285-315.

Jenkins, H.M. and Moore, B.R. (1973) The form of the autoshaped response with food or water reinforcers. Journal of the Experimental Analysis of Behaviour, 20, 163-81.

Knutson, B., Adams, C. M., Fong, G. W., & Hommer, D. (2001). Anticipation of increasing monetary reward selectively recruits nucleus accumbens. Journal of Neuroscience, 21(16), U1-U5.

Konorski, J. and Miller, S. (1937) On two types of conditioned reflex. Journal of General Psychology, 16, 264-72.

A complete on-line version of this is provided in the “Classics in History of Psychology Series” (U of York, Canada)

Krebs, J.R. and Davies, N.B. (eds.) (1983) Behavioural Ecology. An Evolutionary Approach. 2nd Edition.. Blackwell Scientific Publications: Oxford.

Krebs, J.R., Kacelnik, A. and Taylor, P. (1978) Tests of optimal sampling by foraging great tits. Nature, 275, 27-31.

Kringelbach, M. L., & Rolls, E. T. (2004). The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Progress in Neurobiology, 72(5), 341-372.

Lea, S. E. G., & Webley, P. (2006). Money as tool, money as drug: The biological psychology of a strong incentive. Behavioral and Brain Sciences, 29(2), 161-+.

Lea, S.E.G. (1979) Foraging and reinforcement schedules in the pigeon: optimal and non-optimal aspects of choice. Animal Behaviour, 27, 875-86.

Lea, S.E.G. (1984a) Instinct, Environment and Behaviour. Methuen: London

Leszczuk, M. H., & Flaherty, C. F. (2000). Lesions of nucleus accumbens reduce instrumental but not consummatory negative contrast in rats. Behavioural Brain Research, 116(1), 61-79.

Lorenzetti, F. D., Mozzachiodi, R., Baxter, D. A., & Byrne, J. H. (2006). Classical and operant conditioning differentially modify the intrinsic properties of an identified neuron. Nature Neuroscience, 9(1), 17-19.

MacFarlane, D.A. (1930) The role of kinaesthesis in maze learning. University of California Publications in Psychology, 4, 277-305.

Mackintosh, N.J. (1983) Conditioning and Associative Learning. Clarendon Press: Oxford. Chapter 4, 'Theoretical analysis of instrumental conditioning, pp 77-112.

Menzel, R., De Marco, R. J., & Greggers, U. (2006). Spatial memory, navigation and dance behaviour in Apis mellifera. Journal of Comparative Physiology a-Neuroethology Sensory Neural and Behavioral Physiology, 192(9), 889-903.

Menzel, R., Greggers, U., Smith, A., Berger, S., Brandt, R., Brunke, S., et al. (2005). Honey bees navigate according to a map-like spatial memory. Proceedings of the National Academy of Sciences of the United States of America, 102(8), 3040-3045.

Miller, N.E. (1969) Learning of visceral and glandular responses. Science, 163, 434-44.

Miller, N.E. (1978) Biofeedback and visceral learning. Annual Review of Psychology, 29, 373-404.

Miller, N.E. and Dicara, L. (1967) Instrumental learning of heart-rate changes in curarized rats: shaping, and specificity to discriminative stimulus. Journal of Comparative and Physiological Psychology, 63, 12-19.

Nelson, A., & Killcross, S. (2006). Amphetamine exposure enhances habit formation. Journal of Neuroscience, 26(14), 3805-3812.

Neuringer, A.J. (1969) Animals respond for food in the presence of free food. Science, 166, 399-401.

Olton, D.S. (1979) Mazes, maps, and memory. American Psychologist, 34, 583-96.

Olton, D.S. and Samuelson, R.J. (1976) Remembrance of places passed: spatial memory in rats. Journal of Experimental Psychology: Animal Behaviour Processes, 2, 97-116.

Osthaus, B., Lea, S. E. G., & Slater, A. M. (2005). Dogs (Canis lupus familiaris) fail to show understanding of means-end connections in a string-pulling task. Animal Cognition, 8(1), 37-47.

Packard, M. G., & Knowlton, B. J. (2002). Learning and memory functions of the basal ganglia. Annual Review of Neuroscience, 25, 563-593.

Raby, C. R., Alexis, D. M., Dickinson, A., & Clayton, N. S. (2007). Planning for the future by western scrub-jays. Nature, 445(7130), 919-921.

Rescorla, R.A. (1991) Associative relations in instrumental learning: The eighteenth Bartlett Memorial Lecture. Quarterly Journal of Experimental Psychology, 43B, 1-23.

Rescorla, R.A. (1994) Control of instrumental performance by Pavlovian and instrumental stimuli. Journal of Experimental Psychology: Animal Behaviour Processes, 20, 44-50.

Roberts, A. C. (2006). Primate orbitofrontal cortex and adaptive behaviour. Trends in Cognitive Sciences, 10(2), 83-90.

Rodd, Z. A., Bell, R. L., Sable, H. J. K., Murphy, J. M., & McBride, W. J. (2004). Recent advances in animal models of alcohol craving and relapse. Pharmacology Biochemistry and Behavior, 79(3), 439-450.

Rolls, E. T. (2004). The functions of the orbitofrontal cortex. Brain and Cognition, 55(1), 11-29.

Schultz, W. (2006). Behavioral theories and the neurophysiology of reward. Annual Review of Psychology, 57, 87-115.

Segal, RL, Wolf, SL. (1994) Operant-conditioning of spinal stretch reflexes in patients with spinal-cord injuries. Experimental Neurology, Vol.130, No.2, Pp.202-213.

Seligman, M.E.P. (1970) On the generality of the laws of learning. Psychological Review, 77, 406-18..

Sheffield, F.D. (1965) Relation between classical conditioning and instrumental learning. In Prokasy, W.F. (ed.) Classical Conditioning. Appleton-Century-Crofts: New York, 302-322.

Shiels, K., Hawk, L. W., Richards, J. B., & Colder, C. R. (2007). Individual differences in conditioned reward: The observing procedure. Personality and Individual Differences, 42(1), 15-25.

Skinner, B.F. (1938) The Behaviour of Organisms. Appleton-Century-Croft: New York.

Skinner, B.F. (1950) Are theories of learning necessary? Psychological Review, 57, 193-216.

A complete on-line version of this is provided in the “Classics in History of Psychology Series” (U of York, Canada)

Skinner, B.F. (1977) Herrnstein and the evolution of behaviourism. American Psychologist, 32, 1006-12.

Small, D. M., Zatorre, R. J., Dagher, A., Evans, A. C., & Jones-Gotman, M. (2001). Changes in brain activity related to eating chocolate - From pleasure to aversion. Brain, 124, 1720-1733.

Spicer, J., Galvan, A., Hare, T. A., Voss, H., Glover, G., & Casey, B. J. (2007). Sensitivity of the nucleus accumbens to violations in expectation of reward. Neuroimage, 34(1), 455-461.


Thorndike, E.L. (1898) Animal Intelligence: an experimental study of the associative processes in animals. Psychological Review, Monograph Supplements, 2(8), 1-109. (excerpt re-printed, 1998, American Psychologist, 53, 1125-1127.)

A complete on-line version of this is provided in the “Classics in History of Psychology Series” (U of York, Canada)

back to text about Thorndike

Thorndike, E.L. (1931) Human Learning. Century: London.

Tindell, A. J., Smith, K. S., Pecina, S., Berridge, K. C., & Aldridge, J. W. (2006). Ventral pallidum firing codes hedonic reward: When a bad taste turns good. Journal of Neurophysiology, 96(5), 2399-2409.

Tinkelpaugh, O.L. (1928) An experimental study of representative factors in monkeys. Journal of Comparative Psychology, 8, 197-236.

Tolman, E.C. (1932) Purposive Behaviour in Animals and Men. Century: New York.

Tolman, E.C. (1948) Cognitive maps in rats and men. Psychological Review, 55, 189-208.

A complete on-line version of this is provided in the “Classics in History of Psychology Series” (U of York, Canada)

Tolman, E.C. (1949) There is more than one kind of learning. Psychological Review, 56, 144-155

Tolman, E.C. and Honzik, C.H. (1930b) Introduction and removal of reward, and maze learning in rats. University of California Publications in Psychology, 4, 257-75.

Tolman, E.C., Ritchie, B.F. and Kalish, D. (1946) Studies in spatial learning: I. Orientation and the short cut. Journal of Experimental Psychology, 36, 13-24.

Vanderschuren, L., & Everitt, B. J. (2004). Drug seeking becomes compulsive after prolonged cocaine self- administration. Science, 305(5686), 1017-1019.

Volkow, N. D., Fowler, J. S., & Wang, G. J. (2004). The addicted human brain viewed in the light of imaging studies: brain circuits and treatment strategies. Neuropharmacology, 47, 3-13.

Wang, G. J., Volkow, N. D., Thanos, P. K., & Fowler, J. S. (2004). Similarity between obesity and drug addiction as assessed by neurofunctional imaging: A concept review. Journal of Addictive Diseases, 23(3), 39-53.

Wang, R. X. F., & Spelke, E. S. (2000). Updating egocentric representations in human navigation. Cognition, 77(3), 215-250.

www.version of this paper (log on first for access outside the College)

Wang, S. H., Ostlund, S. B., Nader, K., & Balleine, B. W. (2005). Consolidation and reconsolidation of incentive learning in the amygdala. Journal of Neuroscience, 25(4), 830-835.

Watanabe, M., Cromwell, H. C., Tremblay, L., Hollerman, J. R., Hikosaka, K., & Schultz, W. (2001). Behavioral reactions reflecting differential reward expectations in monkeys. Experimental Brain Research, 140(4), 511-518.

Wehner, R., Boyer, M., Loertscher, F., Sommer, S., & Menzi, U. (2006). Ant navigation: One-way routes rather than maps. Current Biology, 16(1), 75-79.

Williams, D.R. and Williams, H. (1969) Auto-maintainance in the pigeon: sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behaviour, 12, 511-20.

Wilson, D. I. G., Laidlaw, A., Butler, E., Hofmann, D., & Bowman, E. M. (2006). Development of a behavioral task measuring reward "wanting" and "liking" in rats. Physiology & Behavior, 87(1), 154-161.

Wise, R. A. (2006). Role of brain dopamine in food reward and reinforcement. Philosophical Transactions of the Royal Society B-Biological Sciences, 361(1471), 1149-1158.

Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004). Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. European Journal of Neuroscience, 19(1), 181-189.