In 1866 the St Petersburg Censorial Committee banned a popular book, and prosecuted its author for undermining public morals. The author was not a political theorist, or a pornographer, but a physiologist called Sechenov. The book, Reflexes of the Brain, introduced the controversial suggestion that 'all acts of conscious or unconscious life are reflexes'. This assertion seemed to over-simplify human existence by reducing thought and action to mechanical processes. However, the Censorial Committee's view that public morals would suffer because of this oversimplification was not supported by the courts, and the case against Sechenov collapsed.
A few years later, in London, The Times was concerned that `morality would lose all elements of stable authority' if the public were to believe the ideas expressed in another controversial book, Darwin's The Descent of Man. But in the next decade Darwin was buried in Westminster Abbey, with The Times' full approval.
The two ideas which initially angered the critics of Sechenov and Darwin are central to the field of `learning and reinforcement' in psychology. The first idea is that human acts and thoughts can be interpreted as reflexes, or learned responses, and the second that human psychology is closely connected to animal psychology. Despite the recognition eventually given to Sechenov and Darwin there is still today considerable unease about this approach to human psychology. It is judged as immoral, or ineffective, and often as both. It is certainly ineffective
in the sense that doing laboratory experiments in the search for psychological principles is a very indirect way of finding out about particularly human activities. For instance, in the psychology of industry or advertising, direct specialized information based on experience might be a much more effective source of knowledge about these activities (and can be found in unit E of Essential Psychology). However, if there are any general principles which underlie human and animal behaviour, then knowledge of them might very well add something to our capacity for understanding and changing human behaviour in a wide variety of contexts. In fact during the last twenty years, considerable advances have been made in the application of general psychological principles in the areas of education and mental illness.
To the extent that such applications work, it is sometimes felt that it is immoral to use them, because psychologists ought not to use their knowledge to manipulate others. Whereas The Times was concerned in 1871 lest the publication of Darwin's book should spread the revolution of the Paris Commune to England, critics of B. F. Skinner, who is in some ways the Sechenov of the present day, seem to fear totalitarian manipulation along the lines of Aldous Huxley's Brave New World. It is wise to take account of such forebodings, but it may be that new theories which apparently demystify and belittle the human psyche are strange and threatening, and become identified with other threatening issues without much justification. The aim of this book is to allow readers to become more familiar with the work of Skinner, and other theories in the tradition of Sechenov and Darwin, so that they may make their own judgements about whether such investigations pose threats, or whether they offer a useful basis for a deeper understanding of human problems with more successful design of practical solutions. In this chapter I will give a brief account of how the current concepts of learning and reinforcement have been developed.
deeply concerned with differences between man and animals. To support the revolutionary belief that man is a species which has gradually evolved from an ape-like ancestor, Darwin devoted two chapters of his book The Descent of Man (the book which outraged The Times in 1871) to evidence which had convinced him that all of the `mental powers' characteristic of man were possessed in a humbler form by lower animals. Because Darwin made rather wild inferences from casual observations of animals, he went further in the direction of over-estimating their abilities than would be justified by modem experimental methods. He attributed to various animals not only `attention', `imagination', `reason' and `abstraction' but primitive forms of the `sense of beauty', `belief in God' and `moral sense'. Although these terms now sound far-fetched, much of the observational data Darwin quoted has remained fresh; for example, tool-using by apes and the use of informative warning signals by birds are current research topics.
While acknowledging the `immense' superiority of human mental powers, Darwin was satisfied that they were not `fundamentally' different from those possessed by other animals, and thus presented no barrier to his theory of evolution. For instance, man's ability to use language was solely due to his `almost infinitely larger power of associating together the most diversified sounds and ideas' (Darwin, 1871, p. 131). So, by firmly connecting human with animal mental capacities, and by identifying the power of association as a critical factor in human intelligence, Darwin prepared the ground for 'associationist' theories about psychology, such as Pavlov's, which derived their supporting evidence from experiments with animals.
Pavlov and the conditioned reflex
concept that some animal is to be discovered or hunted?' asked Darwin. The answer is no. There are any number of other possibilities. The eagerness of Darwin's voice may simply have excited her. Darwin's attention or attitude may have encouraged (and `reinforced') her hunting responses. There is just no way to deduce an explanation of the dog's behaviour from the casual observation.
Pavlov provided observations that were the opposite of casual. Dogs were placed in a separate room from the observer. The most scrupulous care was taken to avoid extraneous influences by such measures as using double walls filled with sand for sound insulation. Above all, a quantitative and completely unambiguous index was used to measure the outcome of experiments : the number of drops of saliva which the dog secreted. How did Pavlov come to choose this unlikely-sounding index? In fact, by not choosing it at all, but by making an accidental discovery. As Alexander Fleming is said to have done in his discovery of penicillin, Pavlov paid full attention to an unexpected result. He was studying salivation from the point of view of a professor of physiology who had already been awarded the Nobel prize in his special field of digestive secretions. There was an unexpected finding - that dogs often salivated when there seemed no physiological reason for them to do so. Pavlov realized that these were `psychological secretions' and went on to make a systematic experimental analysis of exactly what psychological factors were needed to produce the salivation. As we shall discuss in Chapter 3, the main factor is that the dog learns an association between food and a signal for food. Thus if a bell is always rung just before a dog is given food, the dog salivates when he hears the bell, and we have Pavlov's well known `conditioned reflex'. The laboratory findings with the reflex of salivation were taken as a model for a wide range of psychological functions: `It is obvious that the different kinds of habits based on training, education and discipline of any sort are nothing but a long chain of conditioned reflexes' (Pavlov, 1927, p. 395). Here is an echo of Sechenov's slogan `all acts are reflexes', and although Pavlov was not taught by him, Pavlov's work seems to put into practice Sechenov's recommendation of studying psychology via reflexes.
Watson and behaviourism
The suggestion that psychologists should ignore inner feelings and thoughts was first made by J. B. Watson in 1913. Watson's initial interest was in animal psychology, and he was concerned by the difficulty of applying the methods used by his colleagues working with people. Their methods were based on self-analysis ('introspection') by either the psychologist or his subjects. Even if animals are capable of introspection, it is very difficult to be certain about what their feelings and thoughts are (but see Ch. 9). However, one can be reasonably sure about what animals are doing, if their behaviour is described in an objective and reliable way. Watson found that some of the questions he was concerned with, such as whether animals see colours, could be answered much more effectively if the issue of the animal's subjective, sensations was ignored. For instance, it is much easier to find out if an animal can respond differently to red and green than it is to answer the question of what it `feels like' to the animal to see red or green colours.
Having shown that it was helpful to ignore the subjective impressions of animals, Watson strongly recommended this strategy to psychologists in general. The methodology of measuring what people do, instead of trying to measure what people say they feel, became very widely used even by psychologists who did not agree with other theories that Watson put forward. This practice of relying on behavioural evidence is the crucial aspect of behaviourism, which was the name given by Watson to his recommendations.
The importance of learning. One of Watson's theories was that all human skills, personality traits and motives are learned. He argued against the view that abilities `run in families' because they are inherited, and against the idea that there are inherited psychological differences between races. Nothing of any psychological importance is inherited in Watson's opinion, but all psychological differences between individuals are due to differences in upbringing, training or past experience. After showing that Pavlovian conditioning procedures could change the emotional preferences of babies (see Ch. 3). Watson came to the conclusion that even complex aspects of adult personality were the products of `millions of conditionings'.
It is still characteristic of behaviourists to believe that processes of learning pervade human life. Learning in this sense is
not just something that happens when you set about `learning French' or `learning to play tennis'. It is something which happens continually, whether you like it or not, and modifies your behaviour towards other people and changes your opinions as well as operating in a more obvious manner to improve skills or academic ability. Watson's version of Sechenov's slogan would be `all acts of conscious or unconscious life are determined by learning'.
Thorndike and the Law of Effect
Thorndike was a contemorary of Watson's whose work was along similar lines. However, Thorndike had his own independent theories about learning, and the most influential of these was expressed in the Law of Effect which was based on his early experiments with animals. Thorndike was attracted to psychology by reading William James' Principles, and went as a postgraduate student to Harvard, where James was preparing to change back to professor of philosophy from professor of psychology. Despite his shift back to philosophy, it was James who, when Thorndike was turned out of his lodgings for hatching chickens in the bedroom, gave the young Thorndike and the chickens room in his own home. Thorndike's experiments on learning in chicks were repeated with cats and dogs, and produced the Law of Effect, which applies to voluntary movements and behaviour. In the experiments animals were left inside a `puzzle-box' to solve the problem of how to get out. The solution involved finding and operating a latch. The results persuaded Thorndike (1898) that, although the animals eventually became adept at escaping from confinement, they did not do it by some form of reasoning or thought but by automatically remembering accidental successes. Cats, for instance, might cry and scratch and turn around in the box in an unsystematic way, eventually being let out when they scratched at the latch. But this does not produce a realization by the cat that `to get out one has to scratch the latch'. On being put back in the box the cat again scratches and turns at random. Learning does take place, but only by gradual adjustments to behaviour so that over a sequence of twenty attempts the cat escapes slightly more quickly each time.
According to the Law of Effect, the improvement comes about because the actions which work the latch are gradually
strengthened by the success, pleasure or satisfaction produced by getting out of the box (which is the `effect' of the actions).
The systematic behaviour theory of C. L. Hull
Hull tried to bring together the findings of Pavlov and Thorndike and use them as the basis of a very special kind of theory. He was an experienced psychologist by the time he started this, having published books on aptitude testing and hypnosis. But he became fascinated by the idea of constructing psychological laws that were not vague suggestions but mathematical equations. Pavlov and Thorndike had demonstrated that scientific methods of measurement could be applied to learned behaviour, and Hull advocated that these measurements could be used to sustain a type of theory more like theories in physics. In his last attempt to construct a system like this himself (Hull, 1952) there were thirty-two equations, but the main one of general application was this:
It means that the intensity or likelihood of any item of learned behaviour (SER) can be calculated if four other factors are known : the drive or motivation associated with it (D), the intensity of the signal for the behaviour (V), the degree of incentive used (K) and the degree of habit (SHR). Under controlled laboratory conditions all the factors can be measured, and the equation checked: SER by the probability or intensity of response, drive by hours of deprivation or physical need, incentive by the level of reward used, and habit by the amount of practice given.
The ideals behind the theory, and the possibility of confirming or disproving theories by laboratory experiment, made Hull's theory extremely influential for a number of years. However, most of the equations were found to be of limited application, if not incorrect, and enthusiasm for such an ambitious approach has waned. Nonetheless, many of the innovations and suggestions made by Hull have survived and his goal of a `hypotheticodeductive' theory of psychology, which allows precise predictions to be made and tested, has not been totally abandoned, especially for use on a small scale.
E. C. Tolman - behaviourism with thinking
Tolman was a behaviourist in so far as he accepted only precise measurements of behaviour as evidence, but he would nowadays be called a`cognitive' theorist, since he inferred from his evidence that his animal subjects had thoughts, in the form of `expectancies', `insights', `cognitive maps' and `hypotheses'. He narrowed the gap between animals and men by attributing thought processes to animals, like Darwin, rather than emphasizing automatic processes in man,like Sechenov. Behaviourists are often accused of mistaking their evidence for the substance of their inquiries, and Tolman's work was an important corrective against this tendency. There is a difference between knowing and doing even for laboratory rats, as Tolman demonstrated in a variety of experiments which measured rats' proficiency at finding their way around ingeniously-designed mazes.
Latent learning. Tolman and Honzik (1930) demonstrated that rats could find their way about in a maze during idle exploration, so that when they were motivated to get out of the maze quickly, by use of a food reward at the end, little additional learning was necessary. After this the distinction between the experiences necessary for learning, and the conditions which favour the performance of learned behaviour, received more emphasis. It is incorporated for instance into Hull's equation given above, in that the reward factor and the practice factor are kept separate.
Cognitive maps. Tolman suggested that an important part of learning is knowledge of 'what-leads-to-what', which he termed an expectancy. This is an alternative to Hull's concept of habit, and emphasizes that information about the environment can be acquired independently of particular responses to the environment. The maze-solving abilities of rats imply that rats learn something about the maze; having learned to run around a maze, they may for instance be able to swim successfully around a similar maze, or negotiate the maze backwards. Tolman said that this ability was due to the formation of `cognitive maps' by the animals, which could be used later to guide various kinds of movements.
Skinner's radical behaviourism
Although Skinner's first book appeared in 1938, his work is currently the most influential of any psychologist in the field of learning. This is partly because he himself is continuing to supply important theoretical contributions and provocative statements of his views on the implication of his findings for the public at large (Skinner, 1971, 1974). There is also much work being done by others directly in the Skinnerian tradition, both in the sense of laboratory experiment and in other applications. However, it is also arguable that Skinner's influence is necessarily less subject to shifts of opinion or advances in other fields, in so far as it depends largely on experimental fact rather than speculative explanations. Although he himself does not do so, it is worth separating Skinner's innovations in laboratory methods and procedures, and his discoveries of experimental facts, from his philosophical position on the use and implication of these methods and facts.
Facts of operant conditioning. By repeatedly giving small bits of food to hungry animals, Skinner demonstrated the practical effects of these food rewards extremely dramatically. He calls these effects positive reinforcement, and the bits of food reinforcers. The drama was supplied by careful training of individual animals in complex tasks. This can be done simply by positive reinforcement, if a little-by-little method is used to develop gradually or shape the behaviour of the subject towards the final form. For instance, Skinner trained a rat to release a marble by pulling a string, and then to carry the marble away and drop it in a tin. This is a fairly difficult task for a rat, and the rat cannot easily be told or shown what to do. But the information can be conveyed, in the limited sense that the rat learns what to do, by the use of the shaping procedure (sometimes termed successive approximation). For instance, the rat might never lift the marble to drop it in the can without training, but Skinner started with a can only a fraction of an inch high, and when the rat had learned, by being reinforced, to lift the marble that distance, increased the height. After several small increases, the rat had learned to make the complete lift.
In addition to such demonstrations, Skinner developed more general experimental methods. In contrast to Thorndike, who conducted experiments in his bedroom (which housed several
species of animal as well as Thorndike), Skinner, like Pavlov, used carefully-controlled laboratory conditions. Animals were typically isolated from outside disturbance by being placed inside a special sound-proofed chamber (often called a 'Skinner box'). The standard procedure was to use the delivery of pellets of food to make a rat press the lever in the box, with automatic devices to record presses of the lever and to deliver the reinforcer. By isolating these two variables, the response of pressing the lever and the reinforcer of food delivery, it was possible to observe how completely the reinforcer controlled the response (see Ch. 4). Contingencies of reinforcement are fixed relations between the response, the reinforcer and signals presented to the subject. One kind of contingency is the schedule of reinforcement, a straight-forward rule about the delivery of reinforcers such as `a reinforcer is delivered at every third response'. Skinner's discovery is that contingencies of reinforcement exert an extremely powerful influence on behaviour.
Skinner's extrapolations. Having established the intimate relationship between behaviour and contingencies of reinforcement in the laboratory, Skinner has gone on to assert the importance of these contingencies in every aspect of psychology. It is fair to paraphrase Skinner's views in the Sechenov form : `All acts of conscious or unconscious life are determined by contingencies of reinforcement'.
Skinner now makes a further leap of the imagination. He insists that in government, education and economics, matters can be much improved if contingencies of reinforcement are designed so as to produce desired behaviours. He suggests that Utopia is indeed attainable by the use of such methods (Skinner, 1971). There may be something in this, and extreme positions are often necessary to make a point. But it is possible to distinguish very sharply between what has been established in the laboratory, what looks promising in some applied fields, and what are Skinner's visions and extrapolations.
The evidence to show that contingencies of reinforcement control behaviour under laboratory conditions is overwhelming. It seems quite probable that principles derived from laboratory conditions can be helpful in some aspects of education and mental health. However, there is absolutely no evidence to show that principles discovered by Skinner are appropriate for largescale application in politics and economics. It is therefore not
surprising that there are reservations about using Skinner's `test-tube' methods in these complicated and sensitive areas of real life.
Radical behaviourism. Another extreme position taken by Skinner may stand him in better stead. This is his self-restraint in the matter of speculative explanations. Explanations of why or how contingencies of reinforcement work would be interesting, but Skinner has avoided making any. His point is that descriptions of behaviour, and relations between behaviour and variables we can be certain of, are reliable, and are sufficient explanations in themselves.
We may want to know how our behaviour is related to (a) our subjective experience and (b) the activity of our brains. Skinner suggests that (a) our subjective experience is simply a kind of behaviour and (b) knowledge of physiology will not change the behavioural facts. This radical position has worked to Skinner's advantage in that the behavioural facts which he emphasized have not become outdated because of advances in neurophysiology or changes in theories of cognition. However, it is likely that as knowledge about the physiology of the brain increases, the advantages of ignoring it will diminish.
Conclusions and summary
Since Darwin and Sechenov, scientists have been giving radical answers to various riddles about human existence. Darwin's claim that the human species has evolved from other animal species is now generally accepted, and there are many psychologists who agree with his argument that evolution applies just as well to psychology as to anatomy. Behaviourists especially, since they discount subjective impressions, usually stress the continuity between human and animal behaviour. The riddle of man's obviously more varied behaviour has been given the answer of `learning', man being supposed to learn faster than and better, though not differently, from other animals. Consequently, a good deal of attention has been devoted to the study of learning in animals, because of this background assumption that learning is important for the rest of psychology.
The remaining chapters are about the different sorts of learning which have been studied experimentally, with examples of how facts discovered in this way are currently being applied to alleviate problems in human behaviour.