Primary Sources

Positive Reinforcement Produced by Electrical Stimulation of Septal Area and Other Regions of Rat Brain

By JAMES OLDS AND PETER MILNER

MCGILL UNIVERSITY

Stimuli have eliciting and reinforcing functions. In studying the former, one concentrates on the responses which come after the stimulus. In studying the latter, one looks mainly at the responses which precede it. In its reinforcing capacity, a stimulus increases, decreases, or leaves unchanged the frequency of preceding responses, and accordingly it is called a reward, a punishment, or a neutral stimulus (cf. 16).

Previous studies using chronic implantation of electrodes have tended to focus on the eliciting functions of electrical stimuli delivered to the brain (2, 3, 4, 5, 7, 10, 12, 14). The present study, on the other hand, has been concerned with the reinforcing function of the electrical stimulation.

Method

GENERAL

Stimulation was carried out by means of chronically implanted electrodes which did not interfere with the health or free behavior of Ss to any appreciable extent. The Ss were 15 male hooded rats, weighing approximately 250gm. at the start of the experiment. Each S was tested in a Skinner box which delivered alternating current to the brain so long as a lever was depressed. The current was delivered over a loose lead, suspended from the ceiling, which connected the stimulator to the rat’s electrode. The Ss were given a total of 6 to 12hr. of acquisition testing, and 1 to 2hr. of extinction testing. During acquisition, the stimulator was turned on so that a response produced electrical stimulation; during extinction, the stimulator was turned off so that a response produced no electrical stimulation. Each S was given a percentage score denoting the proportion of his total acquisition time given to responding. This score could be compared with the animal’s extinction score to determine whether the stimulation had a positive, negative, or neutral reinforcing effect. After testing, the animal was sacrificed. Its brain was frozen, sectioned, stained, and examined microscopically to determine which structure of the brain had been stimulated. This permitted correlation of acquisition scores with anatomical structures.

ELECTRODE IMPLANTATION

Electrodes are constructed by cementing a pair of enameled silver wires of 0.010-in. diameter into a Lucite block, as shown in Figure 1.01. The parts of the wires which penetrate the brain are cemented together to form a needle, and this is cut to the correct length to reach the desired structure in the brain. This length is determined from Krieg’s rat brain atlas (11) with slight modifications as found necessary by experience. The exposed cross section of the wire is the only part of the needle not insulated from the brain by enamel; stimulation therefore occurs only at the tip. Contact with the lead from the stimulator is made through two blobs of solder on the upper ends of the electrode wires; these blobs make contact with the jaws of an alligator clip which has been modified to insulate the two jaws from one another. A light, flexible hearing-aid lead connects the clip to the voltage source.

FIGURE 1.01

Electrode design (see text for detailed description).

The operation of implantation is performed with the rat under Nembutal anesthesia (0.88 cc/Kg) and held in a Johnson-Krieg stereotaxic instrument (11). A mid-line incision is made in the scalp and the skin held out of the way by muscle retractors. A small hole is drilled in the skull with a dental burr at the point indicated by the stereotaxic instrument for the structure it is desired to stimulate. The electrode, which is clamped into the needle carrier of the instrument, is lowered until the flange of the Lucite block rests firmly on the skull. Four screw holes are then drilled in the skull through four fixing holes in the flange, and the electrode, still clamped firmly in the instrument, is fastened to the skull with jeweler’s screws which exceed the diameter of the screw holes in the skull by 0.006in. The electrode is then released from the clamp and the scalp wound closed with silk sutures. The skin is pulled tightly around the base of the Lucite block and kept well away from the contact plates. A recovery period of three days is allowed after the operation before testing.

TESTING

The testing apparatus consisted of a large-levered Skinner box 11in. long, 5in. wide, and 12in. high. The top was open to allow passage for the stimulating lead. The lever actuated a microswitch in the stimulating circuit so that when it was depressed, the rat received electrical stimulation. The current was obtained from the 60-cycle power line, through a step-down transformer, and was adjustable between 0 and 10v. r.m.s. by means of a variable potentiometer. In the experiments described here the stimulation continued as long as the lever was pressed, though for some tests a time delay switch was incorporated which cut the current off after a predetermined interval if the rat continued to hold the lever down. Responses were recorded automatically on paper strip.

On the fourth day after the operation rats were given a pretesting session of about an hour in the boxes. Each rat was placed in the box and on the lever by E with the stimulus set at 0.5v. During the hour, stimulation voltage was varied to determine the threshold of a "just noticeable" effect on the rat’s behavior. If the animal did not respond regularly from the start, it was placed on the lever periodically (at about 5-min. intervals). Data collected on the first day were not used in later calculations. On subsequent days, Ss were placed in the box for about 3hr. a day; these were 3hr. of acquisition and hr. of extinction. During the former, the rats were allowed to stimulate themselves with a voltage which was just high enough to produce some noticeable response in the resting animal. As this threshold voltage fluctuated with the passage of time, E would make a determination of it every half hour, unless S was responding regularly. At the beginning of each acquisition period, and after each voltage test, the animal was placed on the lever once by E. During extinction periods, conditions were precisely the same except that a bar press produced no electrical stimulation. At the beginning of each extinction period, animals which were not responding regularly were placed on the lever once by E. At first, rats were tested in this way for four days, but as there appeared to be little difference between the results on different days, this period was reduced to three and then to two days for subsequent animals. Thus, the first rats had about 12hr. of acquisition after pretesting whereas later rats had about 6hr. However, in computing the scores in our table, we have used only the first 6hr. of acquisition for all animals, so the scores are strictly comparable. In behavioral curves, we have shown the full 12hr. of acquisition on the earlier animals so as to illustrate the stability of the behavior over time.

At no time during the experiment were the rats deprived of food or water, and no reinforcement was used except the electrical stimulus.

Animals were scored on the percentage of time which they spent bar pressing regularly during acquisition. In order to find how much time the animal would spend in the absence of reward or punishment, a similar score was computed for periods of extinction. This extinction score provided a base line. When the acquisition score is above the extinction score, we have reward; when it is below the extinction score, we have punishment.

In order to determine percentage scores, periods when the animal was responding regularly (at least one response every 30 sec.) were counted as periods of responding; i.e., intervals of 30 sec. or longer without a response were counted as periods of no responding. The percentage scores were computed as the proportion of total acquisition or extinction time given to periods of responding.

DETERMINATION OF LOCUS

On completion of testing, animals were perfused with physiological saline, followed by 10 per cent formalin. The brains were removed, and after further fixation in formalin for about a week, frozen sections 40 microns thick were cut through the region of the electrode track. These were stained with cresyl violet and the position of the electrode tip determined.

Results

LOCUS

In Table 1.01, acquisition and extinction scores are correlated with electrode placements. Figure 1.02 presents the acquisition scores again, this time on three cross-sectional maps of the rat brain, one at the forebrain level, one at the thalamic level, and one at the mid-brain level. The position of a score on the map indicates the electrode placement from which this acquisition score was obtained.

The highest scores are found together in the central portion of the forebrain. Beneath the corpus callosum and between the two lateral ventricles in section I of Figure 1.02, we find four acquisition scores ranging from 75 to 92 per cent. This is the septal area. The Ss which produced these scores are numbered 32, 34, M-1, and M-4 in Table 1.01. It will be noticed that while all of them spent more than 75 per cent of their acquisition time responding, they all spent less than 22 per cent of their extinction time responding. Thus the electrical stimulus in the septal area has an effect which is apparently equivalent to that of a conventional primary reward as far as the maintenance of a lever-pressing response is concerned.

If we move outside the septal area, either in the direction of the caudate nucleus (across the lateral ventricle) or in the direction of the corpus callosum, we find acquisition scores drop abruptly to levels of from 4 to 6 per cent. These are definitely indications of neutral (neither rewarding nor punishing) effects.

However, above the corpus callosum in the cingulate cortex we find an acquisition score of 37 per cent. As the extinction score in this case was 9 per cent, we may say that stimulation was rewarding.

At the thalamic level (section II of Fig. 1.02) we find a 36 per cent acquisition score produced by an electrode placed again in the cingulate cortex, an 11 per cent score produced by an electrode placed in the hippocampus, a 71 per cent score produced by an electrode placed exactly in the mammillothalamic tract, and a zero per cent score produced by an electrode placed in the medial lemniscus. The zero denotes negative reinforcement.

At the mid-brain level (section III of Fig. 1.02) there are two zero scores produced by electrodes which are in the posterior portion of the medial geniculate bodies; here again, the scores indicate a negative effect, as the corresponding extinction scores are 31 and 21 per cent. There is an electrode deep in the medial, posterior tegmentum which produces a 2 per cent score; this seems quite neutral, as the extinction score in this case is 1 per cent. Finally, there is an electrode shown on this section which actually stands 1mm. anterior to the point where it is shown; it was between the red nucleus and the posterior commissure. It produced an acquisition score of 77 per cent, but an extinction score of 81 per cent. This must be a rewarding placement, but the high extinction score makes it difficult to interpret.

TABLE 1.01

ACQUISITION AND EXTINCTION SCORE FOR ALL ANIMALS TOGETHER WITH ELECTRODE PLACEMENTS AND THRESHOLD VOLTAGES USED DURING ACQUISITION TESTS

ANIMAL’S NO.	LOCUS OF ELECTRODE	STIMULATION VOLTAGE R.M.S.	PERCENTAGE OF ACQUISITION TIME SPENT RESPONDING	PERCENTAGE OF EXTINCTION TIME SPENT RESPONDING
32	septal	2.2 - 2.8	75	18
34	septal	1.4	92	6
M-1	septal	1.7 - 4.8	85	21
M-4	septal	2.3 - 4.8	88	13
40	c.c.	.7 - 1.1	6	3
41	caudate	.9 - 1.2	4	4
31	cingulate	1.8	37	9
82	cingulate	.5 - 1.8	36	10
36	hip.	.8 - 2.8	11	14
3	m.l.	.5	0	4
A-5	m.t.	1.4	71	9
6	m.g.	.5	0	31
11	m.g.	.5	0	21
17	teg.	.7	2	1
9	teg.	.5	77	81

KEY: c.c., corpus callosum; hip., hippocampus; m.l., medial lemnisus; m.t., Mammilothalamic tract; m.g., medial geniculate; teg., tegmentum.

BEHAVIOR

We turn our attention briefly to the behavioral data produced by the more rewarding electrode placements.

The graph in Figure 1.03 is a smoothed cumulative response curve illustrating the rate of responding of rat No. 32 (the lowest-scoring septal area rat) during acquisition and extinction. The animal gave a total of slightly over 3000 responses in the 12hr. of acquisition. When the current was turned on, the animal responded at a rate of 285 responses an hour; when the current was turned off, the rate fell close to zero.

The graph in Figure 1.04 gives similar data on rat No. 34 (the highest-scoring septal rat). The animal stimulated itself over 7500 times in 12hr. Its average response rate during acquisition was 742 responses an hour; during extinction, practically zero.

FIGURE 1.02

Maps of three sections, (I) through the forebrain, (II) through the thalamus, (III) through the mid-brain of the rat. Boxed numbers give acquisition percentage scores produced by animals with electrodes stimulating at these points. On section I the acquisition scores 75, 88, 92, 85 fall in the septal forebrain area. On the same section there is a score of 4 in the caudate nucleus, a score of 6 in the white matter below the cortex, and a score of 37 in the medial (cingulate) cortex. On section II the acquisition score of 36 is in the medial (cingulate) cortex, II is in the hippocampus, 71 is in the mammillothalamic tract, and 0 is in the medial lemniscus. On section III the two zeroes are in the medial geniculate, 2 is in the tegmental reticular substance, 77 falls 2mm. anterior to the section shown—it is between the posterior commissure and the red nucleus.

Figure 1.05 presents an unsmoothed cumulative response curve for one day of responding for rat No. A-5. This is to illustrate in detail the degree of control exercised by the electrical reward stimulus. While this rat was actually bar pressing, it did so at 1920 responses an hour; that is, about one response for every 2 sec. During the first period of the day it responded regularly while on acquisition, extinguished very rapidly when the current was turned off, and reconditioned readily when the current was turned on again. At reconditioning points, E gave S one stimulus to show that the current was turned on again, but E did not place S on the lever. During longer periods of acquisition, S occasionally stopped responding for short periods, but in the long run S spent almost three-quarters of its acquisition time responding. During the long period of extinction at the end of the day, there was very little responding, but S could be brought back to the lever quite quickly if a stimulus was delivered to show that the current had been turned on again.

Discussion

It is clear that electrical stimulation in certain parts of the brain, particularly the septal area, produces acquisition and extinction curves which compare favorably with those produced by a conventional primary reward. With other electrode placements, the stimulation appears to be neutral or punishing.

Because the rewarding effect has been produced maximally by electrical stimulation in the septal area, but also in lesser degrees in the mammillothalamic tract and cingulate cortex, we are led to speculate that a system of structures previously attributed to the rhinencephalon may provide the locus for the reward phenomenon. However, as localization studies which will map the whole brain with respect to the reward and punishment dimension are continuing, we will not discuss in detail the problem of locus. We will use the term "reinforcing structures" in further discussion as a general name for the septal area and other structures which produce the reward phenomenon.

To provide an adequate canvass of the possible explanations for the rewarding effect would require considerably more argument than could possibly fit within the confines of a research paper. We have decided, therefore, to rule out briefly the possibility that the implantation produces pain which is reduced by electrical stimulation of reinforcing structures, and to confine further discussion to suggestions of ways the phenomenon may provide a methodological basis for study of physiological mechanisms of reward.

FIGURE 1.03

Smoothed cumulative response curve for rat No. 32. Cumulative response totals are given along the ordinate, and hours along the abscissa. The steepness of the slope indicates the response rate. Stimulating voltages are given between black lines. Shading indicates extinction.

The possibility that the implantation produces some painful "drive stimulus" which is alleviated by electrical stimulation of reinforcing structures does not comport with the facts which we have observed. If there were some chronic, painful drive state, it would be indicated by emotional signs in the animal’s daily behavior. Our Ss, from the first day after the operation, are normally quiet, nonaggressive; they eat regularly, sleep regularly, gain weight. There is no evidence in their behavior to support the postulation of chronic pain. Septal preparations which have lived healthy and normal lives for months after the operation have given excellent response rates.

FIGURE 1.04

Smoothed cumulative response curve for rat No. 34.

As there is no evidence of a painful condition preceding the electrical stimulation, and as the animals are given free access to food and water at all times except while actually in the Skinner boxes, there is no explicitly manipulated drive to be reduced by electrical stimulation. Barring the possibility that stimulation of a reinforcing structure specifically inhibits the "residual drive" state of the animal, or the alternative possibility that the first electrical stimulus has noxious after-effects which are reduced by a second one, we have some evidence here for a primary rewarding effect which
is not associated with the reduction of a primary drive state. It is perhaps fair in a discussion to re-port the "clinical impression" of the Es that the phenomenon represents strong pursuit of a positive stimulus rather than escape from some negative condition.

FIGURE 1.05

Unsmoothed cumulative response curve showing about hr. of acquisition and hr. extinction for rat No. A-5. Shading indicates extinction.

Should the latter interpretation prove correct, we have perhaps located a system within the brain whose peculiar function is to produce a rewarding effect on behavior. The location of such a system puts us in a position to collect information that may lead to a decision among conflicting theories of reward. By physiological studies, for example, we may find that the reinforcing structures act selectively on sensory or motor areas of the cortex. This would have relevance to current S-S versus S-R controversies (8, 9, 13, 16).

Similarly, extirpation studies may show whether reinforcing structures have primarily a quieting or an activating effect on behavior; this would be relevant to activation versus negative feedback theories of reward (6, 13, 15, 17). A recent study by Brady and Nauta (1) already suggests that the septal area is a quieting system, for its surgical removal produced an extremely active animal.

Such examples, we believe, make it reasonable to hope that the methodology reported here should have important consequences for physiological studies of mechanisms of reward.

Summary

A preliminary study was made of rewarding effects produced by electrical stimulation of certain areas of the brain. In all cases rats were used and stimulation was by 60-cycle alternating current with voltages ranging from to 5v. Bipolar needle electrodes were permanently implanted at various points in the brain. Animals were tested in Skinner boxes where they could stimulate themselves by pressing a lever. They received no other reward than the electrical stimulus in the course of the experiments. The primary findings may be listed as follows: (a) There are numerous places in the lower centers of the brain where electrical stimulation is rewarding in the sense that the experimental animal will stimulate itself in these places frequently and regularly for long periods of time if permitted to do so. (b) It is possible to obtain these results from as far back as the tegmentum, and as far forward as the septal area; from as far down as the subthalamus, and as far up as the cingulate gyrus of the cortex. (c) There are also sites in the lower centers where the effect is just the opposite: animals do everything possible to avoid stimulation. And there are neutral sites: animals do nothing to obtain or to avoid stimulation. (d) The reward results are obtained more dependably with electrode placements in some areas than others, the septal area being the most dependable to date. (e) In septal area preparations, the control exercised over the animal’s behavior by means of this reward is extreme, possibly exceeding that exercised by any other reward previously used in animal experimentation.

The possibility that the reward results depended on some chronic painful consequences of the implantation operation was ruled out on the evidence that no physiological or behavioral signs of such pain could be found. The phenomenon was discussed as possibly laying a methodological foundation for a physiological study of the mechanisms of reward.

References

1. Brady, J. V., & Nauta, W. J. H. Subcortical mechanisms in emotional behavior: affective changes following septal forebrain lesions in the albino rat. J. Comp. Physiol. Psychol., 1953, 46, 339–346.

2. Delgado, J. M. R. Permanent implantation of multilead electrodes in the brain. Yale J. Biol. Med., 1952, 24, 351–358.

3. Delgado, J. M. R. Responses evoked in waking cat by electrical stimulation of motor cortex. Amer. J. Physiol., 1952, 171, 436–446.

4. Delgado, J. M. R., & Anand, B. K. Increase of food intake induced by electrical stimulation of the lateral hypothalamus. Amer. J. Physiol., 1953, 172, 162–168.

5. Dell, P. Correlations entre le système vegetatif et le système de la vie relation: mesencephale, diencephale, et cortex cerebral. J. Physiol. (Paris), 1952, 44, 471–557.

6. Deutsch, J. A. A new type of behavior theory. Brit. J. Psychol., 1953, 44, 304–317.

7. Gastaut, H. Correlations entre le système nerveux vegetatif et le système de la vie de relation dans le rhinencephale. J. Physiol. (Paris), 1952, 44, 431–470.

8. Hebb, D. O. The organization of behavior. New York: Wiley, 1949.

9. Hull, C. L. Principles of behavior. New York: D. Appleton-Century, 1943.

10. Hunter, J., & Jasper, H. H. Effects of thalamic stimulation in unanaesthetized animals. EEG Clin. Neurophysiol., 1949, 1, 305–324.

11. Krieg, W. J. S. Accurate placement of minute lesions in the brain of the albino rat. Quart. Bull., Northwestern Univer. Med. School, 1946, 20, 199–208.

12. MacLean, P. D., & Delgado, J. M. R. Electrical and chemical stimulation of frontotemporal portion of limbic system in the waking animal. EEG Clin. Neurophysiol., 1953, 5, 91–100.

13. Olds, J. A neural model for sign-gestalt theory. Psychol. Rev., 1954, 61, 59–72.

14. Rosvold, H. E., & Delgado, J. M. R. The effect on the behavior of monkeys of electrically stimulating or destroying small areas within the frontal lobes. Amer. Psychologist, 1953, 8, 425–426. (Abstract)

15. Seward, J. P. Introduction to a theory of motivation in learning. Psychol. Rev., 1952, 59, 405–413.

16. Skinner, B. F. The behavior of organisms. New York: D. Appleton-Century, 1938.

17. Wiener, N. Cybernetics. New York: Wiley, 1949. Received July 15, 1954.

Questions For Reflection And Discussion:

1. Why do you think that the tone or style of the authors is so dry and matter-of-fact?

2. What ethical considerations are involved in the treatment of the animals in this study? What does it mean to say that the rats were "sacrificed" after testing so that the placement of the electrodes could be better determined?

3. What evidence do the authors present that the animals were not in pain during testing?

4. Why do you think some scientists object to use of the term pleasure center in discussions of the behavior of rats? Why do behaviorists prefer to stick to terms like reinforcement? (Hint: In your answer, you may wish to refer to the term anthropomorphizing.)

5. People who refer to this study usually speak of the discovery of a pleasure center in the brain. What else did the researchers discover?

6. Humans have brain areas like those of the rats in this study, but stimulation of these areas has not been shown to have the same dramatic effects. Nevertheless, what are the implications of this line of research for control of the behavior of humans? (Why have some social commentators been frightened by this sort of discovery?)

7. Assume that people have pleasure centers (or reinforcement centers) in the brain and that it is possible to stimulate them as with the rats in the Olds and Milner study. Do you think that legislators might try to make such self-stimulation illegal? Why or why not?

SOURCE

Olds, J., & Milner, P. (1954). Positive reinforcement produced by electrical stimulation of the septal area and other regions of rat brain. Journal of Comparative and Physiological Psychology, 47, 419–427.

Return to index.