Direct and indirect pathway striatal neurons are known to exert opposing control over motor output. In this review, we discuss a hypothetical extension of this framework, in which direct pathway striatal neurons also mediate reinforcement and reward, and indirect pathway neurons mediate punishment and aversion.
Sports matches often end with a similar sight. The winning team runs around the field, pumps their fists, and jumps on top of one another. In contrast, the losing team kneels or lies on the ground, sits on the bench, or walks slowly off the field. In terms of motor output, the winning team can be described as hyperactive and the losing team as hypoactive. In addition to these motor effects, a constellation of hedonic feelings accompany winning and losing. Winning is associated with feelings of vigor and elation, whereas losing is associated with feelings of dejection and regret. Recent evidence suggests that this link between motor activity and hedonic feelings is not a coincidence. Positive feelings may share common neural circuitry with cells that drive movement, whereas negative feelings may share common circuitry with cells that inhibit movement.
Links between movement, reinforcement, and reward are apparent in many neurological and psychiatric diseases. Depression is commonly described by deficits in reward function, coupled with heightened punishment from negative consequences (9). However, other symptoms of depression include slowing of speech, eye and limb movements, as well as abnormalities in posture and facial expression, together termed “psychomotor retardation” (26, 155). In severe cases of depression, individuals become almost akinetic, rarely leaving their beds (15, 134). Conversely, the vast majority of Parkinson's patients experience comorbid non-motor dysfunctions, of which depression is particularly common (50, 122, 165, 173, 177, 185, 189). Medications that improve the motor deficits in Parkinson's disease can relieve this depression (particularly monoamine oxidase inhibitors such as selegiline) (8, 71, 81, 178, 185, 198), suggesting that motor and depressive symptoms might result from common neural pathology. The most prominent cellular pathology in Parkinson's disease is the death of dopaminergic neurons in the substantia nigra pars compacta and the resulting loss of dopamine in the dorsal striatum (48, 49). Although it is clear that this contributes to parkinsonian motor deficits, several recent studies have highlighted non-motor functions of dorsal striatal dopamine and its target neurons (53, 72, 112, 114). These studies support the hypothesis that movement, reinforcement, and reward are mediated by common basal ganglia circuits. Specifically, direct pathway striatal neurons may mediate movement, reinforcement, and reward, whereas indirect pathway neurons inhibit movement and mediate punishment and aversion. In this review, we will discuss evidence that supports this hypothesis and the implications it raises for treating disorders of movement, reinforcement, and reward.
The Involvement of the Striatum in the Generation and Inhibition of Movement
In 1876, David Ferrier summarized decades of prior work on the striatum with the statement: “The results of stimulation of the corpora striata in monkeys, cats, dogs, jackals and rabbits are so uniform as to admit of being generalized together. Irritation of the corpus striatum causes general muscular contraction on the opposite side of the body” (56). Rather than describing the striatum as a purely motor structure, however, Ferrier distinguished that “the corpus striatum is the centre in which movements primarily dependent on volition proper tend to become organized.” This view has withstood the test of time, and modern literature routinely links the striatum and the other basal ganglia structures to the organization and generation of voluntary movement (21, 65, 73).
However, from the earliest investigations, the relationship between the striatum and movement was complex, since different experiments implicated the striatum in both the generation and inhibition of movement. In 1841, Magendie reported that a bilateral lesion of the striatum caused a rabbit to race forward as if possessed by an “irresistible impulse” (116). In 1873, Nothnagel reported that smaller lesions, in a specific place near the medial striatal border, caused a similar phenomenon in dogs (137). In the 1940s, Mettler carefully characterized this phenomenon, terming it “cursive hyperkinesia,” and describing bilateral striatal lesions that would cause animals to run forward without regard to obstacles or walls in their paths (124–126). Similar phenomena were seen in subsequent animal experiments (41, 85) and in parkinsonian patients who have been described at times as unable to stop running, sending themselves “headlong” into walls and furniture (117). James Parkinson described such a patient in “An Essay on the Shaking Palsy” who exhibited “an inability for motion, except in a running pace,” and required the support of an attendant who ran in front of him to prevent him from falling (144, 145). Other findings from the same period differed from these, however, reporting little effect or decreases in movement following striatal lesions (42, 68, 90, 127, 196). The likely explanation for these discrepancies rests in differences in size or location of the lesions, or differential damage to neighboring structures such as the overlying cortex, which were not well quantified in these studies.
The technique of microstimulation provided a less destructive method for interrogating striatal function. However, like prior lesion experiments, microstimulation also implicated the striatum in both the generation and inhibition of movement, with the specific result again likely due to differences in stimulation parameters or location. The most widely cited effects of striatal microstimulation across multiple species of animal are contralateral head turning or circling (10, 24, 32, 35, 38, 59, 77, 96, 106, 123, 131, 140, 201) and contraversive limb movements (7, 24, 28, 32, 59, 106). However, a number of investigators have also reported suppression, arrest, or freezing of movement during striatal stimulation, as well as a general slowing of motor behavior following striatal stimulation (38, 95, 97, 105, 131). Finally, electrophysiological recordings have shown that individual striatal neurons respond during multiple phases of movement, with some neurons responding to the initiation of movements, others responding during holding or waiting periods, and still others responding near the termination of movements (5, 34, 37, 60, 74–76, 92, 94, 108, 109, 129, 157–160, 168, 169, 172). Overall, the varied nature of these findings supports the view that the striatum is involved in both the generation and inhibition of movement.
An anatomical scheme for understanding these opposing roles was defined in the late 1980s (FIGURE 1). Briefly, this scheme recognized that the majority of dorsal striatal neurons are medium spiny neurons (MSNs), of which there are two distinct classes, termed the “direct” and the “indirect” pathway projection neurons (6, 36, 63, 65, 107, 139). These populations exhibit distinct neurochemical expression patterns and anatomical projection targets. Direct pathway MSNs project to the internal globus pallidus and substantia nigra pars reticulata (SNr), whereas indirect pathway MSNs express project indirectly to the SNr by way of the external globus pallidus (GPe) and subthalamic nucleus (STN). Based on this anatomy, these authors hypothesized that activation of direct pathway striatal neurons facilitated motor output, whereas activation of indirect pathway neurons inhibited motor output. Recent explicit tests of this model have supported it, demonstrating that direct pathway promotes movement, whereas the indirect pathway inhibits movement (46, 100, 164).
A Role for Striatal Dopamine
Dopamine entered the discussion of striatal function in the late 1950s. Specifically, the caudate was reported to contain the highest levels of dopamine in the brain and also to contain relatively low levels of the other monoamines norepinephrine and serotonin, implicating dopamine as the most important monoamine in this structure (16). Following this finding, Ehringer and Hornykiewicz discovered that the main pathology underlying Parkinson's disease was the loss of dopamine from the striatum (48, 49), firmly establishing that dopamine was critical for striatal function and, in particular, striatal-dependent movement.
Parkinsonian motor deficits can be largely explained by the predicted effects of dopamine on the two striatal output pathways. The most widely cited expression difference between the two pathways are the dopamine Drd1 receptor, which is selectively expressed by direct pathway neurons, and the dopamine Drd2 receptor, which is selectively expressed by indirect pathway neurons. Due to this differential expression, dopamine affects the neurons of each pathway differently (63). The dopamine Drd1 receptor is coupled to Gαs, which activates adenyl cyclase and increases intracellular cyclic AMP (cAMP) (62, 89, 179). This increase in cAMP results in multiple intracellular effects that increase the excitability of direct pathway neurons. In contrast, the dopamine Drd2 receptor is coupled to Gαi, which inhibits adenyl cyclase and decreases intracellular cAMP (62, 89, 179). This decrease in cAMP is believed to decrease the excitability of indirect pathway neurons. In addition, regulation of cAMP/PKA signaling by dopamine receptors may directly or indirectly regulate the induction of striatal synaptic plasticity (104, 180). When striatal dopamine levels decrease in Parkinson's disease, activity in the direct pathway is believed to decrease and activity in the indirect pathway is believed to increase (4, 66, 138, 139, 176). Therapies aimed at rebalancing the activity in these pathways form the basis for most therapeutic interventions (13, 45, 57, 83, 86, 100, 110, 119, 162). Although many pharmaceutical therapies for Parkinson's disease target dopamine receptors, it should be noted that these dopamine receptors are far from the only expression difference between these pathways. Gene array studies have identified hundreds of genes that are enriched in neurons of one pathway or the other, and many of these likely contribute to controlling the activity of each pathway (70, 113).
The Involvement of the Striatum in Reinforcement and Reward
Although many studies implicated striatal dopamine in movement, a parallel view emerged for striatal dopamine in reinforcement and reward. The term reinforcement describes processes that maintain or increase behavior, whereas the term punishment describes processes that decrease behavior (11, 175). Reinforcement can be caused by the addition of a positive stimulus (termed positive reinforcement, +R) or the removal of a negative stimulus (termed negative reinforcement, −R). Similarly, punishment can be caused by the addition of a negative stimulus (termed positive punishment, +P) or the removal of a positive stimulus (termed negative punishment, −P) (FIGURE 2). It is important to note that these terms are behavioral and make no conclusions about an organism's hedonic state or whether it “likes” or “dislikes” the stimuli. Instead, the hedonic state of an organism can be described by the terms reward and aversion. Rewarding stimuli are those to which an animal assigns a positive hedonic value, whereas aversive stimuli are those to which an animal assigns a negative hedonic value. In general, behaviors that increase rewarding stimuli or decrease aversive stimuli are reinforced (+R or −R, respectively), whereas those that increase aversive stimuli or decrease rewarding stimuli are punished (+P or −P, respectively). For example, after tasting a novel food, a person may increase or decrease the frequency of future behavior directed at obtaining that food, depending on how much they liked it.
Early lesion studies revealed a role for the striatum in both reinforcement and reward. Striatal lesions in cats produced a “compulsory approaching syndrome,” in which the cats would compulsively follow the experimenters, other cats, or even stationary objects, with a strong tendency to make contact with these targets (191, 192). Other effects included a “marked docility and a friendly disposition, persistent purring, repetitive treading or kneading of the forelimbs,” and the normally female-specific lordosis behavior in male cats (143, 190). Rather than the undirected “cursive hyperkinesias” or the limb or postural contractions described in the movement literature, these behavioral changes indicated that there was a goal-directed and hedonic nature to the striatum's contribution to movement. Further studies of these cats revealed perseverative deficits in reversal learning tasks (142), a finding that has been replicated with striatal inactivation or lesions in rodents (27, 150, 151) and primates (29). Other lesion or inactivation studies have implicated the striatum in multiple features of pavlovian and instrumental processing, at times clearly independent of any motor impairments caused by these manipulations (19, 20, 25, 33, 43, 47, 51, 52, 54).
The application of microstimulation to the striatum provided additional evidence for its involvement in reinforcement and reward. In a seminal 1954 study, Olds and Milner demonstrated that rats would self-administer electrical stimulation to specific sites in their brains, indicating that intracranial self-stimulation was reinforcing (141). The stimulation site that supported the highest level of responding in this study was the septum, followed by regions of frontal cortex, thalamus, and midbrain. One animal in this study was stimulated in the caudate, but this did not support responding, a finding replicated with several caudate stimulation sites in rabbits (22). Reminiscent of the opposing results from motor studies of the striatum, a large study in cats showed that intracranial striatal stimulation could reinforce or punish lever-pressing behavior, depending on the specific stimulation site (195). Later studies have conclusively shown that certain locations in the striatum, in particular the head of the caudate and the ventral regions, support responding for intracranial stimulation (147–149, 183). Direct evidence that basal ganglia stimulation elicits hedonic responses comes from reports of humans who underwent intracranial stimulation for treating psychiatric disease. Such patients have reported the hedonic effects of stimulation of multiple sites throughout their brains. In general, the rewarding and aversive sites in humans overlap strongly with the reinforcing and punishing sites in animals (69, 111, 195). For example, self-stimulation of the septum is highly reinforcing in humans (patients in one study stimulated this site about 400 times/h), and one patient explained that the reason he kept stimulating this site was that it “felt great,” as if he were building up to a sexual orgasm (69). Caudate stimulation was also highly reinforcing in this study (averaging a little under 400 stimulations/h), and this same patient described caudate stimulation as feeling “good,” although not as pleasurable as septal stimulation. Interestingly, there is evidence of a somatotopic mapping of hedonic responses in human basal ganglia circuitry, such that stimulation of various parts of the globus pallidus resulted in intense feelings of pleasure in specific regions of the body, contralateral to the stimulation side (111, 195). This is similar to the well described somatotopic mapping of motor responses in this circuitry (12, 61, 128, 133, 193) and supports the hypothesis that activity in basal ganglia circuits mediate both movement and reward.
The reinforcing and rewarding effects of intracranial self-stimulation appear to be largely attributable to striatal dopamine release. For example, the stimulation sites that support the strongest responding are the lateral hypothalamus, midbrain dopaminergic nuclei, and medial forebrain bundle, all of which produce robust increases in striatal dopamine release (82, 91, 154, 199). Striatal dopamine antagonism also attenuates the rewarding effect of striatal stimulation, indicating that striatal dopamine release may be necessary for the rewarding properties of these stimulation sites (64, 130, 153). Recent optogenetic experiments have directly elicited dopamine release in the striatum and revealed that certain stimulation paradigms support conditioning, further demonstrating the importance of striatal dopamine to reinforcement and reward (2, 14, 186, 197).
Electrophysiological studies of both striatal neurons and midbrain dopamine neurons further supported the role of the striatum and striatal dopamine in reward and reinforcement, perhaps more strongly than its role in movement. For example, midbrain dopamine neurons do not respond to movement itself but rather to the discrepancies between expected and actual outcomes, termed “prediction error” (58, 166, 167). Consistent with these findings, studies that have examined the issue generally find that striatal neurons do not respond to movements per se but rather to features of the movements that support reinforcement, such as the anticipation of, or expected reward value, associated with those movements (76, 78, 79, 87, 88, 92–94, 170, 171). Finally, it should be noted that a number of studies have revealed a role for the striatum in aversive processing (54, 55, 194). Dopamine itself has also been implicated in aversive processing (146, 163), and specific populations of midbrain dopamine neurons are specifically excited by aversive cues and events (21, 30, 67, 120).
Selective Contributions of the Direct and Indirect Pathways to Reinforcement and Reward
Despite the utility of the direct and indirect pathway model for understanding striatal contributions to movement, these pathways have not been historically used to describe striatal contributions to reinforcement and reward. In part, this reflects a historical lack of clarity on whether these pathways exist in the ventral striatal regions that are often investigated in studies of reinforcement and reward. Even in the dorsal striatum, for a long time it was unclear whether dopamine Drd1 and Drd2 receptors were expressed on distinct populations of medium spiny neurons (63) or whether a significant population expressed both receptor types (3, 181). Only in recent years, bacterial articicial chromosome (BAC) transgenic mice provided conclusive evidence for the segregation of these receptor types in nearly all (∼95%) medium spiny neurons of the dorsal striatum (17, 18, 118). This conclusion applies to the accumbens core as well, where cells that express Drd1 and Drd2 receptors are also almost completely (∼95%) segregated (118). However, these pathways appear to be less segregated in the accumbens shell, where up to 17% of the shell neurons co-express both Drd1 and Drd2 receptors, potentially as heteromeric Drd1/Drd2 receptor complexes (17, 135, 152). The projection patterns from the core and shell mimic this story, with the accumbens core being relatively indistinguishable from the dorsal striatum and the shell being somewhat unique. The direct pathway core neurons project to the substantia nigra pars reticulata, whereas the indirect pathway core neurons project to the external globus pallidus (39, 40, 121). A population of dynorphin and substance P (and presumably Drd1) positive shell neurons project to the ventral tegmental area, constituting a ventral analog of the dorsal striatum direct pathway (115, 200). However, the shell projection to the ventral pallidum (the ventral analog of the dorsal striatum indirect pathway) contains equal numbers of enkephalin positive neurons and direct pathway-like substance P and dynorphin positive neurons (115, 200). It appears that Drd1 and Drd2 co-expressing shell neurons project solely to the ventral pallidum, based on the lack of enkephalin positive projections to the ventral tegmental area (80, 115). For a thorough review of the anatomical and dopamine receptor expression differences between the dorsal and ventral striatal subdivisions, the reader is referred to Ref. 80.
Historically, electrophysiological studies of the striatum have been hampered by the inability to differentiate direct and indirect pathway neurons in the recordings. As a result, all recorded neurons in these studies are routinely lumped together and described as a homogeneous group of medium spiny neurons (5, 34, 37, 60, 74–76, 84, 92, 94, 102, 108, 109, 129, 136, 156–161, 168, 169, 172, 182). Many rodent studies have characterized the responses of accumbal neurons to events such as cues and behaviors linked to sucrose delivery or to sucrose consumption itself. The findings of these studies are mixed, with most studies reporting more frequent inhibitory responses to sucrose consumption (84, 136, 156, 182) and other studies reporting more frequent excitatory responses (161) or relatively equal proportions of neurons excited or inhibited (102). One thing that these studies do agree on, however, is that distinct populations of accumbal neurons are either excited or inhibited by sucrose. In an elegant study, Roitman and colleagues showed that distinct populations of accumbal neurons innately respond to rewarding and aversive tastes (sucrose and quinine) (156). When considered in the context of the direct and indirect pathways, two questions jump out from these studies: Are neurons that are activated by rewarding stimuli direct pathway neurons? Are neurons that are activated by aversive stimuli indirect pathway neurons?
To answer these questions, we would need to know the identity of the recorded neurons in the above studies, and techniques for accomplishing this have only recently been developed. Direct or indirect pathway neurons are not distinguishable by any known electrophysiological characteristics. However, it is possible to target channelrhodopsin-2 (ChR2) to a specific pathway and use the presence of light-evoked spiking as a “tag” for in vivo identification of neurons of one pathway or the other (101, 103). Such techniques should be able to provide conclusive answers to the above two questions. It is important to note that the processes of reward, reinforcement, and movement are not mutually exclusive from one another. Since they exist in separate planes of observation, nothing precludes the same structures from mediating all three of these processes simultaneously: movement can be mediated by the anatomical connections between the basal ganglia and motor circuitry (FIGURE 1); reward can be mediated by the relationship between brain states and hedonic states, a topic that is not well understood; and reinforcement can be mediated by plasticity in these cell types or in their afferent inputs (FIGURE 3).
Although conclusive evidence that the same cells mediate both movement and reward will require future experiments, recent behavioral studies have been supportive of this hypothesis. Lobo and colleagues expressed ChR2 selectively in direct or indirect pathway accumbal neurons using a cre-dependent viral strategy. Optogenetic activation of these cell populations alone did not produce conditioned place preference or aversion in this study. However, optogenetic activation of direct pathway neurons heightened the strength of cocaine-induced conditioned place preference, whereas activation of the indirect pathway impaired this place preference, possibly because the optogenetic stimulation overlaid additional positive or negative hedonic responses onto the cocaine exposure. (112). Ferguson and colleagues observed a consistent result with a different methodology. They expressed a synthetic inhibitory Gαi-coupled DREADD (designer receptor exclusively activated by a designer drug) receptor in direct or indirect pathway neurons of the dorsal or ventral striatum. DREADD receptors are GPCRs that have been modified such that they no longer respond to their endogenous ligand but instead can be activated by the administration of synthetic ligands such as clozapine-N-oxide (CNO) (31, 44). Activation of this receptor in direct pathway neurons heightened, whereas activation in indirect pathway neurons impaired, amphetamine sensitization (53). Hikida and colleagues selectively impaired synaptic transmission in direct or indirect pathway neurons using cell-type-specific expression of tetanus toxin. When they impaired synaptic transmission in direct pathway neurons, animals exhibited reduced cocaine locomotor sensitization and impaired conditioned place preference for a food reward. When they impaired synaptic transmission in indirect pathway neurons, animals exhibited learning deficits in an inhibitory avoidance task, which is a measure of aversive learning (72). Finally, in a recent study, we expressed ChR2 in the direct or indirect pathway neurons of the dorsomedial striatum and found that mice rapidly learned to contact a trigger that resulted in stimulation of the direct pathway and avoid a trigger that resulted in stimulation of the indirect pathway. Based on these results, we concluded that direct pathway activation was reinforcing, whereas indirect pathway activation was punishing. The rapid speed at which mice expressed this behavior (the behavioral changes were highly significant within the first 10 min of being placed in the stimulating chamber) indicates that activation of direct or indirect pathways may evoke primary hedonic responses, such that direct pathway activation is rewarding and indirect pathway activation is aversive (103).
The above data support the hypothesis that activity of direct and indirect pathway striatal neurons exerts opposing control over not just movement but hedonic reward states as well. If true, a corollary of this hypothesis would suggest that plasticity of the inputs onto these cell types would alter future motor or hedonic responses, as in reinforcement or punishment. Specifically, positive reinforcement (+R) may be associated with plasticity that enhances synaptic efficacy onto direct pathway neurons [i.e., long-term potentiation (LTP)], whereas positive punishment (+P) may be associated with plasticity that enhances synaptic efficacy onto indirect pathway neurons (FIGURE 3). Conversely, negative reinforcement (−R) may be associated with plasticity that depresses synaptic efficacy onto indirect pathway neurons [i.e., long-term depression (LTD)], whereas negative punishment (−P) may be associated with plasticity that depresses synaptic efficacy onto direct pathway neurons.
There are many known differences in the rules that govern plasticity in the inputs onto direct and indirect pathway neurons (104). These differences in plasticity may result in differences in the expression of reinforcement and punishment. Interestingly, differences in the persistence of reinforcement and punishment (which likely relate to differences in plasticity) have long been observed at a behavioral level. Punishment has repeatedly been shown to be more transient than reinforcement. In describing early work, Skinner concluded that “The effect of punishment was a temporary suppression of the behavior, not a reduction in the total number of responses. Even under severe and prolonged punishment, the rate of responding will rise when punishment has been discontinued …” (174). Although Skinner's original conclusions may be attributed to the use of relatively weak punishers in those studies, the transient nature of punishment and its utility in modifying human and animal behavior is still an active field of research. Perhaps the best controlled studies of this issue have been performed with insects. Studies in Drosophila showed that reinforcement caused long-lasting changes in behavior (>24 h), whereas the effects of punishment rapidly declined and were no longer observable after ∼3 h following training (184). Another set of studies carefully investigated this issue with crickets and found that punishment memory was relatively short-lived compared with reward memory, using both olfactory and visual stimuli for conditioning (132, 187, 188). In our recent study, we found that positive reinforcement caused by direct pathway stimulation persisted for long durations in mice (at least 30 min), whereas positive punishment caused by indirect pathway stimulation was transient, appearing to last for only 30–45 s (103). Even in humans, punishment appears to be more transient than reinforcement (1). It has been suggested that this is adaptive, e.g., fruit that is bitter in one season may become ripe and sweet during another season, and it would therefore be maladaptive to persistently avoid that fruit based on an aversive experience(132). Based on these differences in persistence, it is likely that different plasticity mechanisms contribute to reinforcement and punishment, and it is possible differences in plasticity onto direct and indirect pathway neurons contribute to these differences.
In addition to increasing our understanding of these pathways, modulating their activity may provide therapeutic opportunities to normalize striatal circuit function in neurological and psychiatric disease. For instance, we have used optogenetic activation of direct pathway neurons to rescue parkinsonian motor deficits (100). Based on our hypothesized role of these pathways in reinforcement and reward, selective manipulations of these pathways may be therapeutic for diseases such as addiction and depression as well.
Direct and indirect pathway neurons have been hypothesized to exhibit opposing control over motor output for ∼30 years. This hypothesis has been extremely useful for understanding and directing therapeutic interventions in movement disorders. A similar hypothesis purporting opposing roles of these pathways in reinforcement and reward has only recently gained the attention of researchers. Although further work is needed to fully understand this hypothesis, this framework may provide insights into understanding the mechanisms underlying reinforcement and reward, as well as therapeutic interventions for disorders of these processes.
No conflicts of interest, financial or otherwise, are declared by the author(s).
Author contributions: A.V.K. conception and design of research; A.V.K. prepared figures; A.V.K. drafted the manuscript; A.V.K. and A.C.K. edited and revised the manuscript; A.V.K. and A.C.K. approved the final version of the manuscript.
- ©2012 Int. Union Physiol. Sci./Am. Physiol. Soc.