The classical pharmacological techniques of using selective agonists, antagonists and inhibitors provided the initial basis for categorizing the receptors, transporters and degrading enzymes involved in serotonergic pathways. But the complexity of the system has overwhelmed the armamentarium of selective drugs, leaving many proteins without specific probes of their function. Recent advances in the field have exploited deliberate or naturally occurring disruptions of the genes that code for proteins important to serotonergic transmission. Four recent contributions [3], [4], [5] and [6] describe the effects of disrupting the genes encoding the 5-HT1B and 5-HT2C receptors in mice, and the monoamine oxidase A (MAOA) genes in mouse and human

5-HT1B knockout — locomotion and aggression
The 5-HT1B receptor was the first protein involved in serotonergic signaling to have its gene disrupted [3]. The 5-HT1B receptor is expressed in the basal ganglia, central gray, hippocampus, amygdala and raphe nuclei; it is targeted primarily to presynaptic terminals, where it can inhibit neurotransmitter release. Highly specific agonists and antagonists are not available for 5-HT1B receptors, but pharmacological experiments suggest that 5-HT1B receptor activation may increase anxiety and locomotion and decrease food intake, sexual activity and aggression. Targeted disruption was achieved by deleting a fragment of the coding sequence, and functional inactivation of the 5-HT1B receptor was confirmed by autoradiography with 125I-cyanopindolol, which binds 5-HT1B receptors, along with appropriate blockers of non-5-HT1B binding sites. Heterozygote animals expressed the same levels of receptor as wild-types, suggesting that a feedback mechanism may regulate the transcriptional activity of the gene.

Two of the behaviors postulated to be modulated by 5-HT1B receptors were analyzed: locomotion and aggression. Untreated wild-type and homozygous mutant (5-HT1B−/−) mice were found to display similar levels of locomotor activity in an open field. However, when treated with the 5-HT1 agonist RU24969, the 5-HT1B−/− mice displayed no change in activity, whereas wild-type mice doubled their activity. This result confirmed previous pharmacological studies indicating that RU24969 operates through 5-HT1B receptors.

The aggressiveness of 5-HT1B−/− male mice was assessed by isolating them for four weeks and then exposing them to a non-isolated male wild-type intruder mouse. The latency and number of attacks displayed by the knockout mice during a three-minute period were used as indices of aggression. The 5-HT1B−/− mice, when compared with wild-type mice, exhibited faster, more intense and more frequent attacks. For example, 46% of the 5-HT1B−/− mice attacked the intruder within 10 seconds of introduction, whereas no heterozygous or wild-type mice attacked during this interval. Tail rattling, an aggressive display, was also more frequent in 5-HT1B−/− mice.

These experiments suggest an involvement of 5-HT1B receptors in the modulation of aggressive behavior. Eltoprazine and fluprazine, drugs classified as serenics because of their anti-aggressive properties, are believed to operate as agonists of 5-HT1 receptors. The lack of subtype-specific antagonists has impeded specific identification of the molecular target of such serenics; the availability of 5-HT1B−/− mice should allow more detailed evaluation of the pathways leading to aggression.

5-HT2C knockout — seizures and overeating
The 5-HT2C gene knockout was generated by insertion of a stop codon that eliminated the carboxyl terminus of the protein [4]. The 5-HT2C gene is X-linked; brains of mutant males were devoid of 5-HT2C immunoreactivity, and mRNA from such males failed to produce functional receptor when injected into Xenopus oocytes.

The patterns of expression of 5-HT2C receptors in the hippocampus and spinal cord suggest that they might modulate memory and nociception. However, hippocampal LTP and nociceptive thresholds were the same in wild-type and 5-HT2C−/− mice. Video monitoring revealed that 5-HT2C−/− mice displayed spontaneous epileptic seizures that were sometimes fatal; survival plots indicated that almost half of the mice had died by 25 weeks of age. Metrazol, a γ-amino butyric acid (GABA) receptor antagonist, was used to quantify seizure susceptibility. Relative to wild-type mice, the 5-HT2C−/− mice displayed a reduced seizure threshold and a more rapid progression through the tonic-clonic phase of the seizure. The unexpected susceptibility of 5-HT2C−/− mice to epileptic seizures raises the possibility that this receptor participates in the modulation of neuronal network excitability. The ability to affect the seizure characteristics of normal mice with 5-HT2C ligands suggests that the phenotype of the 5-HT2C−/− mice is not a result of chronic compensation by other receptors for the loss of 5-HT2C receptors.

5-HT2C−/− mice were also noted to be significantly heavier than matched wild-type controls. Close analysis of their feeding behavior indicated that the weight increase was due entirely to increased intake, and not to a metabolic derangement that increases the efficiency of caloric storage. The serotonergic agonist 1-(3-chlorophenyl) piperazine (mCPP) drastically reduced food intake by wild-type mice, but had no effect on food intake by 5-HT2C−/− mice, demonstrating that appetite suppressants may operate via 5-HT2C receptors.

The 5-HT1 receptor family contains receptors that are negatively coupled to adenylyl cyclase and includes the 5-HT1A, 5-HT1B, 5-HT1D, 5-ht1E and 5-ht1F receptors. The 5-HT1A receptor is coupled via G proteins to two distinct effector systems: (i) inhibition of adenylyl cyclase activity and (ii) the opening of K+ channels, which results in neuronal hyperpolarization. In terminal field areas of serotonergic innervation, such as the hippocampus, 5-HT1A receptors are coupled to both effector systems (Table 13-2). However, in the dorsal raphe nucleus, 5-HT1A receptors are coupled only to the opening of potassium channels.The 5-HT1B and 5-HT1D receptor subtypes are also linked to inhibition of adenylyl cyclase activity (Table 13-2). Binding sites that have been defined pharmacologically as 5-HT1B receptors have been characterized in the rat, mouse and hamster, whereas the 5-HT1D receptor has been characterized using pharmacological criteria in species such as guinea pig, pig, cow and human. In the substantia nigra, where a high density of 5-HT1B or 5-HT1D receptors has been demonstrated by radioligand-binding studies, these serotonin receptors are linked to the inhibition of adenylyl cyclase through a G protein.

An issue raised by the use of molecular biological techniques for the study of neurotransmitter receptors is whether a receptor is a subtype or a species homolog, that is, an equivalent receptor in different species. For example, the 5-HT1B and 5-HT1D receptors originally were considered to be species variants of the same receptor because the pharmacological profiles of these two receptors are similar, although not identical; the distribution of these two receptors in brain is very similar; and both receptors are coupled to the inhibition of adenylyl cyclase. Although biochemical, pharmacological and functional data suggest that the 5-HT1B receptor found in rats and mice and the 5-HT1D receptor found in other species, including humans, are functionally equivalent species homologs, the story has been complicated somewhat by the discovery of two genes encoding the human 5-HT1D receptor, 5-HT1Dα and 5-HT1Dβ [16].

Radioligand-binding studies currently do not allow the differentiation of 5-HT1Dα and 5-HT1Dβ receptors, and the binding profiles of these receptor subtypes match the previously described 5-HT1D-binding site. Furthermore, a rat homolog of the human 5-HT1Dα receptor has been isolated and shown to encode a receptor with a 5-HT1D-binding site profile, suggesting that the 5-HT1B and 5-HT1D receptors may not be species homologs but distinct 5-HT receptor subtypes. There is still some debate as to whether a common appellation should be used to refer to the protein products of two distinct genes, the 5-HT1Dα and the 5-HT1Dβ receptor, and whether the human 5-HT1Dβ receptor should be called the human 5-HT1B receptor, even though it has a distinct pharmacological profile from that of the rat 5-HT1B receptor. Because there are no compounds currently available to differentiate between the 5-HT1Dα and the 5-HT1Dβ receptors, we will refer to them in this chapter as 5-HT1D. Furthermore, because of distinct pharmacological profiles of the 5-HT1B receptor found in rat and the 5-HT1D receptor found in other species, we will not refer to the 5-HT1Dβ as the human 5-HT1B receptor.

The 5-ht1E receptor originally was identified in homogenates of human frontal cortex by radioligand-binding studies with [3H]5-HT in the presence of 5-carboxamidotryptamine (5-CT) to block 5-HT1A and 5-HT1D receptor sites. Because of the lack of specific radioligands for the 5-ht1E receptor, the overall distribution in brain is unknown. With the cloning of the various subtypes of receptors for serotonin, knowledge of receptor sequences can be used to generate radioactive probes for mRNAs encoding individual serotonin receptor subtypes. Using in situ hybridization histochemistry, the localization of these mRNAs and, thus, the distribution of cells expressing the mRNAs for serotonin receptors can be established in brain. 5-ht1E receptor mRNA has been found in the caudate putamen, parietal cortex and olfactory tubercle [17]. The function of the 5-ht1E receptor in intact tissue is not known due to the lack of selective agonists or antagonists. In transfected cells, the 5ht1E receptor is coupled to the inhibition of adenylyl cyclase activity. The 5-ht1E receptor displays a higher degree of homology with the 5-HT1D receptor (64%) than any other 5-HT1 receptors [16].

The 5-HT1F receptor was cloned and sequenced in 1993 and shares the greatest sequence homology with the 5-ht1E receptor (61%). 5-ht1F receptor mRNA is found in cortex, hippocampus, dentate gyrus, nucleus of the solitary tract, spinal cord, trigeminal ganglion neurons, uterus and mesentery. In transfected cells, the 5-ht1F receptor is coupled to the inhibition of adenylyl cyclase [16]. Because selective agonists or antagonists for the 5-ht1F receptor have not been available until very recently, little is known about the distribution or function of the 5-ht1F receptor in brain. The selective agonist radioligand [3H]LY334370 has been used to demonstrate the presence of 5-ht1F receptor sites in cortex, striatum, hippocampus and olfactory bulb [18]. Activation of 5-ht1F receptors in vivo inhibits neurogenic dural inflammation and dural protein extravasation.

The 5-HT2 receptor family stimulates phosphoinositide-specific phospholipase C (PI-PLC) and includes the 5-HT2A, 5-HT2B and 5-HT2C (formerly the 5-HT1C) receptors. 5-HT2A receptor-mediated stimulation of phosphoinositide hydrolysis has been well characterized in cerebral cortex. 5-HT2C receptor-mediated stimulation of inositol lipid hydrolysis has been studied in the choroid plexus (Table 13-2). Stimulation of phosphoinositide turnover by 5-HT in these tissues is not dependent on the activity of lipoxygenase or cyclooxygenase pathways, nor is it blocked by agents that inhibit neuronal firing, suggesting that coupling of the 5-HT2A or 5-HT2C receptor to the enzyme PI-PLC mediates the enhanced response (see Chap. 21). Activation of 5-HT2A receptors also mediates neuronal depolarization, a result of the closing of potassium channels. The 5-HT2A receptor was first cloned in the rat by homology with the rat 5-HT2C receptor. The rat 5-HT2A receptor is 49% homologous to the rat 5-HT2C receptor.

Cloning of the 5-HT2A receptor has been used to gain insight into a controversy over the nature of agonist binding to the 5-HT2A receptor. The hallucinogenic amphetamine derivative [3H]2,5-Dimethoxy-4-bromoamphetamine (DOB), an agonist, binds to a small number of sites with properties very similar to those of the receptor labeled with the antagonist [3H]ketanserin. Agonists, though, have higher affinities for the receptor labeled with [3H]DOB than for that labeled with [3H]ketanserin. Some investigators have interpreted these and other data as evidence for the existence of a new subtype of 5-HT2A receptor, whereas others have interpreted these data as indicative of agonist high-affinity and agonist low-affinity preferring states of the 5-HT2A receptor. In experiments in which the cDNA encoding the 5-HT2A receptor was transfected into clonal cells, binding sites for both the 5-HT2A receptor antagonist [3H]ketanserin and the 5-HT2 receptor agonist [3H]DOB were found. Furthermore, agonists had higher affinities for [3H]DOB binding than for [3H]ketanserin binding. Thus, a single gene produces a protein with both binding sites, substantiating the view that agonist and antagonist binding are to different states, rather than to two different subtypes, of the 5-HT2A receptor.

Although the 5-HT2B receptor is the most recently cloned of the 5-HT2 receptor class, it was among the first of the serotonin receptors to be characterized using pharmacological criteria. The first report of the sensitivity of rat stomach fundus to serotonin was published by Vane in 1959. This receptor, whose activation results in the contraction of fundus smooth muscle, originally was placed in the 5-HT1 receptor class by Bradley and associates [15] because of its sensitivity to serotonin and because responses mediated by it were not blocked by 5-HT2 or 5-HT3 receptor antagonists. It has been reclassified as a 5-HT2 receptor because of its similar pharmacological profile to the 5-HT2C receptor (Table 13-2). The recombinant receptor expressed in clonal cells is coupled to the stimulation of inositol lipid hydrolysis. However, in rat stomach fundus, the 5-HT2B appears not to be coupled to phosphoinositide hydrolysis. 5-HT2B receptor-mediated contraction of rat stomach fundus is dependent on the influx of calcium through voltage-sensitive channels, intracellular calcium release and activation of PKC [19]. The effector system to which this receptor is coupled in the CNS remains to be established. Using quantitative polymerase chain reaction (PCR), 5-HT2B mRNA has been detected in the rat stomach fundus, intestine, kidney, heart, lung and dura mater but not in rat brain. In humans, 5-HT2B receptor mRNA has been found peripherally and in cerebellum, cerebral cortex, amygdala, substantia nigra, caudate, thalamus, hypothalamus and retina

The 5-HT3 receptor is homomeric and belongs to the ligand-gated ion channel superfamily. As mentioned above, the 5-HT3 receptor is a serotonin-gated cation channel that causes the rapid depolarization of neurons (Table 13-2). The depolarization mediated by 5-HT3 receptors is caused by a transient inward current, specifically the opening of a channel for cations. A single subunit of the 5-HT3 receptor, the 5-HT3-A receptor subunit, has been cloned. An alternatively spiced variant, the 5-HT3-As receptor subunit, has been identified in mouse, rat and human. The cloned receptor subunit exhibits sequence similarity to the α subunit of the nicotinic acetylcholine receptor and to the β1 subunit of the GABAA receptor. It is not known whether the native 5-HT3 receptor is composed of this single subunit or several different subunits. Although single subunits of members of the ligand-gated ion channel receptor family can form functional homomeric receptors, they generally lack some of the properties of the native, multisubunit receptor. The cloned subunit of the 5-HT3 receptor has been studied in Xenopus oocytes injected with mRNA encoding this receptor. Although the expressed 5-HT3-A and 5-HT3-As receptors are functional, they do not display all of the characteristics of native 5-HT3 receptors. The 5-HT3 receptor, like other members of the ligand-gated ion channel superfamily, appears to possess additional pharmacologically distinct recognition sites for alcohols and anesthetic agents, by which the function of this receptor can be allosterically modulated

5-HT4 , 5-ht6 and 5-HT7 receptors are included in a family of serotonin receptors coupled to the stimulation of adenylyl cyclase. The 5-HT4 receptor originally was described in cultured murine collicular neurons as a serotonin receptor coupled to the stimulation of adenylyl cyclase activity, possessing pharmacological characteristics distinct from those of the 5-HT1, 5-HT2 or 5-HT3 receptors. The 5-HT4 receptor gene has been cloned from rat brain RNA by reverse transcriptase (RT)-PCR [21]. Two different cDNA clones, the long isoform, 5.5-kb 5-HT41, and the short isoform, 4.5-kb 5-HT4s, have been isolated and are most likely the result of alternative splicing of 5-HT4 receptor mRNA.

The 5-ht6 receptor is approximately 30% homologous to other serotonin receptors. When expressed in transfected cells, it shows high affinity for [125I]LSD and [3H]5-HT. The pharmacology of this recombinant receptor is unique. Interestingly, this receptor has high affinity for various antipsychotic and antidepressant drugs, such as clozapine, amitriptyline, clomipramine, mianserin and ritanserin. The 5-ht6 receptor stimulates adenylyl cyclase when expressed in some, but not all, cell systems. The function of the 5-ht6 receptor in intact tissue has not been characterized due to the lack of selective agonists or antagonists. Expression of 5-ht6 receptor mRNA has been detected in the striatum, nucleus accumbens, olfactory tubercle, hippocampus and cerebral cortex [16].

Two rat 5-HT7 receptor clones, which differ only in the C terminus and presumably result from alternative mRNA splicing, have been identified. The 5-HT7 receptor shows the highest amino acid sequence homology with the Drosophila 5-HT1A receptor, 42%, and approximately 35% homology with all other serotonin receptors. To date, no selective agonists or antagonists have been described for the 5-HT7 receptor. In transfected cells, the 5-HT7 receptor stimulates adenylyl cyclase. 5-HT7 receptors have been identified in human vascular smooth muscle cells and frontal cortical astrocytes in primary culture, where they are coupled to the stimulation of adenylyl cyclase.

The 5-ht5A and 5-HT5B receptors may constitute a new family of serotonin receptors since neither is coupled to adenylyl cyclase or PI-PLC; their effector systems are currently unknown. Both the 5-ht5A and 5-HT5B receptors were cloned by using degenerate oligonucleotides derived from TMDs III and VI of G protein-coupled serotonin receptors. Both genomic clones possess one intron in the middle of the third cytoplasmic loop. The receptor proteins are 77% identical to each other, whereas the homology to other serotonin receptors is low.

5-ht5A receptor mRNA transcripts have been detected by in situ hybridization in the cerebral cortex, hippocampus, granule cells of the cerebellum, medial habenula, amygdala, septum, several thalamic nuclei and olfactory bulb of the rat and mouse. 5-HT5B mRNA has been detected by in situ hybridization in the hippocampus, habenula and the dorsal raphe nucleus of rat and human [16].

Immunohistochemical studies with antibodies to the 5-ht5A receptor have shown this receptor to be expressed predominantly by astrocytes, although some neurons in cortex were labeled as well. In transfected cells expressing the 5-ht5A receptor, 5-HT does not stimulate the formation of cAMP as it does in wild-type cells. Furthermore, 5-HT inhibits forskolin-stimulated cAMP formation, an effect not seen in wild-type cells. Thus, the 5-ht5A receptor appears to be coupled to the inhibition of adenylyl cyclase activity [22]. At the present time, the functional correlate and transductional properties are unknown for the 5-HT5B receptor.