pmc logo image
Logo of bloodOriginal ArticleBlood JournalCurrent IssueAbout BloodSubmissionsSubscriptionsContact UsASH Homepage

Formats:

Blood. 2006 December 15; 108(13): 4126–4135.
Prepublished online 2006 August 29. doi: 10.1182/blood-2006-04-017046.
PMCID: PMC1895447
Selective leukemic-cell killing by a novel functional class of thalidomide analogs
Yun Ge, Idalia Montano, Gabriella Rustici, Wendy J. Freebern, Cynthia M. Haggerty, Wenwu Cui, Damaris Ponciano-Jackson, G. V. R. Chandramouli, Erin R. Gardner, William D. Figg, Mones Abu-Asab, Maria Tsokos, Sharon H. Jackson, and Kevin Gardner
The Advanced Technology Center, Laboratory of Receptor Biology and Gene Expression, National Cancer Institute (NCI), National Institutes of Health (NIH) Bethesda, MD; Clinical Pharmacology Research Core, SAIC-Frederick Inc, NCI-Frederick, Frederick, MD; Molecular Pharmacology Section, Cancer Therapeutics Branch, Center for Cancer Research, NCI, NIH, Bethesda, MD; Laboratory of Pathology, NCI, Bethesda, MD; and Laboratory of Host Defenses, National Institute of Allergy and Infectious Diseases, NIH, Bethesda, MD.
Correspondence: Kevin Gardner, The Advanced Technology Center, Laboratory of Receptor Biology and Gene Expression, National Cancer Institute, Bethesda, MD 20892-4605; e-mail: gardnerk/at/mail.nih.gov; and Sharon H. Jackson, Laboratory of Host Defenses, National Institute of Allergy and Infectious Diseases, National Institutes of Health, 10 Center Dr, CRC, MSC 1456, 5 West Labs, Rm 5-3942, Bethesda, MD 20892; sjackson/at/niaid.nih.gov.
Received April 13, 2006; Accepted August 3, 2006.
Using a novel cell-based assay to profile transcriptional pathway targeting, we have identified a new functional class of thalidomide analogs with distinct and selective antileukemic activity. These agents activate nuclear factor of activated T cells (NFAT) transcriptional pathways while simultaneously repressing nuclear factor-κB (NF-κB) via a rapid intracellular amplification of reactive oxygen species (ROS). The elevated ROS is associated with increased intracellular free calcium, rapid dissipation of the mitochondrial membrane potential, disrupted mitochondrial structure, and caspase-independent cell death. This cytotoxicity is highly selective for transformed lymphoid cells, is reversed by free radical scavengers, synergizes with the antileukemic activity of other redox-directed compounds, and preferentially targets cells in the S phase of the cell cycle. Live-cell imaging reveals a rapid drug-induced burst of ROS originating in the endoplasmic reticulum and associated mitochondria just prior to spreading throughout the cell. As members of a novel functional class of “redoxreactive” thalidomides, these compounds provide a new tool through which selective cellular properties of redox status and intracellular bioactivation can be leveraged by rational combinatorial therapeutic strategies and appropriate drug design to exploit cell-specific vulnerabilities for maximum drug efficacy.
Thalidomide is a synthetic glutamic acid derivative originally marketed as a sedative and antiemetic in 1954.1,2 However, in 1961 it was quickly withdrawn from distribution when its teratogenic properties were discovered.1,2 Several years later the serendipitous finding that thalidomide could allay the symptoms of erythema nodosum led to its re-emergence as a treatment for various proinflammatory and autoimmune conditions.3 Although many of the anti-inflammatory properties of thalidomide have been linked to its ability to repress tumor necrosis factor-α (TNF-α) expression,4 the mechanisms underlying most of its therapeutic effects, including its ability to costimulate T cells,5 remained a mystery. In 1994, speculation that thalidomide teratogenicity is linked to the repression of angiogenesis6 spawned a new wave of clinical investigations that expanded the use of thalidomide for the treatment of various malignancies, including multiple myeloma, melanoma, renal-cell carcinoma, and prostate cancer.1,2
The therapeutic promise of thalidomide became a motivation to develop more effective derivatives with reduced toxicity. Several chemical classes of compounds were subsequently developed. One group, referred to as immunomodulatory drugs (IMiDs),2 was identified because of its potential to promote T-cell costimulatory activity. A second group, referred to as the selective cytokine inhibitory drugs (SelCIDs), were found to be potent phosphodiesterase 4 (PDE4) inhibitors.7 Both groups repress TNF-α expression. IMiDs are currently in phase 2 and phase 3 clinical trials for multiple myeloma, metastatic melanoma, and prostate cancer.1,2 Although SelCIDs are also effective inhibitors of angiogenesis activity, and recent studies indicate that SelCIDs possess significant antitumor activity, they have no immunomodulatory properties.
In 1999, it was observed that thalidomide teratogenicity may be due to a species-specific conversion to free radical intermediates in embryonic tissue.8 This observation was complemented by the finding that thalidomide underwent a species-dependent intracellular bioactivation in the endoplasmic reticulum (ER) to generate antiangiogenic intermediates containing 5-OH and 5′-OH modifications of pthalimide and glutarimide rings, respectively.9 These 5-OH and 5′-OH intermediates of thalidomide therefore became the structural basis for the design of a third chemical class of thalidomide analogs, represented by N-substituted and tetraflourinated analogs.10 Like the IMiDs and SelCiDs, this category of drugs also possesses significant antitumor activity,10,11 although the precise mechanism of action is unclear. Thus, despite an extensive history as an important group of therapeutic compounds with promising results in countless clinical and preclinical studies, the functional pathways and cellular processes underlying the mechanism of action of thalidomide and its derivatives remain loosely defined.
Recently, we developed a high-throughput transcriptional assay that profiles transcriptional pathway targeting of various immunomodulatory compounds against an array of extracellular stimulatory conditions in human T cells.12 This expanded regulatory view provides a more selective and pathway-specific means of evaluating the drug-targeted integration of signal transduction events and transcriptional responses. In this study we examined and compared the transcriptional targeting of thalidomide (cc2001); the IMiDs Revlimid (cc5013, lenalidomide),2 and Actimid (cc4047);2 the SelCiD Rolipram (cc4001);13 and the thalidomide analogs CPS11, CPS45, and CPS49.11,14 By profiling the transcriptional pathway and signal transduction targeting of these “second-generation” derivatives and thalidomide we have identified a novel functional class of “redox-reactive” thalidomide analogs that act through direct manipulation of the cellular redox state to selectively kill leukemic cells.
Cell culture and high-throughput transfections
Growth of Jurkat T cells and high-throughput transfection were performed as previously described.12 Cells were harvested 5 hours after treatment. Normalization of luciferase values was performed as previously described.12 Phytohaemagglutinin (PHA; Amersham Biosciences, Piscataway, NJ), PMA (Sigma, St Louis, MO), ionomycin (Ion; EMD Biosciences, La Jolla, CA), anti-CD3 (OKT3 ascites; ATCC, Manassas, VA), cc4047 (Celgene, Summit, NJ), lenalidomide (Celgene), thalidomide (Celgene), BSO (Sigma), and anti-CD28 monoclonal antibody (Dr Carl June, University of Pennsylvania, Philadelphia, PA) were added as indicated. The average standard error for all measurements in the entire experiment was less than 15% of the mean, with a median of 14% of the mean. All plasmids used in the analysis are previously described.12 Peripheral-blood mononuclear cells (PBMCs) were isolated by apheresis from healthy human donors and proliferated by incubation with 1:1000 dilution of anti-CD3 in the presence of interleukin-2 (IL-2) at 10 U/mL for 5 days in RPMI with 10% fetal calf serum. Cell viability studies were performed in triplicate in 3 independent studies using the MTT assay according to the manufacturer's instructions (Roche Applied Sciences, Indianapolis, IN).
Cell-cycle synchronization by centrifugal elutriation
Elutriation was performed beginning with approximately 3 to 3.5 × 108 L1210 cells using a Beckman JE-5.0 elutriation rotor (Beckman Coulter, Fullerton, CA) equipped with a standard chamber and eluted with increasing flow rates (20-35 mL/minute) at constant rotor speed (2000 rpm) as previously described.15 Fractions (175 mL) were collected and analyzed by fluorescence-activated cell sorting (FACS) for cell-cycle position.
Antibodies and immunoblot analysis
Antibodies used were specific for Iκ-Bα, phospho-IκBα (Ser32), RelA(p65) and phospho-RelA (Ser536) (Cell Signaling Technology, Beverly, MA), PARP, caspase 7, and caspase 3 (BD Biosciences, Mountain View, CA). Whole-cell lysates were prepared in buffers containing 50 mM HEPES, 20 mM sodium pyrophosphate, 25 mM β-glycerophosphate, 50 mM sodium fluoride, 5 mM sodium molybdate, 5 mM EDTA, 150 mM orthophenanthroline, 1% NP-40, 0.5% deoxycholate, 1% Triton X-100, mammalian protease inhibitor (MPI; Sigma) and 0.2 mM Na2V04. Nuclear and cytosolic fractions were prepared by modification of a previously described procedure.16 A portion of the fractions (10-30 μg) were used for immunoblot analysis.
Cytokine measurements
Cytokine measurements were made on triplicate independent cell supernatants using Meso Scale Discovery (MSD) Multi-Spot plates and an MSD Sector Imager 6000 reader (Meso Scale Discovery, Gaithersburg, MD) as previously described.12 Ten cytokines were measured simultaneously in each well of 96-well plates using MSD's 10-Plex Human Cytokine Panel (granulocyte-macrophage colony-stimulating factor [GM-CSF], IL1β, IL2, IL-4, IL-5, IL-6, IL-8, IL-10, IL-12p70, and TNF-α) according to the manufacturer's instructions.
Intracellular Ca2+, mitochondrial membrane potential (Δψm), and cell death assays
Intracellular calcium was measured by FACS analysis of Fluo-3 (Molecular Probes, Eugene, OR)–loaded cells according to the manufacturer's instructions. Changes in the mitochondrial membrane potential were followed by FACS analysis of cells stained with DiIC15 (Molecular Probes) as recommended by the manufacturer. Cell death was measured by FACS analysis of membrane permeability and phosphatidylserine exposure by staining cells with 2 μg/mL FITC-labeled Annexin-V (Caltag Laboratories, Burlingame, CA) and/or 2.5 mg/mL of propidium iodide (PI; Caltag Laboratories) with incubation at 25°C for 10 minutes. All FACS analyses were carried out on a FASCalibur flow cytometer using CellQuest software (Becton Dickinson, San Jose, CA). Data were collected for 10 000 events. All measurements were performed in duplicate and are representative of at least 2 independent experiments.
ROS determinations
Reactive oxygen species (ROS) generation was determined by the increase in DCFDA fluorescence17 after mitogen and/or drug stimulation. Jurkat cells were washed, resuspended in 1% bovine serum albumin in Hanks buffered saline solution (BSA-HBSS) at 1 × 106 cells/mL and maintained at 37°C for analysis. At 15-minute intervals, resting or PHA/PMA-stimulated cells were treated with 10 μM of the indicated drugs (thalidomide, CPS11, CPS45, and CPS49). The oxidation-sensitive dye DCFDA (Invitrogen, Carlsbad, CA), was added, 15 minutes prior to harvest, at 2 μM final concentration. The incubation was terminated by 3-fold dilution of the sample with ice-cold 1% BSA-HBSS. The cells were washed with ice-cold 1% BSA-HBSS before FACS analysis. The stimulated increase in dye oxidation was calculated as the percentage increase in mean channel fluorescence (MCF) of drug-stimulated cells over that of unstimulated cells for each time point with the following equation: [(MCF (stimulated) – MCF (unstimulated)/MCF (unstimulated)] × 100. Results are an average of 3 independent experiments.
Electron microscopy
Cell pellets were fixed in 2.5% glutaraldehyde, postfixed in 0.5% osmium tetroxide, dehydrated, and embedded in Spurs epoxy resin. Ultrathin sections (90 nm) were made and double-stained with uranyl acetate and lead citrate, and viewed with a Philips CM10 transmission electron microscope (Phillips Electronics, Mahway, NJ).
Statistical analysis
Principal component analysis (PCA) was performed using Partek Pro 5.1 (Partek, St Louis, MO). MMC GeneLinker Gold (Molecular Mining Corp, Raleigh, NC) was used for hierarchical clustering and dendrogram assembly. Median dose effect analysis18 to calculate the combination index (CI) for coadministration of drugs was performed with Calcysyn (Biosoft, Ferguson, MO) using cell viability (MTT assay) as the endpoint.
Transcriptional pathway targeting by thalidomide and its analogs are highly pleiotropic in human T cells
All 7 thalidomide compounds (Figure 1AFigure 1.) were compared for their ability to influence the transcriptional pathway targeting of mitogen-stimulated T cells. Six transcriptional reporters (or “transcriptional targets”) were used in the study: (1) CRE, to detect the activation of CREB-mediated transcriptional pathways; (2) nuclear factor-κB (NF-κB), to detect activation of NF-κB signaling; (3) nuclear factor of activated T cells (NFAT), to detect calcium/calcineurin-mediated activation of NFAT transcriptional pathways; (4) AP1, to detect activation of AP1-driven transcriptional pathways; (5) CD28RE/AP1, to detect ras-mediated combinatorial targeting of NF-κB, AP1, and CREB complexes at the CD28RE/AP1 composite element of the IL-2 promoter16; and (6) IL-2, to assess the integrated targeting of the previously mentioned 5 elements within the context of the IL-2 300-bp proximal promoter.12 The individually transfected cells were then stimulated with 1 of 16 different combinations of 5 known T-cell mitogens, including PMA, Ion, PHA, stimulating monoclonal antibodies against the T-cell receptor (αCD3), and stimulating monoclonal antibodies against the CD28 receptor (αCD28)12 (Figure 1BFigure 1., left panel). For each drug and control this produces a 96-element (16 × 6) analytical matrix containing the transcriptional output from 6 reporters stimulated under 16 mitogenic conditions.12 When this transcriptional output is analyzed by hierarchical clustering, a distinct transcriptional “fingerprint” for each compound becomes apparent, demonstrating the true pleiotropic nature of their activity (Figure 1BFigure 1.).
Figure 1.
Figure 1.
Figure 1.
Transcriptional pathway profiling in human T cells reveals a novel functional class of thalidomide analogs. (A) Chemical structures of thalidomide compounds profiled in this study. (B) The transcriptional targeting of thalidomide (100 μM), CPS11, (more ...)
Specific trends in thalidomide analog transcriptional pathway targeting reveal a novel functional class
To objectively discern regulatory trends or patterns common to any of the 7 compounds, we used the PCA method.12 PCA is a computational technique used to analyze data with multiple variables or “dimensions.” This is accomplished by a mathematical transformation that reduces the 96 variables (shown in Figure 1BFigure 1.) to a smaller number of composite independent variables that capture most of the characteristics of the data set.12,19 This reduction in variables or “dimensions” simplifies the analysis graphically and allows for trends in the data to be more easily appreciated. In Figure 1CFigure 1. the 7 compounds and control are compared with each other by a PCA that reduces the 96 different variables, measured for each compound, to 3 independent composite variables or principal components (PCs) that capture more than 75% of the characteristics of the whole data set and are graphed as 3 orthogonal vectors (PC1-PC3). The 7 thalidomide compounds and control are then plotted within these 3 dimensions to graphically illustrate their relative similarities and differences by their spatial separation (Figure 1CFigure 1.). A plot of the amount of variance or “characteristics” of the data set captured by each PC is shown in Figure 1DFigure 1.. Most of the variance within the data set (ie, its characteristics) is captured in the first 3 PCs.
CPS11, CPS45, and CPS49 preferentially induce NFAT transcriptional pathways while simultaneously inhibiting NF-κB activity
One readily apparent trend in the PCA shown in Figure 1CFigure 1. is the clear separation of CPS11, CPS45, and CPS49 from the other thalidomide compounds along PC1. This indicates these compounds share 1 or more characteristics defined by the composition of PC1 that distinguishes them from the others. To determine relevant contributions to the composition of PC1, we examined its eigenvector coefficients.12 A profile of the magnitude of the PC1 eigenvector coefficients reveals a strong contribution from both NF-κB– and NFAT-derived variables (Figure 2AFigure 2.). Notably, the NFAT and NF-κB contributions are anticorrelated, indicating that 1 of these pathways is activated while the other is repressed. Re-examination of the hierarchical clustering in Figure 1BFigure 1. confirms these trends. The CPS compounds induce NFAT transcription pathways while repressing NF-κB pathways.
Figure 2.
Figure 2.
Figure 2.
Preferential pattern of NFAT and NF-κB transcriptional pathway targeting by CPS11, CPS45, and CPS49. (A) Plotting of the magnitude (y-axis) of the Eigenvector coefficients (x-axis) derived from PC1 reveals strong anticorrelated contribution from (more ...)
An alternate PCA approach to determine the factors that distinguish CPS11, CPS45, and CPS49 as a group is shown in Figure 2BFigure 2.. Each mitogen combination is evaluated using a 6 × 8 matrix (6 reporters and 8 drug conditions, including control), and the PCA is presented in biplot format where the vectors representing the original variables of the reporter responses are displayed within the PCA.12 The averaged influence of each of the 16 mitogen combinations on the 6 transcriptional pathways is grouped in projections based on how they respond to the 7 different drugs.12 As shown in Figure 2BFigure 2., the grouped data points representing the transcriptional activity of the 16 mitogen combinations in the presence of CPS11, CPS45, and CPS49 all project in a direction that correlates most with the projection of the NFAT axis and are nearly orthogonal to the grouped projections in the presence of the other thalidomide compounds. This correlation in projection indicates that these compounds make significant contribution to the variance of the transcriptional profiles by targeting NFAT transcriptional pathways.
A quantitative presentation of the targeting of NFAT and NF-κB pathways by CPS11, CPS45, and CPS49 is presented in histogram format in Figure 2CFigure 2.. All 3 drugs strongly repress NF-κB while stimulating NFAT with relative potencies of CPS45 > CPS49 > CPS11. These differences are most pronounced in cells stimulated with PMA/αCD28, PHA/PMA, and PHA/PMA/αCD28. CPS45 showed the highest potency and therefore became the major focus for subsequent molecular characterization as the lead representative of this novel functional class.
Several thalidomide compounds have been shown to inhibit NF-κB activation; however, the influence on NF-κB activation is highly dependent on the mode of stimulation.1,2 One of the major mechanisms of NF-κB regulation is through its cytoplasmic sequestration by I-κBα, whose degradation following phosphorylation-dependent ubiquitylation leads to NF-κB activation.20 Treatment of cells with CPS45 prevents degradation of IκBα, resulting in elevated steady-state levels of IκBα after 1 hour of mitogen stimulation (Figure 3AFigure 3., top panel). In addition to IκB phosphorylation, NF-κB pathways are also stimulated by rapid phosphorylation of the RelA (p65) subunit on serine 536 following mitogen stimulation.21 Like IκB phosphorylation, this modification is mediated by the IκB kinase complex and can be detected as early as 15 minutes following stimulation (Figure 3AFigure 3., bottom panel). As shown in Figure 3AFigure 3., rapid phosphorylation of both IκB and RelA in whole-cell lysates of mitogen-stimulated cells is repressed by CPS45. These findings suggest that processes mediating both NF-κB nuclear translocation and transactivation are inhibited by CPS45.
Figure 3.
Figure 3.
Figure 3.
CPS11, CPS45, and CPS49 thalidomide analogs inhibit NF-κB, activate NFAT, and repress cytokine expression through elevated ROS. (A) Top panel shows immunoblot of phospho-IκBα, IκBα, and actin in the cytosol of Jurkat (more ...)
The primary mechanism that activates the NFAT pathway is through the calcium-dependent dephosphorylation of NFAT family members by the calmodulin-dependent phosphatase calcineurin.22 CPS45 stimulates NFAT activation in the presence or absence of mitogen stimulation (PHA/PMA), and both forms of stimulation are inhibited by the calcineurin inhibitor cyclosporin A (Figure 3BFigure 3., left panel). Interestingly, the combination of CPS45 and PHA/PMA is more than additive and confers some resistance to cyclosporine inhibition (Figure 3BFigure 3., right panel). This suggests that the targeting of NFAT pathways by CPS45 occurs at a step upstream of the regulation of NFAT by calcineurin.
CPS11, CPS45, and CPS49 do not activate classic targets of NFAT pathway activation
NFAT transcriptional pathways control expression of multiple proinflammatory cytokines, including IL-2, IL-4, IL-5, IL-8, IL-12, and GM-CSF.22 To assess how these NFAT targets are influenced by CPS45 and the other members of its functional class, the mitogen-stimulated expression of GM-CSF, IL-1β, IL-2, IL-4, IL-5, IL-6, IL-8, IL-10, IL-12p70, and TNF-α was measured in the presence of increasing doses of thalidomide, CPS11, CPS45, and CPS49 at 1, 2, and 5 hours following stimulation (Figure 3C-DFigure 3.). PCA comparison of cytokine targeting by these compounds shows that CPS11, CPS45, and CPS49 have very different collective targeting profiles compared with that of thalidomide (Figure 3CFigure 3.). Hierarchical clustering analysis of the proteomic profiles reveals the unexpected finding that the CPS class has a general tendency to repress rather than stimulate inflammatory cytokine secretion (Figure 3DFigure 3.). In contrast, thalidomide showed substantial induction of IL-2 expression as has been reported previously.5,23
CPS45 and CPS49 elevate intracellular ROS
The inability of the CPS-induced NFAT pathways to stimulated cytokine production suggested that this process might represent an atypical activation of NFAT. ROS have been shown to activate NFAT by mobilization of intracellular calcium secondary to ER stress.24,25 Such elevated ROS could antagonize cytokine gene expression. We therefore measured the ability of thalidomide and the CPS drugs to increase intracellular ROS using the cell-permeant oxidation-sensitive fluorescent dye DCFDA.17 Cells treated for 30 minutes with CPS45 or CPS49 generated significant levels of ROS. In contrast, thalidomide showed no difference and CPS11 showed only minimal elevation compared with that of untreated cells (Figure 3EFigure 3.). Notably, the ability of the 4 drugs to generate ROS was directly proportional to their capacity to stimulate NFAT. Significant levels of ROS are routinely generated during mitogen stimulation of T cells.17,26 Therefore, the generation of ROS by CPS45 was compared in both resting and PHA/PMA-stimulated T cells (Figure 3FFigure 3.). The addition of CPS45 results in a dramatic increase in ROS in PHA/PMA-stimulated T cells, indicating that CPS45 readily amplifies ROS levels generated from intracellular sources.
CPS45 induces a rapid ROS-mediated, caspase-independent cell death
The effects of ROS on cellular homeostasis are complex. Low levels can promote protein phosphorylation, mobilize calcium stores, and activate transcription factors.27-29 At higher concentrations it can damage membranes, inhibit caspases and other enzymes, and promote a necrotic cell death.30 The influence of 10 μM CPS45 on death pathways in T cells treated from 0 to 3 hours was examined by annexin and PI staining (Figure 4AFigure 4.). This time course reveals that within 30 minutes of treatment, annexin V–positive/PI-negative cells are detected, suggesting activation of programmed cell death apoptotic pathways. After 60 minutes there is a rapid conversion to annexin V and PI double-positive cells with a gradual increase of both classes thereafter, suggesting that both apoptotic and necrotic death pathways are triggered during CPS45 treatment.
Figure 4.
Figure 4.
Figure 4.
CPS45 induces rapid caspase-independent cell death reversed by antioxidants. (A) FACS measurement of 10 μM CPS45–induced cell death by annexin and PI staining after 0 to 3 hours of drug exposure. Percentages indicate percentage of cells (more ...)
CPS45 cytotoxicity is not significantly reversed by the broad-spectrum caspase inhibitor z-VAD-fmk (z-VAD). However, the addition of several different antioxidants produced substantial rescue from cell death (Figure 4BFigure 4.). These included DPI, an inhibitor of superoxide anion production by flavoprotein hydrogenases; manganese III TBAP (MnTBAP), a compound shown to mimic both superoxide dismutase and peroxidase activity; ebselen, a glutathione peroxidase mimetic; and NAC, a common dietary antioxidant found in Brussels spouts. The ROS generation is also associated with a rapid increase in intracellular calcium (within 15 minutes of treatment), and an equally rapid dissipation of the mitochondrial membrane potential (Figure 4CFigure 4.).
CPS45 does not increase the cleavage of caspase 7, caspase 3, or the major caspase substrate PARP; nor does it synergize with caspase activation following stimulation with PHA/PMA (Figure 4DFigure 4.). This is consistent with the observation that caspase inhibition provides little rescue from CPS45-induced cell death (Figure 4BFigure 4.).
Selective killing of transformed leukemic cells by CPS45
The levels of ROS are much higher in transformed and/or malignant cells when compared with normal cells due to their higher metabolic status.31 Since CPS45 amplifies the elevation of ROS in mitogen-stimulated Jurkat cells (Figure 3FFigure 3.), it is reasonable to assume that the threshold or “tolerance” of transformed cells for the CPS drugs will be lower in stimulated or proliferating cells compared with resting or nonproliferating cells. To test this hypothesis, the influence of increasing doses of CPS11, CPS45, CPS49 (1 nM-10 μM), and thalidomide (1 nM-200 μM) on the viability of Jurkat T cells (an acute lymphoblastic leukemia T-cell line) was compared with that of donor PBMCs following 8-, 16-, and 32-hour treatments (Figure 5AFigure 5.). CPS45 and CPS49 lead to significant killing of Jurkat cells as early as 8 hours after treatment at low micromolar concentrations. In contrast, the viability of PBMCs shows negligible changes. At 5 μM, CPS45 and CPS49 readily kill Jurkat cells, have little influence on PBMCs, and are reversed by micromolar concentrations of the N-acetyl-cysteine (Figure 5BFigure 5.). The cytotoxicity of CPS45, CPS49, CPS11, and thalidomide is in direct correlation with their ability to induce NFAT and generate ROS.
Figure 5.
Figure 5.
Figure 5.
CPS45 shows selective redox-dependent killing of transformed leukemic cells and acts in synergy with parthenolide to promote cell death. (A) Jurkat cells (left panels) and donor PBMCs (right panels) were incubated with increasing concentrations of CPS11, (more ...)
To explore the selectivity of the CPS drugs more thoroughly, the viability of 15 different transformed human cell lines (Table S1, available on the Blood website; see the Supplemental Materials link at the top of the online article) was compared with resting and proliferating primary human PBMCs (mitogen-induced PBMC blasts) after 16 hours of treatment with increasing concentrations of CPS45 (Figure 5CFigure 5.). As demonstrated by hierarchical clustering of the viability data, most of the lymphoid leukemia cell lines are sensitive to the drug. Those cells that showed significant resistance were resting PMBCs, K562 (erythroleukemia), HeLa (cervical carcinoma), and HUT78 (cutaneous T-cell lymphoma). PMBC blasts show greater sensitivity to the drugs, demonstrating that metabolic states are critical parameter for the selective cytotoxicity of CPS45/49.
Pharmacologic manipulation of the cellular redox state influences sensitivity to cell killing by redox-reactive thalidomide compounds
The redox state of the cell is a balance between ROS production and removal by enzymatic conversion or buffering by endogenous molecular antioxidants. Glutathione (GSH) is the major intracellular buffer for ROS. BSO depletes intracellular GSH through inhibition of γ-glutamylcysteine synthetase, an enzyme required for GSH synthesis.32 As shown in Figure 5DFigure 5., 24-hour preincubation with BSO significantly increases the sensitivity of HeLa cells to CPS45.
CPS45 synergizes with the antileukemic activity of parthenolide
Parthenolide (PTL) is a sesquiterpene lactone derived from the Mexican-Indian plant European feverfew (Tanacetium parthenium) that exhibits anti-inflammatory and antileukemic activity through a mechanism linked to increased ROS and inhibition of NF-κB activity.33 These similarities suggest that the combination of PTL and CPS45 might be synergistic. This prediction was tested by dose response curves and median dose effect analysis18 of cell viability following treatment with fixed ratios (1:2.5) of CPS45 and PTL for 24 hours (Figure 5EFigure 5.). At all doses tested, coadministration of CPS45 and PTL yielded CIs that were considerably less than 1, indicating clear a synergy between CPS45 and PTL.
CPS45 shows preferential cytotoxicity against cells in S phase
The mechanism by which ROS regulates cell cycle is not clear. However, ROS is known to play an important role in growth factor–stimulated progression from G1 to S phase, and decreases in ROS are associated with impaired G1 to S transition.34,35 Thus, it is likely that the elevation of ROS required for cells in S phase may increase their vulnerability to redox-reactive thalidomides. To answer this question the mouse L1210 cells were used, as they are easily synchronized by centrifugal elutriation without the confounding cellular stress of mitotic poisons or antimetabolic drugs. As shown in Figure 6AFigure 6., L1210 cells are sensitive to CPS45 in contrast to thalidomide. Moreover, like Jurkat cells and in contrast to HeLa cells, treatment of L1210 with CPS45 causes significant elevation of intracellular ROS, while thalidomide at a 20-fold–higher concentration has no effect (Figure 6BFigure 6.). As anticipated, treatment of L1210 cells synchronized in G1, S, and G2 phases shows increased cytotoxicity in S phase (Figure 6CFigure 6.). This observation also explains the previously observed increased sensitivity of PMBC blasts to CPS45 treatment (Figure 5CFigure 5.).
Figure 6.
Figure 6.
Figure 6.
Dose-dependent elevation of intracellular ROS by CPS45 is cell specific and shows preferential targeting of cells in S phase. (A) Comparison of dose-dependent killing of the mouse L1210 leukemia cell line by CPS45 and thalidomide. (B) Profile of cell (more ...)
CPS45-induced ROS arises in the ER and mitochondrial compartment
To identify the subcellular compartment in which the CPS45-induced ROS arises, we used the oxidation-sensitive fluorescent dye DCFDA coupled with live-cell fluorescent imaging to localize the initial ROS formation (Figure 7AFigure 7.). Resting Jurkat T cells previously loaded with DCFDA were applied to lysine-coated slides, and video images of the cells were captured over a 45-minute duration at 10-second intervals after the addition of 10 μM CPS45 (Video S1). In less than 20 minutes, increased ROS is detected by fluorescence in an internal, perinuclear compartment consistent with the ER and associated mitochondria (Figure 7AFigure 7., center panel) before spreading throughout the entire cell (Figure 7AFigure 7., right panel).
Figure 7.
Figure 7.
Figure 7.
ROS develop within minutes in an intracellular compartment in CPS45-treated leukemic cells and selectively cause damage to the mitochondria of leukemic cells. (A) Jurkat T cells were bound to poly-lysine–coated slides and preincubated with the (more ...)
Confirmatory results are provided by ultrastructural studies. When examined by electron microscopy, Jurkat cells treated for 4 hours with 10 μM CPS45 show dramatic changes consistent with necrotic cell death, while PBMCs are unaffected (Figure 7BFigure 7., left panel). Moreover, examination of Jurkat mitochondria following 1 hour of exposure to 10 μM CPS45 reveal severe swelling, loss of cristae, and the formation of protein precipitates within the mitochondrial matrix (Figure 7BFigure 7., top right panel). In contrast, the mitochondria of PBMCs show no detectable alterations, even after a 4-hour exposure to 10 μM CPS45 (Figure 7BFigure 7., bottom right panel).
In this study we profiled 7 different thalidomide-related compounds based on their transcriptional pathway targeting in response to multiple T-cell mitogens. Although we found that thalidomide and its analogs have very pleiotropic influences, a regulatory signature was identified that defines a new functional class of “redox-reactive” thalidomide analogs uniquely characterized by their ability to up-regulate NFAT transcriptional pathways through the amplification of intracellular ROS. A schematic of the proposed sequence of events is shown in Figure 7CFigure 7..
Oxidative stress plays a well-recognized role in mediating multiple forms of cell death in a variety of tissues.26,36 The contribution of ROS to the mechanism of chemotherapeutic cytotoxicity was recognized as early as 40 years ago; however, its manipulation as a treatment paradigm in cancer has only recently emerged as a promising new strategy.37-39 Cellular ROS is a dynamic state defined by the equilibrium between the cell's internal generation of ROS and its antioxidant system.36 These properties determine the selective nature of its influence on individual cells and cell types. Malignant cells tend to have higher basal levels of ROS due to higher metabolic rates associated with multiple activated oncogenic pathways and the increased likelihood of mitochondrial dysfunction.31,37 Mitochondrial genes have a higher frequency of mutation due to their lack of histones and introns and a limited DNA repair capacity.37 This higher vulnerability to mitochondrial dysfunction can be exploited by redox-targeting therapeutic strategies.
Leukemias have a higher susceptibility to elevated ROS that is often enhanced following chemotherapy, possibly secondary to accrued mitochondrial DNA damage.40-43 This is consistent with reports that cells obtained from patients with chronic lymphocytic leukemia (CLL) that is refractory to chemotherapy show higher levels of ROS and are more sensitive to cytotoxic compounds that elevate ROS.42 The in vitro antileukemic activity of the redox-reactive thalidomides indicates that this class of compounds may be effective against CLL and other leukemias.
Although the inhibition of NF-κB is a common feature of most thalidomide analogs, prior studies have shown that this activity is highly dependent on the mode of stimulation. In fact, earlier observations in Jurkat cells have shown that PMA-activated NF-κB is resistant to thalidomide.44 Moreover, hydrogen peroxide–activated NF-κB is paradoxically induced rather than inhibited by thalidomide.44 The ability to rapidly and directly amplify endogenous ROS levels is a property unique to the redox-reactive thalidomides. Like PTL, the repression of NF-κB pathways by CPS45 suggests a role for this pathway in its antileukemic activity and is also likely to partially explain the observed repression of inflammatory cytokine secretion (Figure 3DFigure 3.). While loss of mitochondrial stability secondary to NF-κB repression undoubtedly contributes to its cytotoxicity, the rapid kinetics of CPS45-mediated mitochondrial destabilization and ROS amplification suggests influences that are less dependent on transcription. In this regard, it is important to note that a dose-dependent dual targeting of NF-κB and ROS by PTL has been recently proposed to occur in leukemic cells.45 Future analysis of the kinetics of this CPS45/PTL similarity and its dose dependence is certain to provide further insights into the mechanism of action of the redox-reactive thalidomides.
The synergy between the redox-reactive thalidomides and PTL suggests that a similar combinatorial potential may be elicited by coadministration with other chemotherapeutic compounds that use ROS in cytotoxic pathways. Known candidates with selective antileukemic activity include arsenic trioxide,46 bortezomib,47 DNA alkylating agents,48 ceramide49 and CD47 ligation,50 BSO,51 rituximab,52 motexafin gadolinium (MGd),53 and 2-methoxyestradiol (2-ME).42
Prior reports had identified a role for free radicals in the action of the parent compound thalidomide, but this property is indirect and requires multiple species-specific bioactivation steps.8 As with other compounds capable of generating free radicals, there are many aspects of the spectrum of action of redox-reactive thalidomides that remain to be determined. Bone marrow suppression and nephrotoxicity will have to be explored. In initial animal studies, high levels of CPS11 and CPS49 were well tolerated, and CPS45 showed no toxicity at all doses tested.10 The redox-reactive thalidomides are therefore ideal candidates for expanded studies in animal models for leukemia and lymphoma.
More than 70 years ago, Otto Warburg noted that cancer-transformed cells displayed an altered metabolism characterized by a shift in energy production from mitochondrial-mediated oxidative phosphorylation to glycolysis in the cytoplasm.54 Although years later it is still debated whether this shift is causative or a result of cancer, this change in cellular state represents a major vulnerability that can be therapeutically exploited. The precise components of the organelles that are rapidly targeted by the redox-reactive thalidomides to amplify ROS levels are currently under investigation and are likely to involve multiple components of the cytochrome P-450 system in addition to other ER and mitochondrial constituents.55 Given the high degree of genetic variation in components of these systems, many effects of the redox-reactive thalidomides are likely to reflect population-specific differences and biochemistries that will require further exploration.
In summary, by transcriptional pathway profiling, we have identified a new functional class of thalidomide analogs that selectively kill leukemic cells by promoting necrotic-programmed cell death. The compounds have a mechanism of action distinct from other known thalidomides and provide a strategy for their use as chemotherapeutic modifiers of redox signaling and cytotoxicity. These findings reinforce the emerging paradigm that targeting the redox state can be an effective strategy against cancer and will influence the design and implementation of other agents that exploit similar cell-specific vulnerabilities to chemotherapeutic intervention.
Authorship
Contribution: Y.G. and I.M. contributed equally to this work.
Conflict-of-interest disclosure: The authors declare no competing financial interests.
Supplementary Material
[Supplemental Table and Video]
Acknowledgments
We would like to thank the NIH Fellows Editorial Board for editorial assistance in the preparation of this manuscript.
This work was supported by the NCI, NIH, under contract N01-CO-12400 and funds from Intramural Research Program of the NIH, NCI, Center for Cancer Research. The content of this publication does not necessarily reflect the views and policies of the Department of Health and Human Services, nor does mention of trade names, commercial products or organizations imply endorsement by the U.S. Government.
Notes
Prepublished online as Blood First Edition Paper, August 29, 2006; DOI 10.1182/blood-2006-04-017046.
The online version of this manuscript contains a data supplement.
The publication costs of this article were defrayed in part by page charge payment. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734.
1. Franks ME, Macpherson GR, Figg WD. Thalidomide. Lancet. 2004;363: 1802-1811. [PubMed]
2. Bartlett JB, Dredge K, Dalgleish AG. The evolution of thalidomide and its IMiD derivatives as anticancer agents. Nat Rev Cancer. 2004;4: 314-322. [PubMed]
3. Teo SK, Resztak KE, Scheffler MA, et al. Thalidomide in the treatment of leprosy. Microbes Infect. 2002;4: 1193-1202. [PubMed]
4. Moreira AL, Sampaio EP, Zmuidzinas A, et al. Thalidomide exerts its inhibitory action on tumor necrosis factor alpha by enhancing mRNA degradation. J Exp Med. 1993;177: 1675-1680. [PubMed]
5. Haslett PA, Corral LG, Albert M, Kaplan G. Thalidomide costimulates primary human T lymphocytes, preferentially inducing proliferation, cytokine production, and cytotoxic responses in the CD8+ subset. J Exp Med. 1998;187: 1885-1892. [PubMed]
6. D'Amato RJ, Loughnan MS, Flynn E, Folkman J. Thalidomide is an inhibitor of angiogenesis. Proc Natl Acad Sci U S A. 1994;91: 4082-4085. [PubMed]
7. Marriott JB, Muller G, Stirling D, Dalgleish AG. Immunotherapeutic and antitumour potential of thalidomide analogues. Expert Opin Biol Ther. 2001;1: 675-682. [PubMed]
8. Parman T, Wiley MJ, Wells PG. Free radical-mediated oxidative DNA damage in the mechanism of thalidomide teratogenicity. Nat Med. 1999;5: 582-585. [PubMed]
9. Bauer KS, Dixon SC, Figg WD. Inhibition of angiogenesis by thalidomide requires metabolic activation, which is species-dependent. Biochem Pharmacol. 1998;55: 1827-1834. [PubMed]
10. Ng SS, Gutschow M, Weiss M, et al. Antiangiogenic activity of N-substituted and tetrafluorinated thalidomide analogues. Cancer Res. 2003;63: 3189-3194. [PubMed]
11. Ng SS, Macpherson GR, Gutschow M, Eger K, Figg WD. Antitumor effects of thalidomide analogs in human prostate cancer xenografts implanted in immunodeficient mice. Clin Cancer Res. 2004;10: 4192-4197. [PubMed]
12. Freebern WJ, Haggerty CM, Montano I, et al. Pharmacologic profiling of transcriptional targets deciphers promoter logic. Pharmacogenomics J. 2005;5: 305-323. [PubMed]
13. Zhu J, Mix E, Winblad B. The antidepressant and antiinflammatory effects of rolipram in the central nervous system. CNS Drug Rev. 2001;7: 387-398. [PubMed]
14. Kumar S, Raje N, Hideshima T, et al. Antimyeloma activity of two novel N-substituted and tetraflourinated thalidomide analogs. Leukemia. 2005;19: 1253-1261. [PubMed]
15. Dowdy SF, Van Dyk LF, Schreiber GS. Synchronisation of cells by elutriation. In: Adolph KW, ed. Human Genome Methods. Boca Raton, FL: CRC Press; 1997: 121-132.
16. Butscher WG, Powers C, Olive M, Vinson C, Gardner K. Coordinate transactivation of the interleukin-2 CD28 response element by c-Rel and ATF-1/CREB2. J Biol Chem. 1998;273: 552-560. [PubMed]
17. Devadas S, Zaritskaya L, Rhee SG, Oberley L, Williams MS. Discrete generation of superoxide and hydrogen peroxide by T cell receptor stimulation: selective regulation of mitogen-activated protein kinase activation and fas ligand expression. J Exp Med. 2002;195: 59-70. [PubMed]
18. Chou TC, Talalay P. Quantitative analysis of dose-effect relationships: the combined effects of multiple drugs or enzyme inhibitors. Adv Enzyme Regul. 1984;22: 27-55.:27-55. [PubMed]
19. Alter O, Brown PO, Botstein D. Singular value decomposition for genome-wide expression data processing and modeling. Proc Natl Acad Sci U S A. 2000;97: 10101-10106. [PubMed]
20. Rothwarf DM, Karin M. The NF-kappa B activation pathway: a paradigm in information transfer from membrane to nucleus. Sci STKE. 1999;1999: RE1. [PubMed]
21. Sakurai H, Suzuki S, Kawasaki N, et al. Tumor necrosis factor-alpha-induced IKK phosphorylation of NF-kappaB p65 on serine 536 is mediated through the TRAF2, TRAF5, and TAK1 signaling pathway. J Biol Chem. 2003;278: 36916-36923. [PubMed]
22. Im SH, Rao A. Activation and deactivation of gene expression by Ca2+/calcineurin-NFAT-mediated signaling. Mol Cells. 2004;18: 1-9. [PubMed]
23. Shannon EJ, Sandoval F. Thalidomide increases the synthesis of IL-2 in cultures of human mononuclear cells stimulated with Concanavalin-A, Staphylococcal enterotoxin A, and purified protein derivative. Immunopharmacology. 1995;31: 109-116. [PubMed]
24. Li J, Huang B, Shi X, et al. Involvement of hydrogen peroxide in asbestos-induced NFAT activation. Mol Cell Biochem. 2002;234-235: 161-168. [PubMed]
25. Huang C, Li J, Costa M, et al. Hydrogen peroxide mediates activation of nuclear factor of activated T cells (NFAT) by nickel subsulfide. Cancer Res. 2001;61: 8051-8057. [PubMed]
26. Hildeman DA, Mitchell T, Teague TK, et al. Reactive oxygen species regulate activation-induced T cell apoptosis. Immunity. 1999;10: 735-744. [PubMed]
27. Bielefeldt K, Whiteis CA, Sharma RV, Abboud FM, Conklin JL. Reactive oxygen species and calcium homeostasis in cultured human intestinal smooth muscle cells. Am J Physiol. 1997;272: G1439-G1450. [PubMed]
28. Rhee SG, Bae YS, Lee SR, Kwon J. Hydrogen peroxide: a key messenger that modulates protein phosphorylation through cysteine oxidation. Sci STKE. 2000;2000: E1.
29. McCullough KD, Martindale JL, Klotz LO, Aw TY, Holbrook NJ. Gadd153 sensitizes cells to endoplasmic reticulum stress by down-regulating Bcl2 and perturbing the cellular redox state. Mol Cell Biol. 2001;21: 1249-1259. [PubMed]
30. Kannan K, Jain SK. Oxidative stress and apoptosis. Pathophysiology. 2000;7: 153-163. [PubMed]
31. Toyokuni S, Okamoto K, Yodoi J, Hiai H. Persistent oxidative stress in cancer. FEBS Lett. 1995; 358: 1-3. [PubMed]
32. Martensson J, Jain A, Stole E, et al. Inhibition of glutathione synthesis in the newborn rat: a model for endogenously produced oxidative stress. Proc Natl Acad Sci U S A. 1991;88: 9360-9364. [PubMed]
33. Guzman ML, Rossi RM, Karnischky L, et al. The sesquiterpene lactone parthenolide induces apoptosis of human acute myelogenous leukemia stem and progenitor cells. Blood. 2005;105: 4163-4169. [PubMed]
34. Sundaresan M, Yu ZX, Ferrans VJ, Irani K, Finkel T. Requirement for generation of H2O2 for platelet-derived growth factor signal transduction. Science. 1995;270: 296-299. [PubMed]
35. Deng X, Gao F, May WS, Jr. Bcl2 retards G1/S cell cycle transition by regulating intracellular ROS. Blood. 2003;102: 3179-3185. [PubMed]
36. Le Bras M, Clement MV, Pervaiz S, Brenner C. Reactive oxygen species and the mitochondrial signaling pathway of cell death. Histol Histopathol. 2005;20: 205-219. [PubMed]
37. Pelicano H, Carney D, Huang P. ROS stress in cancer cells and therapeutic implications. Drug Resist Updat. 2004;7: 97-110. [PubMed]
38. Engel RH, Evens AM. Oxidative stress and apoptosis: a new treatment paradigm in cancer. Front Biosci. 2006;11: 300-312. [PubMed]
39. Hileman EO, Liu J, Albitar M, Keating MJ, Huang P. Intrinsic oxidative stress in cancer cells: a biochemical basis for therapeutic selectivity. Cancer Chemother Pharmacol. 2004;53: 209-219. [PubMed]
40. Devi GS, Prasad MH, Saraswathi I, et al. Free radicals antioxidant enzymes and lipid peroxidation in different types of leukemias. Clin Chim Acta. 2000;293: 53-62. [PubMed]
41. Carew JS, Huang P. Mitochondrial defects in cancer [review]. Mol Cancer. 2002;1: 9. [PubMed]
42. Zhou Y, Hileman EO, Plunkett W, Keating MJ, Huang P. Free radical stress in chronic lymphocytic leukemia cells and its role in cellular sensitivity to ROS-generating anticancer agents. Blood. 2003;101: 4098-4104. [PubMed]
43. Carew JS, Zhou Y, Albitar M, et al. Mitochondrial DNA mutations in primary leukemia cells after chemotherapy: clinical significance and therapeutic implications. Leukemia. 2003;17: 1437-1447. [PubMed]
44. Majumdar S, Lamothe B, Aggarwal BB. Thalidomide suppresses NF-kappa B activation induced by TNF and H2O2, but not that activated by ceramide, lipopolysaccharides, or phorbol ester. J Immunol. 2002;168: 2644-2651. [PubMed]
45. Li-Weber M, Palfi K, Giaisi M, Krammer PH. Dual role of the anti-inflammatory sesquiterpene lactone: regulation of life and death by parthenolide. Cell Death Differ. 2005;12: 408-409. [PubMed]
46. Chou WC, Dang CV. Acute promyelocytic leukemia: recent advances in therapy and molecular basis of response to arsenic therapies. Curr Opin Hematol. 2005;12: 1-6. [PubMed]
47. Fribley A, Zeng Q, Wang CY. Proteasome inhibitor PS-341 induces apoptosis through induction of endoplasmic reticulum stress-reactive oxygen species in head and neck squamous cell carcinoma cells. Mol Cell Biol. 2004;24: 9695-9704. [PubMed]
48. Zong WX, Ditsworth D, Bauer DE, Wang ZQ, Thompson CB. Alkylating DNA damage stimulates a regulated form of necrotic cell death. Genes Dev. 2004;18: 1272-1282. [PubMed]
49. Ogretmen B, Hannun YA. Biologically active sphingolipids in cancer pathogenesis and treatment. Nat Rev Cancer. 2004;4: 604-616. [PubMed]
50. Mateo V, Lagneaux L, Bron D, et al. CD47 ligation induces caspase-independent cell death in chronic lymphocytic leukemia. Nat Med. 1999;5: 1277-1284. [PubMed]
51. Gartenhaus RB, Prachand SN, Paniaqua M, Li Y, Gordon LI. Arsenic trioxide cytotoxicity in steroid and chemotherapy-resistant myeloma cell lines: enhancement of apoptosis by manipulation of cellular redox state. Clin Cancer Res. 2002;8: 566-572. [PubMed]
52. Bellosillo B, Villamor N, Lopez-Guillermo A, et al. Complement-mediated cell death induced by rituximab in B-cell lymphoproliferative disorders is mediated in vitro by a caspase-independent mechanism involving the generation of reactive oxygen species. Blood. 2001;98: 2771-2777. [PubMed]
53. Evens AM. Motexafin gadolinium: a redox-active tumor selective agent for the treatment of cancer. Curr Opin Oncol. 2004;16: 576-580. [PubMed]
54. Weinhouse S. The Warburg hypothesis fifty years later. Z Krebsforsch Klin Onkol Cancer Res Clin Oncol. 1976;87: 115-126. [PubMed]
55. Ando Y, Price DK, Dahut WL, et al. Pharmacogenetic associations of CYP2C19 genotype with in vivo metabolisms and pharmacological effects of thalidomide. Cancer Biol Ther. 2002;1: 669-673. [PubMed]

See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph
See more articles cited in this paragraph