• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Abnorm Psychol. Author manuscript; available in PMC Feb 1, 2014.
Published in final edited form as:
PMCID: PMC3570590
NIHMSID: NIHMS434488

The Structure of Psychopathology: Toward an Expanded Quantitative Empirical Model

Abstract

There has been substantial recent interest in the development of a quantitative, empirically based model of psychopathology. However, the majority of pertinent research has focused on analyses of diagnoses, as described in current official nosologies. This is a significant limitation because existing diagnostic categories are often heterogeneous. In the current research, we aimed to redress this limitation of the existing literature, and to directly compare the fit of categorical, continuous, and hybrid (i.e., combined categorical and continuous) models of syndromes derived from indicators more fine-grained than diagnoses. We analyzed data from a large representative epidemiologic sample (the 2007 Australian National Survey of Mental Health and Wellbeing; N = 8,841). Continuous models provided the best fit for each syndrome we observed (Distress, Obsessive Compulsivity, Fear, Alcohol Problems, Drug Problems, and Psychotic Experiences). In addition, the best fitting higher-order model of these syndromes grouped them into three broad spectra: Internalizing, Externalizing, and Psychotic Experiences. We discuss these results in terms of future efforts to refine emerging empirically based, dimensional-spectrum model of psychopathology, and to use the model to frame psychopathology research more broadly.

Keywords: DSM, Classification, Nosology, Mental Disorder, Psychopathology

The Structure of Psychopathology: Toward an Expanded Quantitative Empirical Model

Historically, there have been two major approaches to classifying psychopathology, which Blashfield (1984) characterized as the neo-Krapelinian and quantitative approaches. The neo-Krapelinian approach is based on expert consensus and has framed the modern Diagnostic and Statistical Manuals of Mental Disorders (DSMs III to IV-TR), and has thereby been highly influential in modern psychopathology research. In addition to being constructed by expert consensus, the modern neo-Krapelinian DSMs embody an a priori conceptualization of nearly all forms of psychopathology as categorical and polythetic (or, occasionally, monothetic). With very few exceptions, the mental disorders delineated in the modern DSMs are assumed to be categorical in nature, such that people either meet criteria for a diagnosis or they do not. Furthermore, people can meet the criteria for most specific disorders in a variety of ways because the DSM’s polythetic diagnostic categories are constructed from sets of criteria, where symptoms within the criteria are often treated as fungible.

The modern DSMs have been helpful in psychopathology research by providing consensus targets for research and clinical application, many of which can be assessed reliably (e.g., Brown, Di Nardo, Lehman, & Campbell, 2001). Nevertheless, research with these constructs has also revealed their limitations and highlighted the need to refine mental disorder classification in a more data-driven way. As described extensively elsewhere, the DSM’s polythetic categorical approach to classification results in a number of structural problems, such as extensive comorbidity among putatively distinct disorders and heterogeneity within ostensibly coherent categories (Brown & Barlow, 2009; Clark & Watson, 1991; Krueger, Markon, Patrick, Benning, & Kramer, 2007; Lilienfeld, Waldman, & Israel, 1994; Regier, Narrow, Kuhl, & Kupfer, 2009; Widiger & Samuel, 2005; Watson, 2009). As the limitations of the extant structure have become clear, new developments in statistical modeling have also emerged. These methods allow for the estimation and direct quantitative comparison of models based on categorical, dimensional, and hybrid (i.e., latent variables that have dimensional and categorical aspects) latent structures (Lubke & Muthén, 2005; Markon & Krueger, 2006; Ruscio, Haslam, & Ruscio, 2006; von Eye & Clogg, 1994). As a result, key conceptual issues that were treated as a priori assumptions in the recent DSMs, such as the notion that most forms of psychopathology are well characterized as discrete dichotomies, can now be addressed empirically via contemporary quantitative modeling. These quantitative approaches have the potential to provide classification systems for research and clinical application with a solid empirical foundation.

Multivariate Modeling of the Structure of DSM Categories

Reliable DSM diagnoses provided a starting point for understanding the nature of psychopathology. Indeed, the empirical evaluation of these diagnoses has led to progress in delineating the risk factors and clinical correlates of psychopathology (Robins & Guze, 1970). Nonetheless, contrary to predictions implicit in Robins & Guze’s (1970) early treatise, work aimed at validating the DSM diagnoses has not identified diagnosis-specific etiological pathways. As a result, research efforts have begun to focus on identifying the reasons that many of the putatively distinct DSM diagnoses share similar risk factors and clinical correlates (Andrews et al., 2009). Additionally, over successive iterations of the diagnostic manual, there has been a notable increase in the number of individual diagnoses; researchers have also identified significant rates of covariation suggesting that the “co-morbidity” among these myriad disorders is at least in part illusory in nature, inasmuch as there may be closer empirical connections among disorders than implied by the historical proliferation of categories in official nosologies (Rutter, 2011).

Latent variable modeling has proven useful for examining the reasons that multiple diagnoses share risk factors and clinical correlates. In particular, these methods can be used to identify the latent sources of covariation among observed psychopathology. One strength of this approach to understanding the nature of psychopathology is that models can examine the structural relationships among the nosological constructs that are intended for clinical and research diagnosis. For instance, these modeling procedures have shown that many of the common DSM diagnoses (e.g., anxiety disorders, unipolar mood disorders, substance use disorders) are undergirded by two, dimensional latent constructs—namely, the Internalizing and Externalizing spectra. This Internalizing-Externalizing model of psychopathology is robust across age, sex, ethnicity, culture, informant type and DSM Axes (Achenbach, 1966; Eaton et al., 2012; Eaton, Krueger, & Oltmanns, 2011; Forbush & Watson, in press; Kramer, Krueger, & Hicks, 2008; Krueger, Capsi, Moffitt, & Silva, 1998; Krueger, Chentsova-Dutton, Markon, Goldberg, & Ormel, 2003; Lahey et al., 2008; Slade & Watson, 2006). Genetic and environmental risks of experiencing these diagnoses are also well-organized by this model (Kendler et al., 2011; Kendler, Prescott, Myers, & Neale, 2003; Lahey, van Hulle, Singh, Waldman, & Rathouz, 2011), and the Internalizing and Externalizing spectra mediate the likelihood of developing additional related diagnoses across the lifespan (Kessler et al., 2011). Some studies also support further divisions within the Internalizing spectrum, in that generalized anxiety disorder, major depression, dysthymia, and post-traumatic stress (PTSD) disorders have been explained by a distress sub-facet of the Internalizing spectrum whereas the co-occurrence of panic and some of the phobic disorders have been explained by a fear sub-facet of the Internalizing spectrum (Eaton et al., in press; Krueger, 1999; Slade & Watson, 2006; Watson, 2009).

The majority of existing structural research has focused on the common mental disorders, omitting diagnoses that are of low prevalence in community samples. Perhaps the most notable omissions have been psychotic and bipolar disorders, as these disorders are relatively rare but are highly burdensome to patients and to public infrastructure (Merikangas et al., 2011; Murray & Lopez, 1996; Saha, Chant, Welham, & McGrath, 2005). There are important exceptions, however, and when low base-rate disorders have been included, they generally suggest that psychosis reflects a distinct liability spectrum. Wolf and colleagues (1988), for example, modeled patterns of comorbidity among Feighner diagnoses (Feighner et al., 1972) in an inpatient unit, including common diagnoses as well as lower base rate diagnoses such as mania and schizophrenia. Principal component analyses identified distinct Internalizing and Externalizing spectra, as well as a third component marked entirely by schizophrenia, suggesting psychosis is best thought of as a distinct major liability spectrum.

Similarly, Kotov et al. (2010) estimated a structural model of psychopathology that included the diagnoses of schizophrenia and schizotypal personality disorder. By using a sample of patients selected for their presentation with psychosis, Kotov and colleagues (2010) were able to bypass the quantitative limitations imposed by the low base rate of these disorders in the community dwelling population. They found that a three-factor model (Internalizing-Externalizing-Psychosis) best fit the data. More recently, Kotov and colleagues (2011) used a large clinical sample which also included richer diagnostic representation of psychotic disorders, but additional less common diagnoses such as histrionic and narcissistic personality disorders, and somatoform disorders. The structural model fit to this data suggested that additional dimensions of Thought Disorder (e.g., Mania, Schizotypal PD), Antagonism (e.g., Histrionic and Narcissistic PD), and Somatoform (e.g., Hypochondriasis) emerge out of this larger group of observed diagnoses. This study used fully diagnosed disorders and assumed continuous latent structures.

One limitation of the majority of the extant research in this area is that it has focused mostly on dichotomous variables that represent diagnosis-level constructs derived from recent DSMs (Brown & Barlow, 2005). Studying the structure of psychopathology based on the DSM-defined categories assumes that each diagnosis represents a single cohesive form of mental disorder. However, the polythetic approach to classification used for most DSM disorders means that this assumption is not accurate in all instances (e.g., Cooper, Balsis, & Zimmerman, 2010). Because of this, some recent structural analyses have begun to focus on psychopathological symptoms rather than diagnoses, albeit such studies are still relatively rare. Such analyses could facilitate a more fine-grained level of analysis than has been pursued in most of the structural literature to date.

Examining individual symptoms as opposed to disorders, Markon (2010) recently presented a symptom level evaluation of the hierarchical structure of psychopathology. In a representative United Kingdom community sample, Markon identified that the co-occurrence of a number of fine-grained symptom indicators were captured by 20 symptom domains. At the next level in the hierarchy, these symptom domains were best modeled by Internalizing, Externalizing, Thought Problems, and Pathological Introversion spectra. Thus, the consistently replicable third factor across Kotov’s (2010 , 2011) and Markon’s analyses is one related to Thought Disorder or Psychosis.

Bipolar pathology, represented in past research as a full disorder (Forbush & Watson, in press), manic episodes (Kotov et al., 2011; Wolf et al., 1988), or manic symptoms (Markon, 2010) has presented a less clear structural picture than psychotic disorders. At times bipolar pathology has joined the Internalizing cluster of disorders (Eaton et al., in press; Forbush & Watson, in press; Kessler et al., 2011; Wolf et al., 1988), and at others it has emerged as a marker of the psychosis spectrum (Kotov et al., 2011) or has remained independent (Markon, 2010). These findings join a complex picture of bipolar disorder’s relationship to both unipolar depression and schizophrenia with past research suggesting it has a shared genetic liability with both in addition to unique variance (Carpenter et al., 2009; Goldberg, Andrews, & Hobbs, 2009). More work is needed in order to better understand where bipolar pathology fits within the hierarchy of psychopathology.

A second limitation of the available structural research is that most of the efforts have exclusively involved fitting continuous latent variables models. For example, although Markon (2010) extended the structural literature by modeling symptom rather than syndromal level data and by including psychosis indices, he did not directly compare continuous models with discrete or hybrid models. The relative fit of categorical latent variable models, compared to continuous latent variable models, requires further exploration. This is an important area for further research because, by only using continuous latent variables, the continuous nature of the latent structure of psychopathology is assumed, as opposed to being empirically evaluated. Moreover, the finding that a continuous model fits the data well in an absolute sense does not rule out the possibility that some aspects of the latent structure of psychopathology may be discrete in nature (Haertel, 1990; Molenaar & von Eye, 1994; Wright, Pincus, & Lenzenweger, 2012). Recently, models with discrete latent variables have been implemented (e.g., Krueger, Markon, Patrick, & Iacono, 2005; Markon & Krueger, 2005; Masyn, Henderson, & Greenbaum, 2010; Muthen, 2008) and thus direct empirical comparisons can now be made about the relative continuity of the distribution of psychopathology in the population without imposing a priori assumptions about its structure.

The Current Study: Toward an Expanded Empirical Model of Psychopathology

In the current study, our aim was to further expand the quantitative approach to classifying psychopathology. Specifically, we modeled psychopathology data from a large community based survey with the aim of better delineating the empirical structure of psychopathology. We begin with finer-grained indicators of psychopathology than have been used in most of the available structural research, and we directly compare the fit of categorical, continuous, and hybrid models to these data, with the aim of converging on an expanded empirically based structural model. We addressed the low base rate of psychotic and bipolar disorders by using symptom level data. With the objective of further refining what is understood about the nature of psychopathology and contributing to a quantitative model to frame future research and clinical efforts, we conducted a series of analyses that (a) evaluated the structure of psychopathology at the lower-order level (symptom level data), (b) tested the relative continuity of the latent structure of psychopathology in the community, and (c) estimated a higher-order quantitative model of psychopathology based on these results.

Methods

Sample and Diagnostic Interview

Data were obtained from the 2007 Australian National Survey of Mental Health and Wellbeing (2007 NSMHWB), a nationally representative epidemiological survey of mental and substance use disorders in the Australian adult population aged between 16 and 85. The survey used a modified version of the World Mental Health Composite International Diagnostic Interview (WMH CIDI; Kessler & Üstun, 2004). The WMH CIDI assesses the lifetime prevalence of mental and substance use disorders. Data were collected from one adult in each selected dwelling via face-to-face computer-assisted personal interviews that were conducted by trained staff at the Australian Bureau of Statistics, the federal statutory authority that conducted the survey. Those residing in special dwellings (e.g., hospitals, nursing homes, jails) and those living in remote and sparsely populated areas of Australia were excluded from the survey. Interviews took, on average, ninety minutes to complete.

The 2007 NSMHWB employed a multistage sampling design. Of the 14,805 individuals initially selected for the survey there were 8,841 fully responding participants (a 60% response rate), who were included in the present analysis (Slade et al., 2009). A purposive non-response follow-up survey was conducted to provide an assessment of the likelihood of non-response bias. A total of 151 initially non-responding individuals provided some demographic and limited psychopathology information (in the form of the Kessler Psychological Distress Scale; Kessler et al., 2002). The results of the non-response follow-up survey indicated that non-response bias appeared to be small at the aggregate level (Slade et al., 2009). A more detailed discussion of the sampling frame and procedures is available elsewhere (Australian Bureau of Statistics, 2009; Slade et al., 2009).

Psychopathology Indicator Construction

General Principles

All indicators of psychopathology analyzed in the current study were derived from survey questions contained in the WMH CIDI. Indicators were generally constructed from questions that represented the key diagnostic constructs (often representing criterion A in DSM-IV or ICD-10) for each mental disorder included in the analysis. In order to maximize the representativeness of the sample (i.e., retain all respondents in the analysis and thus ensure that the sample is representative of the Australian population) items were considered as candidate psychopathology indicators if they satisfied one of two conditions. First, indicators were constructed from items that were asked of all respondents. For example, every respondent was asked three separate questions relating to the core symptom criteria for major depressive episode (lifetime periods of feeling sad, empty or depressed; lifetime periods of feeling discouraged; and lifetime periods of losing interest in usually enjoyable things). Thus, three separate binary (yes/no) coded indicators of depression were constructed from these three items.

Secondly, due to the stem and branch nature of the WMH CIDI interview (i.e., branch items were only asked of those respondents who had endorsed stem items), some items were not asked of all respondents but were still considered to tap the core diagnostic symptoms of a given disorder. These items were combined according to specific rules to produce ordinal indicators. For example, there are two separate items that assess the irritable mood component (DSM-IV criterion A) of manic episode. However, the second of these items (ever feeling so irritable that you started arguments, shouted at people, or hit people) is only asked if the respondent answers positively to the first (lifetime episodes of feeling irritable). Due to the conditional dependence of these two items they could not be included in the analysis as separate indicators of mania. However, to ensure comprehensive content coverage of the mania construct, these items were combined to produce one indicator of mania. Combining these items yielded an indicator of mania for which respondents received either a code of zero (those who do not report lifetime episodes of irritability), a code of one (those who report lifetime irritability but do not report feeling so irritable they started arguments, shouted at people or hit people) or a code of two (those who report lifetime episodes of irritability as well as episodes of excessive irritability). In essence these indicators represent a coarsely graded severity measure of the construct they are intending to tap.

Detailed Description of each Indicator

The following section details each of the indicators that were used in the current study. Where possible the words used to describe the indicators are taken directly from the wording of the WMH CIDI items.

Depression

As described above we constructed three binary (coded 0-1) indicators reflecting lifetime episodes of feeling sad, empty or depressed (DEP1); discouraged (DEP2); and losing interest in usually enjoyable things (DEP3).

Mania

We constructed two indicators of mania. One indicator, MAN1, was binary (coded 0-1), reflecting lifetime episodes of feeling much more excited and full of energy than usual, including increased speed of thought and production of speech, restlessness, and impulsivity. The second, MAN2 was ordinal (coded 0-2). This indicator reflected feeling irritable, grumpy or in a bad mood (coded 1); or feeling so irritable that arguments, shouting, or fights ensued (coded 2). All positive responses coded for these items required that the symptom be present for a period “lasting four days or longer.”

Post-Traumatic Stress

We constructed one binary (PTS1; coded 0-1) indicator. A code of 0 reflected a lifetime absence of all 29 candidate traumatic events or the lifetime presence of trauma but the self-reported absence of PTSD-related reactions. A code of 1 reflected the presence of both trauma and PTSD-related reactions to the trauma.

Generalized Anxiety

One binary (GAD1; coded 0-1) indicator reflected lifetime episodes of either being a “worrier”, being much more nervous or anxious than most other people, or having one month or more of being anxious and worried most days.

Panic Attacks

One binary (PAN1; coded 0-1) indicator reflected lifetime attacks of fear or panic or lifetime attacks of suddenly feeling very uncomfortable; becoming short of breath, dizzy, or nauseous; or feeling you might lose control, die or go crazy.

Social Phobia

One ordinal (SOC1; coded 0-4) indicator was created. A code of 1 reflected the presence of lifetime episodes of feeling very afraid or really, really shy with people or having to do something in front of a group of people; codes of 2–4 reflected the presence of these factors plus one or more of the following: becoming very upset or nervous in these situations; avoiding these situations; and feeling the fear was much stronger than it should have been.

Agoraphobia

One ordinal (AGO1; coded 0-4) indicator was created. A code of 1 reflected the presence of lifetime episodes of feeling afraid of being in crowds, going to public places, traveling by yourself or traveling away from home; codes of 2–4 reflecting the presence of this factor in addition to one or more of the following: becoming very upset or nervous in these situations; staying away from these situations; and feeling the fear was much stronger than it should have been.

Obsessive-Compulsive Disorder (OCD)

Five ordinal (OCD1-5; coded 0-2) indicators reflecting the lifetime presence of five different OCD symptoms. Each symptom was assessed as an obsession and a matching compulsion separately. Thus, the respondent could report the absence of both the obsession and the compulsion (code 0), the presence of the obsession but not the compulsion (or vice versa, coded 1) or the presence of both the obsession and the compulsion (coded 2). The first indicator reflected thoughts of dirt, germs or contamination with or without washing, cleaning or decontamination behaviors. The second indicator reflected concerns of harming others with or without repeated checking behaviors. The third indicator reflected concerns about having things symmetrical, lined up or in order with or without repeated straightening, lining up or arranging behavior. The fourth indicator reflected concerns about having to keep things safe with or without hoarding behavior. Finally, the fifth indicator reflected any other disturbing thought that kept entering the respondents mind with or without any other repetitive behaviors the respondent felt driven to do.

Psychosis

Three ordinal (PSY1-3; coded 0-2) indicators were created to separately reflect ever feeling as though thoughts were being controlled; ever feeling as though things were arranged so as to have a special meaning; and ever feeling in possession of special powers. For each symptom, a code of 0 indicated the absence of the feelings, a code of 1 indicated their presence and a code of 2 further probed the feelings for clinical significance. For example, if a respondent indicated that their thoughts were being controlled they were further probed to determine if this thought control would come about in a way that many people would find hard to believe, for instance through telepathy.

Alcohol Use Disorders

Eight binary (ALC1-8; coded 0-1) indicators of problems associated with alcohol use reflected the presence or absence of a craving or strong desire to use alcohol; needing to drink alcohol in larger amounts than once did; symptoms of withdrawal from alcohol; drinking a lot more than intending to; having an inability to stop or cut down; spending a lot of time drinking; giving up or greatly reducing important activities because of drinking; and continuing to drink with the knowledge of serious physical or psychological problems caused by drinking.

Drug Use Disorders

Seven binary (DRG1-7; coded 0-1) indicators reflected problems associated with the use of any of four drug classes (cannabis, opioids, sedatives and/or stimulants). These indicators reflected the presence or absence of a strong desire to use drugs; needing more of the drugs than once did to get the same effect; symptoms of withdrawal from drugs; using drugs a lot more than intending to; spending a lot of time using drugs; giving up or greatly reducing important activities to use drugs; and continuing to use drugs with the knowledge of serious physical or psychological problems caused by using. The alcohol and drug indicators were constructed from question items asked of respondents who satisfied a pre-determined consumption threshold. Respondents were asked the alcohol symptom question items if they had consumed alcohol at least 3 days per week (regardless of quantity) either in the past 12 months or during a period in their lives when they drank the most. A further group of respondents were asked the alcohol symptom question items if they consumed less frequently than 3 days per week but on days when they did drink they drank at least three drinks per day either in the past 12 months or during a period in their lives when they drank the most. Respondents were asked the drug use symptom question items if they had used prescription medicines more than five times either without the recommendation of a health professional or for any other reason than a health professional said they should be used. Respondents were also asked the drug use symptom question items if they had used illicit drugs more than five times in their lifetime.

Results

Exploratory Factor Analysis of Psychopathology Indicators

The first analytic step was to examine the lower-order structure of the 33 symptom-level indicators using exploratory factor analysis (EFA). All analyses were conducted using the Mplus6.11 software package (Muthén & Muthén, 1998–2010). For the EFA models, all indicators were treated as categorical and accordingly we used the mean and variance adjusted weighted least squares estimator (WLSMV). Additionally, case weights for the 2007 NSMHWB were incorporated in to all analyses. Our goal in the initial EFA was to extract the largest number of interpretable factors, with the aim of delineating the maximum number of interpretable domains for subsequent model comparisons. Table 1 shows the fit indices for the exploratory models that included between two and seven factors. Starting with the model with four factors, fit was generally excellent across all indices (i.e., the comparative fit and Tucker Lewis indices > .95, the root-mean error of approximation < .05, and the standardized root mean square residual < .05; Hu & Bentler, 1999), but continued to improve nominally with increasing numbers of factors. However, in the seven-factor solution, the seventh factor had only a single loading above .30, and was essentially uninterpretable. Therefore, we proceeded with a six-factor solution.

Table 1
Fit Indices for Exploratory Factor Models of Psychopathology Indicators

The factor loadings and correlations for the six-factor solution are given in Table 2 and are readily interpretable. Geomin rotation was specified to provide a desirable balance between factor complexity and interpretability (Sass & Schmitt, 2010). Factor 1 appears to be the distress dimension delineated in past comorbidity analyses, indicated most strongly by the depression indicators. Factor 2 appears to be alcohol problems, with strong loadings from all alcohol indicators. Factor 3 appears be obsessive compulsivity, with strong loadings from all OCD indicators. Factor 4 appears to be drug problems, with strong loadings from all drug indicators. Factor 5 appears to be the fear dimension delineated in past comorbidity analyses. Factor 6 appears to be psychotic experiences, with strong loadings from all psychosis indicators.

Table 2
Exploratory Factor Loadings of the Psychopathology Indicators

Latent Modeling of Psychopathology Syndromes

We next compared the fit of categorical, hybrid, and continuous latent models of the psychopathology syndromes delineated in the six-factor EFA. In these analyses, we assigned the indicators to the domains on which they had their highest loadings. There were two exceptions to this rule. First, the PTS1 indicator had no loading of .30 or greater on any of the six factors and was therefore not included in these or subsequent models because it appears not to be even a moderately strong indicator of any of the broader, estimated domain-level constructs. The potential implications of the finding that post-traumatic distress does not fit well into the psychopathology structure delineated in Table 2 are interesting, and we return to these findings in the discussion. Second, PAN1, which had a secondary loading of .30 on the fear domain, was included in the fear analyses so as to avoid difficulties associated with only two indicators of the domain.

Table 3 catalogues the model details and resulting fit statistics associated with the latent class, non-parametric latent trait, and dimensional latent trait models for each of the six identified domains. Figure 1 offers conceptual diagrams of the distributions associated with each of the different types of estimated models. Latent class analysis models the pattern of individual responses to the manifest variables (i.e., the symptom criteria) by estimating distinct “classes” or categories of individuals (see panel A of Figure 1) which differ in their probability of symptom endorsement (Collins & Lanza, 2010). These models represent one pole of a “dimensional-categorical spectrum” of models (see Masyn et al., 2010), and estimate classes of individuals whose response probabilities, when mixed together, give rise to the observed pattern of scores. For each domain we estimated LCA models starting with two classes and ranging until there was a decrement in model fit (i.e., the BIC increased) or the model became non-identified because the available points of information were exhausted (Collins & Lanza, 2010). If any these models were the best fitting within a given domain, this would suggest that there were distinct groups of individuals who differ in their probability of symptom endorsement. At the other pole of this spectrum of models lie the latent trait models (see Panel C in Figure 1). Latent trait models estimate the observed pattern of scores using continuous latent dimensions. For each of the six domains identified in Table 1, we estimated a model with a single latent dimension (i.e., a 1-factor latent trait model). If this model were the best fitting within a given domain, it would suggest that the liability of symptom endorsement was distributed continuously along a latent dimension associated with that domain.

Figure 1
Graphical depiction of hypothetical distributions associated with the estimated model types.
Table 3
Fit Index Values for Latent Class and Latent Trait Models of Psychopathology Indicators

Between fully discrete latent class and continuous latent trait models lie a large group of hybrid models, which are conceptual and mathematical blends of latent classes and traits. Often referred to as factor mixture models, there is wide variability in the degree to which models employ dimensional and categorical latent variables.1 Here we chose the conceptual midpoint between the fully categorical latent class, and the fully dimensional latent trait models, and selected a model that is conceptually both dimensional and categorical at the same time. Referred to as either a located latent class model (LLCM) or non-parametric factor model (NP-FA; Masyn et al., 2010; see also Clark et al., 2012), in this structure the variability in scores is modeled as discrete densities of individuals with the same score along a latent dimension. In other words, this model estimates distinct groups of individuals who possess the same score along a shared latent dimension. Models were estimated such that the latent trait was invariant across classes and a unit score separated the classes along the latent trait. Similar to the latent class models, NP-FA models were estimated with increasing classes until BIC increased. If these models were the best fitting, it would suggest that for a given domain there is a latent dimension along which exist discrete groups of individuals who share the same level of symptoms. Examination of the BIC values in Table 3 shows that, for every domain, the continuous latent trait model fit better than either the latent class or NP-FA models. The conclusion is that variation in each of these domains is better understood as continuous distributions of psychopathology as opposed to categorically discrete phenotypes.

Higher Order Structure of Psychopathology Domains

Comparison of the ordered latent class and the latent trait models indicated the existence of six dimensional domains of psychopathological variation in our data, which we labeled distress, obsessive compulsivity, fear, alcohol problems, drug problems, and psychotic experiences. We next proceeded to fit confirmatory factor models to compare alternative conceptualizations of the higher order structure of these six continuous domains, using the robust maximum likelihood (MLR) estimator. To index the domains of distress, obsessive compulsivity, fear, alcohol problems, and drug problems in these models, we estimated latent trait scores (i.e., factor scores) for each domain that were then used as the observed variables in the hierarchical model. Psychosis was indicated directly by PSY1, PSY2, and PSY3 (as opposed to a single variable representing the latent trait of psychosis) because we aimed to test the hypothesis that psychosis may form a separate higher order domain, and thus more than one indicator of a potential higher order psychotic factor was needed for model identification.2

We began with a one-factor model; although very unlikely to fit best, given past research, the fit of this model provided a baseline for comparison with other models. We then fit two, two-factor Internalizing-Externalizing models. In the first of these models, internalizing was indicated by distress, obsessive compulsivity, and fear; externalizing was indicated by alcohol problems and drug problems; and psychotic experiences loaded on internalizing. The second of these models was the same as the first, but psychotic experiences loaded on the externalizing factor. The final hierarchical model conceptualized internalizing, externalizing, and psychosis as three correlated higher order latent traits.

Table 4 shows the fit indices of the hierarchical analyses. Both Internalizing-Externalizing models demonstrated gains in model fit in comparison to the baseline model, as can be seen by the decreased BIC values. Nonetheless, the three factor hierarchical model that distinguished psychotic experiences from Internalizing and Externalizing domains provided the best fit of the occurrence of psychopathology in the community. In this final model the correlations among the higher-order factors were as follows: .48 (Externalizing with Internalizing), .59 (Psychosis with Internalizing), and .36 (Psychosis with Externalizing). See Figure 2 for a depiction of the full model, portions of which were estimated separately (i.e., the loadings of the observed symptoms on the latent Distress, Fear, OCD, Alcohol, and Drug dimensions).

Figure 2
The best fitting higher-order model of psychopathology. Note that portions of the model were estimated in isolation. Coefficients represent standardized parameter estimates. DEP = Depression criteria; MAN = Manic episode criteria; GAD = Generalized anxiety ...
Table 4
Fit Index Values for Confirmatory Factor Models of Psychopathology Domains

Discussion

There has been longstanding dissatisfaction with the manner in which descriptive psychiatry summarizes clinically observable phenomena (e.g., Leary, 1957; Menninger, Ellenberger, Pruyser, & Mayman, 1959), a dissatisfaction echoed in recent literature underlining how descriptive categories have not led to the identification of correspondingly categorical etiologies (e.g., Jablensky, 2010; Rutter, 2011). Moving beyond current categorical classification systems must be accomplished in a quantitatively defensible way. Here we estimated a series of models to evaluate the structure of psychopathology symptoms in a large representative community sample with a view towards developing an expanded quantitative model of psychopathology. A primary goal of this study was to test the a priori assumption of the current diagnostic nomenclature that treats diagnoses as distinct from sub-threshold presentations of diagnostic criteria. Additionally, we accounted for the well-known heterogeneity of the DSM-defined syndromes by modeling what were primarily symptom level data as opposed to disorder level data. We found that six domains of psychopathology—Distress, Obsessive Compulsivity, Fear, Alcohol Problems, Drug Problems, and Psychotic Experiences—capture the observed patterns of co-occurrence of these symptoms in the community. Contrary to the a priori DSM assumption, these data show that the six domains of psychopathology are continuously distributed in the community. Furthermore, when these domains were examined together, the Distress, Obsessive Compulsivity and Fear factors emerged as facets of the higher-order Internalizing spectrum; Alcohol and Drug Problems emerged as facets of the higher-order Externalizing Domain; and Psychotic Experiences were found to mark a psychopathology domain that is separable from the well-established Internalizing and Externalizing spectra.

Working Toward an Empirical Model of Adult Psychopathology

These findings advance our understanding of the structure of psychopathology in a number of specific ways. First, this research directly compared the fit of continuous and categorical models of the latent domains of psychopathology and thus directly tested the structure of psychopathology in the community without making a priori assumptions about its distribution. Second, by analyzing primarily symptom rather than syndromal level indicators, we found in the EFA that panic attacks demonstrated a distinct pattern of loadings from the agoraphobia component of panic disorder. Additionally, low loadings were observed for some items, notably the post-traumatic stress indicator unexpectedly did not index any of the six latent domains of psychopathology strongly. Similarly, the elevated mood component of mania (i.e., MAN1 in our models) did not demonstrate a large loading on any factor, although it met the threshold (i.e., .30) for inclusion in subsequent models.

Continuity and Discontinuity in Psychopathology

What distinguishes these analyses from much of the prior structural modeling of psychopathology is the use of a range of models, which are associated with continuous dimensions (latent trait models), discrete groupings along a dimension (NP-FA), and discrete classes of individuals (LCA). Rather than assume that the domains that marked the clusters of symptoms/disorders examined here are dimensional, we tested this by comparing the fit across models of three types. We selected the poles of what can be considered a “dimensional-categorical” spectrum of models (Masyn et al., 2010), and the conceptual midpoint in an effort to stringently test what is often taken as a given—namely, that psychopathology of various types is distributed either continuously or discretely. Ultimately, across the six domains, in each case fit favored the latent trait models, although this could not have been assumed beforehand.

The use of the intermediary models that are a blend of dimensional and categorical models is notable because they allow for discrete or discontinuous aspects in what is otherwise a latent dimensional space. This provides a stronger test of the continuity of the latent domains as opposed to just testing categorical vs. dimensional models. In looking at the models in Table 3, the best fitting NP-FA models tended to have a relatively large number of discrete groupings (e.g., 5–7 values). Had any of those been the best fitting models, it would still have prompted the question: How many discrete groupings are allowed before a latent dimension effectively becomes continuous? It would be hard to argue that a large number of values are more parsimonious and conceptually more accessible than a continuous dimension. Thus, both empirically (in terms of model fit) and conceptually, the results favored continuous dimensions of the six domains of psychopathology identified here.

Higher-Order Dimensions of Psychopathology

After identifying the domains of psychopathology represented in these data using EFA, and testing the latent distributions of those domains, we then examined a number of possible higher-order structural models of psychopathology. Although we tested a one-factor model in an effort to be comprehensive, the main goal of these analyses was to attempt to place the psychosis domain within the well-replicated Internalizing-Externalizing structure of psychopathology (Achenbach, 1966; Kendler et al., 2011; Krueger, 1999; Krueger et al., 1998; Lahey et al., 2008; Slade & Watson, 2006). Empirically, the psychosis domain appears to exist as a separate and stand-alone dimension as opposed to being subsumed as an indicator of the Internalizing or Externalizing domains. If we take these results in the context of additional studies which have used a broader selection of indicators (Kotov et al., 2010; Kotov et al., 2011; Markon, 2010; Wolf et al., 1988), the domain which consistently replicates across these recent studies and the current study is the psychosis/thought disorder domain. Given the highly replicable nature of the psychosis domain, across multiple epidemiological and clinical samples, we feel confident that the empirical structure of psychopathology can now be considered to be comprised of, at the minimum, Internalizing, Externalizing, and Psychosis.

This structure accords well with and is buttressed by empirical work examining the familial patterning and molecular genetics of psychosis (Carpenter et al., 2009; Craddock & Owen, 2010). Psychosis can emerge as a feature of the current clinical categories of mood disorders (e.g., bipolor and unipolar depression), thought disorders, and the interstitial schizoaffective disorders. In turn, these disorders have been found to have both shared and unique genetic variance, suggesting that there are both common and specific etiological pathways associated with the specific disorders (Carpenter et al., 2009; Craddock & Owen, 2010; Kendler & Gardner, 1997; Lichtenstein et al., 2009). Complementary psychometric work, conducted with a large group of probands with psychosis (including schizophrenia, schizoafective, and bipolar disorders), found overlapping and contnuous distributions of affective and psychotic features across diagnoses (Keshavan et al., 2011). However, existing work in this domain has not been carried out with sufficient phenomenological or temporal resolution to tease apart which aspects of each clinical disorder are associated with the shared, as opposed to the specific, liability (Carpenter et al., 2009). Future work should continue to focus on the more fine grained expression of symptoms, as opposed to diagnostic categories, even as the current and past work supports the existence of a dimension of psychotic experiences in the population (cf. Lenzenweger, 2010 for both convergent and divergent views on this issue).

The current findings relate to broader conceptions of the hierarchical nature of the structure of psychopathology and personality. Internalizing and Externalizing spectra emerged from the analysis of common psychiatric diagnoses in both child and adult samples (e.g., Achenbach, 1966; Krueger, 1999) and are now well-replicated many times over as reviewed above. Hierarchies of functioning have also long been an interest to personality scientists (e.g., Digman, 1997; Guildford, 1975; Markon, Krueger, & Watson, 2005). Personality structure merits mention for at least two reasons. First, a large body of research has convincingly demonstrated that disorders of all types (i.e., both PD and syndrome disorders) are related to basic personality traits (Andersen & Bienvenu, 2011; Kotov et al., 2010). Second, especially at the higher-order levels, the structure of abnormal personality bears a striking resemblance to that of mental disorders (see e.g., Wright et al., in press).

An early model that outlined a structure that would seem to bridge the psychopathology and personality traditions is the Personality Psychopathology Five (PSY-5; Harkness, Finn, McNulty, & Shields, 2012; Harkness & Mcnulty, 1994) which adopts a now well-know five-factor structure common to both basic and pathological personality traits (Markon et al., 2005). The PSY-5 constructs can be assessed with the Minnesota Multiphasic Personality Inventory–2–Restructured Form (MMPI-2-RF; Ben-Porath & Tellegen) and similar models have since emerged across other measures (e.g., Tacket et al., 2008; Watson et al., 2008; Wright et al., in press; see Harkness et al., 2012).

Additionally, the MMPI-2-RF has adopted a higher-order structure, such that at the most general level exist domains of Emotional/Internalizing Dysfunction, Behavioral/Externalizing Dysfunction, and Thought Dysfunction. Conceptually, Emotional/Internalizing Dysfunction combines PSY-5 Negative Emotionality and (low) Extraversion, Behavioral/Externalizing Dysfunction combines PSY-5 Aggressiveness and (low) Constraint, and Thougth Dysfunction as a domain is related to PSY-5 Psychoticism. Hierarchies of this type, and those that have emerged in analyses of other measures (e.g., Markon et al., 2005; Wright et al., in press) suggest that there is undoubtedly value in exploring potentially broader models, that simultaneously include more disorders and more levels of a putative hierarchy. Initial efforts suggest that more factors will likely emerge (e.g., Kotov et al., 2011; Markon, 2010), but to date the only factor to replicate beyond Internalizing and Externalizing is one related to psychosis.

As a practical matter, the estimation of hierarchical factor models will be contingent on the exact admixture of variables involved in the analysis. Guilford (1975) pointed out that this will influence not only which variables combine to form domains, but also whether the factors that emerge in a given analysis will be from the equal or similar levels of generality. In this study, we can observe some variability in the level of generality that emerge based on the available variables. For example, separate domains emerge for alcohol use, substance use, and distress, fear, and OCD, which each serve as lower order markers for the higher-order domains of Externalizing and Internalizing. At the same time, the psychosis items load directly on the higher-order domain without an intermediary level differentiating between psychotic experiences. If the sample included content related to perceptiual abberations (e.g., hallucinations) in addition to the delusional content in the items, additional levels within the psychosis domain may have emerged. The key is not to assume that all factors that emerge from observed markers will exist at the same level of generality, leading to potentially artificial hierarchies. We avoid this by testing the higher-order structure and allowing for a model with observed indicators loading directly on a higher order factor (see also Forbush & Watson, in press for an example of this).

Indicators with low loadings

An intriguing aspect of our findings was that two indicators appear to evidence a quite modest relationship with the other indices of psychopathology. This may suggest that these indicators are not well accounted for within this structure, and may instead define additional, separable domains of psychopathology. Alternatively, these indicators may not have coalesced into the factors in this analysis because there may not have been enough symptom indicators to define these additional domains. We turn now to explore these findings.

First, the PTSD indicator was found not to co-occur extensively with the other measured symptoms. Previous structural analyses of the common mental diagnoses that have included PTSD have found that the diagnosis indexes the Internalizing domain of psychopathology (Cox, Clara, & Enns, 2002; Slade & Watson, 2006). The current findings may be driven in large part by the fact that PTSD requires an exogenous (i.e., external) event to have either occurred to the person or that they witnessed, whereas other diagnostic indicators do not share this requirement and are primarily related to what are ostensibly endogenous cognitive or affective phenomenology. This finding may also reflect the multidimensional structure of the PTSD diagnosis (Carragher, Mills, Slade, Teesson, & Silove, 2010; King, Leskin, King, & Weathers, 1998; Simms, Watson, & Doebbeling, 2002), in that using a single indicator may have failed to capture the different relationships between the distinguishable symptom dimensions of PTSD. For instance, Simms and colleagues (2002) have shown that PTSD symptoms mark four factors, and Watson (2005) found that the hyper-arousal and numbing symptoms of PTSD are more closely associated with the distress facet of Internalizing than the intrusive and avoidance symptoms of the diagnosis. Second, we found that the irritability facet of mania was an indicator of Internalizing psychopathology to a much stronger degree than was the expansive mood facet of mania. This suggests that the criteria for a manic episode in the DSM may conflate two constructs (expansive mood and irritability) that are empirically separable. Consistent with this notion are the results from structural analyses of clinical screeners for hypomania and bipolar disorder. For instance, in clinical and community samples the Hypomanic Checklist and the Mood Disorder Questionnaire are composed of distinguishable irritability and psychomotor activation (i.e., closely related to the expansive mood facet of mania) factors (Angst et al., 2005; Benazzi & Akiskal, 2003; Meyer et al., 2007). The EPIMAN-II study, a nationally representative sample of 1,090 manic psychiatric patients, also confirmed the distinction between the expansive mood and irritability components of mania (Hantouche, Akiskal, Azorin, Chêtenet-Duchêne, & Lancrenon, 2006). These data thus lend partial support to the distinction between “hypomania” and “depressive or anxious mania” (Cassidy, Pieper, & Carroll, 2001).

The only moderate loading of MAN1 (i.e., elevated mood and energy) on the distress domain and additional but more modest loadings on other domains is consistent with prior research. As mentioned above, very little structural work has involved symptoms or episodes of mania or full bipolar disorder. Among this work, the markers have joined psychosis once (Kotov et al., 2011), Internalizing disorders three times (Eaton et al., in press; Forbush & Watson, in press; Wolf et al., 1988), and found no relationship in other analyses (Markon, 2010). Our findings are somewhere in between those reported by Forbush and Watson (in press) and Markon (2010). Secondary loadings were not found with psychosis items, as might be predicted based on reviews of other analytic approaches (e.g., Goldberg et al., 2009). We note that psychotic experiences are not a requirement for a manic episode, and with low base rate pathology of this type, the association with other variables will be highly influenced by the particular representation in the relatively fewer individuals in the sample. Alternatively, it could be argued that despite the fact that MAN1 and MAN2 had as a requirement of symptom persistence for 4 days along with characteristic features of manic episodes, they may be capturing more general psychological processes, not mania per se. This argument is more tenable for MAN2, as it relates to general irritability, as that is a diagnostic criterion or associated feature of a number of the distress disorders. However, MAN 1 is clearly not merely “positive emotionality” as it has a positive loading on the distress factor. Nevertheless, it remains an open question how dimensional or discrete individual manic episodes are (e.g., manic episode vs. hypomanic episode vs. normal variation elevated mood), and in turn, how much this contributes to patterns observed in epidemiological samplings such as the data analyzed here.

Limitations

As with all studies, there are a number of limitations to consider when interpreting these results. The most notable limitation is that not all diagnoses and their criteria were sampled at the same degree of fidelity, nor were they all amenable to being transformed in to symptom criteria that were equivalent in number or structure (e.g., dichotomous vs. polytimous items). In some cases, such as GAD, or social phobia, the resulting criteria were reduced to essentially one item. However, using the rules employed here allowed us to retain the cardinal features of each disorder as separate markers, which are the core discriminable material, and what is important for these structural models. Relatedly, not all diagnoses from the DSM were sampled. One area where this limitation was evident was in the fear domain. The fear domain, with only three dichotomous indicators, only contained enough information to test 2-class and 2-value NP-FA models. Ideally more indicators would be available to tests models with a larger range of classes and values. Nonetheless, we were able to extract a distinct fear spectrum, and the results were consistent with all other models allowing for more confidence in the results. Additionally, there was no marker available for antisocial behavior, one of the traditional markers of the Externalizing domain. Thus, the manner in which this spectrum is represented here likely emphasizes features associated with disconstraint as opposed to antagonism. Although this may have affected the results in some unforeseen way, the structure of Externalizing has been highly replicable across previous studies, and it has generally been characterized by strong loadings from substance use and conduct problems/antisocial behaviors, and represents a very cohesive domain. Therefore, our markers of alcohol and drug problems are most likely sufficient to mark Externalizing as a spectrum here.

We also note that the sampling of items associated with the psychosis domain were limited, as there were only three, and they did not contain items associated with hallucinations or disorganization common to many of the more severe psychotic disorders. As such, we echo Carpenter and colleagues’ (2009) call for more fine-grained research in the psychotic domains to establish whether the phenotypic heterogeneity in the psychotic disorders will result in the emergence of additional or sub-factors of this domain. Undoubtedly, Internalizing, Externalizing, and Psychosis are not the only spectra that will emerge from structural models that contain a broader sampling of disorders as evidenced by other work that has found spectra associated with antagonism, somatization, or pathological introversion (Kotov et al., 2011). The emergence of some of these factors (i.e., antagonism, pathological introversion) appears contingent on the inclusion of markers of personality disorders within the model (Roysamb et al., 2011). Although compelling, those domains have yet to be replicated. The current sample was not assessed for personality disorders and somatization disorders, which precludes the evaluation of these more elaborated structures here.

Footnotes

1Latent class models and latent trait models can be understood to represent special cases of factor mixture models.

2We used factor score estimates of the EFA generated lower-order domains as markers for the Internalizing and Externalizing domains for these analyses, as opposed to working from the level of the primary indicators to delineate a higher-order factor model with two factor levels (i.e., the domains in Table 2 represented as factors below the level of factors representing the constructs in Table 4). This was because formal maximum likelihood (ML) factor models fit directly to ordinal variables are limited in the number of estimable factors because of current computational limitations. An alternative option would be to fit a higher order factor model via WLS estimation, which is adequate for initial exploratory modeling (see Table 2), but not ideally suited to fine-grained confirmatory model comparisons because sensitive fit indices such as BIC are technically not defined for WLS estimators.

Contributor Information

Aidan G.C. Wright, Western Psychiatric Institute and Clinic.

Robert F. Krueger, University of Minnesota.

Megan J. Hobbs, University of New South Wales.

Kristian E. Markon, University of Iowa.

Nicholas R. Eaton, University of Minnesota.

Tim Slade, University of New South Wales.

References

  • Achenbach TM. The classification of children’s psychiatric symptoms: a factor–analytic study. Psychological Monographs. 1966;80(615) [PubMed]
  • Andersen AM, Bienvenu OJ. Personality and psychopathology. International Review of Psychiatry. 2011;23:234–247. [PubMed]
  • Andrews G, Goldberg DP, Krueger RF, Carpenter WT, Hyman SE, Sachdev PS, et al. Exploring the feasibility of a meta-structure for DSM-V and ICD-11: Could it improve utility and validity? Psychological Medicine. 2009;39:1993–2000. [PubMed]
  • Angst J, Adolfsson R, Benazzi F, Gamma A, Hantouche E, Meyer TD, et al. The HCL-32: towards a self-assessment tool for hypomanic symptoms in outpatients. Journal of Affective Disorders. 2005;88:217–233. [PubMed]
  • Australian Bureau of Statistics. National Survey of Mental Health and Wellbeing: User’s guide, 2007. Canberra, Australia: Australian Bureau of Statistics; 2009.
  • Benazzi F, Akiskal HS. The dual factor structure of self-rated MDQ hypomania: energized-activity versus irritable-thought racing. Journal of Affective Disorders. 2003;73:59–64. [PubMed]
  • Blashfield RK. Classification of psychopathology. New York, NY: Plenum; 1984.
  • Brown TA, Barlow DH. A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: Implications for assessment and treatment. Psychological Assessment. 2009;21:256–271. [PMC free article] [PubMed]
  • Brown TA, Di Nardo PA, Lehman CL, Campbell LA. Reliability of DSM-IV anxiety and mood disorders: Implications for the classification of emotional disorders. Journal of Abnormal Psychology. 2001;110:49–58. [PubMed]
  • Carpenter WT, Bustillo JR, Thaker GK, van Os J, Krueger RF, Green MJ. The psychoses: Cluster 3 of the proposed meta-structure for DSM-V and ICD-11. Psychological Medicine. 2009;39:2025–2042. [PubMed]
  • Cassidy F, Pieper CF, Carroll BJ. Subtypes of mania determined by grade of membership analysis. Neuropsychopharmacology. 2001;25:373–383. [PubMed]
  • Clark SL, Muthén B, Kaprio J, D’Onofrio BM, Viken R, Rose RJ, Smalley SL. Models and strategies for factor mixture analysis: Two examples concerning the structure underlying psychological disorders. 2012 Manuscript submitted for publication available at http//statmodel.com/papers.shtml. [PMC free article] [PubMed]
  • Clark LA, Watson D. Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology. 1991;100:316–336. [PubMed]
  • Collins LM, Lanza ST. Latent class and latent transition analysis: with applications in the social, behavioral and health sciences. Hoboken, NJ: Wiley; 2010.
  • Cooper LD, Balsis S, Zimmerman M. Challenges associated with a polythetic diagnostic system: Criteria combintations in the personality disorders. Journal of Abnormal Psychology. 2010;119:886–895. [PubMed]
  • Cox BJ, Clara IP, Enns MW. Posttraumatic stress disorder and the structure of common mental disorders. Depression and Anxiety. 2002;15:168–171. [PubMed]
  • Craddock N, Owen M. Molecular genetics and the Kraepelinian dichotomy: One disorder, two disorders, or do we need to start thinking afresh? Psychiatric Annals. 2010;40(2):88–91.
  • Digman JM. Higher-order factors of the Big Five. Journal of Personality and Social Psychology. 1997;73:1246–1256. [PubMed]
  • Eaton NR, Keyes KM, Krueger RF, Balsis S, Skodol AE, Markon KE, Grant BF, Hasin DS. An invariant dimensional liability model of gender differences in mental disorder prevalence: Evidence from a national sample. Journal of Abnormal Psychology. 121:282–288. (in press) [PMC free article] [PubMed]
  • Eaton NR, Krueger RF, Markon KE, Keyes KM, Skodol AE, Wall M, Hasin DS, Grant BF. The structure and predictive validity of the internalizing disorders. Journal of Abnormal Psychology (in press) [PMC free article] [PubMed]
  • Eaton NR, Krueger RF, Oltmanns TF. The structure and long-term stability of the internalizing spectrum of personality and psychopathology across the lifespan. Psychology and Aging. 26:987–993. (in press) [PMC free article] [PubMed]
  • Feighner JP, Robins E, Guze SB, Woodruff RA, Jr, Winokur G, Munoz R. Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry. 1972;26:57–63. [PubMed]
  • Forbush KT, Watson D. The structure of common and uncommon mental disorders. Psychological Medicine. doi: 10.1017/S0033291712001092. (in press) [PubMed] [Cross Ref]
  • Goldberg DP, Andrews G, Hobbs MJ. Psychological Medicine. 2009. Where should bipolar disorder appear in the meta-structure? [PubMed]
  • Carragher N, Mills K, Slade T, Teesson M, Silove D. Factor structure of posttraumatic stress disorder symptoms in the Australian general population. Journal of Anxiety Disorders. 2010;24:520–527. [PubMed]
  • Guilford JP. Factors and factors of personality. Psychological Bulletin. 1975;82:802–814. [PubMed]
  • Haertel EH. Continuous and discrete latent structure models for item response data. Psychometrika. 1990;55:477–494.
  • Hantouche E, Akiskal HS, Azorin JM, Chêtenet-Duchêne L, Lancrenon S. Clinical and psychometric characterization of depression in mixed mania: A report from the French National Cohort of 1090 manic patients. Journal of Affective Disorders. 2006;96:225–232. [PubMed]
  • Harkness AR, Finn JA, McNulty JL, Shields SM. The Personality Psychopathology—Five (PSY–5): Recent constructive replication and assessment literature review. Psychological Assessment. 2012;24:432–443. [PubMed]
  • Harkness AR, McNulty JL. The Personality Psychopathology Five (PSY–5): Issue from the pages of a diagnostic manual instead of a dictionary. In: Strack S, Lorr M, editors. Differentiating normal and abnormal personality. New York, NY: Springer; 1994. pp. 291–315.
  • Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55.
  • Jablensky A. Psychiatry in crisis? Back to fundamentals. World Psychiatry: Official Journal Of The World Psychiatric Association. 2010;9(1):29. [PMC free article] [PubMed]
  • Kendler KS, Aggen SH, Knudsen GP, Røysamb E, Neale MC, Reichborn-Kjeennerud T. The structure of genetic and environmental risk factors for syndromal and subsyndromal common DSM-IV Axis I and Axis II disorders. American Journal of Psychiatry. 2011;168:29–39. [PMC free article] [PubMed]
  • Kendler KS, Gardner CO. The risk for psychiatric disorders in relatives of schizophrenic and control probands: a comparison of three independent studies. Psychological Medicine. 1997;27:411–419. [PubMed]
  • Kendler KS, Prescott CA, Myers JM, Neale MC. The structure of genetic and environmental risk factors for common psychiatric and substance use disorders in men and women. Archives of General Psychiatry. 2003;60:929–937. [PubMed]
  • Keshavan MS, Morris DW, Sweeney JA, Pearlson G, Thaker G, Seidman LJ, Eack SM, Tamminga C. A dimensional approach to the psychosis spectrum between bipolar disorder and schizophrenia: The Schizo-Bipolar Scale. Schizophrenia Research. 2011:250–254. [PMC free article] [PubMed]
  • Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, Walters EE, Zaslavsky AM. Short screening scales to monitor population prevalences and trends in nonspecific psychological distress. Psychological Medicine. 2002;32:959–976. [PubMed]
  • Kessler RC, Cox AD, Green JG, Ormel J, McLaughlin KA, Merikangas KR, et al. The effects of latent variables in the development of comorbidity among common mental disorders. Depression and Anxiety. 2011;28:29–39. [PMC free article] [PubMed]
  • Kessler RC, Üstun TB. The World Mental health (WMH) survey initiative version of the World Health Organization Composite International Diagnostic Interview (CIDI) International Journal of Methods in Psychiatric Research. 2004;13:93–117. [PubMed]
  • King DW, Leskin GA, King LA, Weathers FW. Confirmatory factor analysis of the clinician-administered PTSD scale: evidence for the dimensionality of posttraumatic stress disorder. Psychological Assessment. 1998;10:90–96.
  • Kotov R, Chang SW, Fochtmann LJ, Mojtabai R, Carlson GA, Sedler MA, Bromet EJ. Schizophrenia in the internalizing-externalizing framework: A third-dimension? Schizophrenia Bulletin. 2010;37:1168–1178. [PMC free article] [PubMed]
  • Kotov R, Gamez W, Schmidt F, Watson D. Linking ‘big’ personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychological Bulletin. 2010;136:768–821. [PubMed]
  • Kotov R, Ruggero CJ, Krueger RF, Watson D, Yuan Q, Zimmerman M. New dimensions in the quantatative classification of mental illness. Archives of General Psychiatry. 2011;68:1003–1011. [PubMed]
  • Kramer MD, Krueger RF, Hicks BM. The role of internalizing and externalizing liability factors in accounting for gender differences in the prevalence of common psychopathological syndromes. Psychological Medicine. 2008;38:51–61. [PubMed]
  • Krueger RF. The structure of common mental disorders. Archives of General Psychiatry. 1999;56:921–926. [PubMed]
  • Krueger RF, Capsi A, Moffitt TE, Silva PA. The structure and stability of common mental disorders (DSM-III-R): A longitudinal-epidemiological study. Journal of Abnormal Psychology. 1998;107:216–227. [PubMed]
  • Krueger RF, Chentsova-Dutton YE, Markon KE, Goldberg DP, Ormel J. A cross-cultural study of the structure of comorbidity among common psychopathological syndromes in the general health care setting. Journal of Abnormal Psychology. 2003;107:216–227. [PubMed]
  • Krueger RF, Markon KE, Patrick CJ, Iacono WG. Externalizing psychopathology in adulthood: A dimensional-spectrum conceptualization and its implications for DSM-V. Journal of Abnormal Psychology. 2005;114:537–550. [PMC free article] [PubMed]
  • Krueger RF, Markon KE, Patrick CJ, Benning SD, Kramer MD. Linking antisocial behavior, substance use, and personality: an integrative quantitative model of the adult externalizing spectrum. Journal of Abnormal Psychology. 2007;116:645–666. [PMC free article] [PubMed]
  • Lahey BB, Rathouz PJ, van Hulle C, Urbano RC, Krueger RF, Applegate B, et al. Testing structural models of DSM-IV symptoms of common forms of child and adolescent psychopathology. Journal of Abnormal Child Psychology. 2008;36:187–206. [PubMed]
  • Lahey BB, van Hulle C, Singh AL, Waldman ID, Rathouz PJ. Higher-order genetic and environmental structure of prevalent forms of child and adolescent psychopathology. Archives of General Psychiatry. 2011;68:181–189. [PMC free article] [PubMed]
  • Leary T. Interpersonal diagnosis of personality. New York: Ronald Press; 1957.
  • Lenzenweger MF. Schizotypy and schizophrenia: The view from experimental psychopathology. New York, NY: Guilford Press; 2010.
  • Lichtenstein P, Yip BH, Bjork C, Pawitan Y, Cannon TD, Sullivan PF, Hultman CM. Common genetic determinants of schizophrenia and bipolar disorder in Swedish families: A population-based study. Lancet. 2009;373:234–239. [PMC free article] [PubMed]
  • Lilienfeld SO, Waldman ID, Israel AC. A critical examination of the term and concept of ‘comorbidity’ in psychopathology research. Clinical Psychology Science and Practice. 1994;1:71–83.
  • Lubke GH, Muthén BO. Investigating population heterogeneity with factor mixture models. Psychological Methods. 2005;10:21–39. [PubMed]
  • Markon KE. Modeling psychopathology structure: A symptom-level analysis of Axis I and II disorders. Psychological Medicine. 2010;40:273–288. [PubMed]
  • Markon KE, Krueger RF. Categorical and continuous models of liability to externalizing disorders: A direct comparison in NESARC. Archives of General Psychiatry. 2005;62:1352–1359. [PMC free article] [PubMed]
  • Markon KE, Krueger RF. Information-theoretic latent distribution modeling: Distinguishing discrete and continuous latent variable models. Psychological Methods. 2006;11:228–243. [PubMed]
  • Masyn KE, Henderson CH, Greenbaum PE. Exploring the latent structures of psychological constructs in social development using the dimensional-categorical spectrum. Social Development. 2010;19:470–493. [PMC free article] [PubMed]
  • Menninger K, Ellenberger H, Pruyser P, Mayman M. The unitary concept of mental illness. Bulletin of the Menninger Clinic. 1959;22:4–12. [PubMed]
  • Merikangas KR, Jin R, He JP, Kessler RC, Lee S, Sampson NA, et al. Prevalence and correlates of bipolar spectrum disorder in the World Mental Health Survey Initiative. Archives of General Psychiatry. 2011;68:241. [PMC free article] [PubMed]
  • Meyer TD, Hammelstein P, Nilsson LG, Skeppar P, Adolfsson R, Angst J. The Hypomania Checklist (HCL-32): its factorial structure and association of indices of impairment in German and Swedish nonclinical samples. Comprehesive Psychiatry. 2007;48:79–87. [PubMed]
  • Molenaar PCM, von Eye A. On the arbitrary nature of latent variables. In: von Eye A, Clogg CC, editors. Latent variables analysis: Applications for developmental research. Thousand Oaks, CA: Sage; 1994. pp. 226–242.
  • Murray CJ, Lopez AD, editors. The global burden of disease: A comprehensive assessment of mortality and disability from diseases, injuries, and risk factors in 1990 and projected to 2020. Boston: Harvard School of Public Health; 1996.
  • Muthén B. Latent variable hybrids. In: Hancock GR, Samuelson KM, editors. Advances in latent variable mixture models. Charlotte, NC: Information Age Publishing; 2008. pp. 1–24.
  • Muthén L, Muthén BO. Mplus users’ guide. 6. Los Angeles, CA: Muthén & Muthén; 1998–2010.
  • Regier DA, Narrow WE, Kuhl WE, Kupfer DJ. The conceptual development of DSM-V. American Journal of Psychiatry. 2009;166:645–650. [PubMed]
  • Robins E, Guze SB. Establishment of diagnostic validity in psychiatric illness: Its application to schizophrenia. Journal of Abnormal Psychology. 1970;110:413–422. [PubMed]
  • Roysamb E, Kendler KS, Tambs K, Orstavik RE, Neale MC, Aggen SH, Torgersen S, Reichborn-Kjennerud T. The joint structure of DSM-IV axis I and axis II disorders. Journal of Abnormal Psychology 2011 [PMC free article] [PubMed]
  • Ruscio J, Haslam N, Ruscio AM. Introduction to the taxometric method: a practical guide. Mahwah, NJ: Lawrence Erlbaum Associates; 2006.
  • Rutter M. Child psychiatric diagnosis and classification: concepts, findings, challenges and potential. Journal of Child Psychology and Psychiatry. 2011;52:647–660. [PubMed]
  • Saha S, Chant D, Welham J, McGrath J. A systematic review of the prevalence of schizophrenia. Plos One. 2005;2:e141. [PMC free article] [PubMed]
  • Sass DA, Schmitt TA. A comparative investigation of rotation criteria within exploratory factor analysis. Multivariate Behavioral Research. 2010;45:73–103.
  • Simms LJ, Watson D, Doebbeling BN. Confirmatory factor analyses of posttraumatic stress symptoms in deployed and non-deployed veterans of the Gulf War. Journal of Abnormal Psychology. 2002;111:637–647. [PubMed]
  • Slade T, Watson D. The structure of common DSM-IV and ICD-10 mental disorder in the Australian general population. Psychological Medicine. 2006;36:1593–1600. [PubMed]
  • Slade T, Johnston A, Oakley Browne MA, Andrews G, Whiteford H. 2007 National Survey of Mental Health and Wellbeing: Methods and key findings. Australian and New Zealand Journal of Psychiatry. 2009;43:594–605. [PubMed]
  • Tackett JL, Silberschmidt AL, Krueger RF, Sponheim SR. A dimensional model of personality disorder: Incorporating DSM Cluster A characteristics. Journal of Abnormal Psychology. 2008;117:454–459. [PubMed]
  • von Eye A, Clogg CC, editors. Latet variables analysis: Applications for developmental research. Thousand Oaks, CA: Sage; 1994.
  • Watson D. Differentiating the mood and anxiety disrders: A quadripartite model. Annual Review of Clinical Psychology. 2009;5:221–247. [PubMed]
  • Watson D, Clark LA, Chmielewski M. Structures of personality and their relevance to psychopathology: II. Further articulation of a comprehensive unified trait structure. Journal of Personality. 2008;76:1545–1586. [PubMed]
  • Widiger TA, Samuel DB. Diagnostic categories or dimensions? A question for the Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition. Journal of Abnormal Psychology. 2005;114:494–504. [PubMed]
  • Wolf AW, Schubert DSP, Patterson MB, Grande TP, Brocco KJ, Pendleton L. Associations among major psychiatric diagnoses. Journal of Consulting and Clinical Psychology. 1988;56(2):292–294. [PubMed]
  • Wright AGC, Pincus AL, Lenzenweger MF. An empirical examination of distributional assumptions underlying the relationship between personality disorder symptoms and personality traits. Journal of Abnormal Psychology. 2012;121(3):699–706. [PMC free article] [PubMed]
  • Wright AGC, Thomas KM, Hopwood CJ, Markon KE, Pincus AL, Krueger RF. The hierarchical structure of DSM-5 pathological personality traits. Journal of Abnormal Psychology (in press) [PMC free article] [PubMed]
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...