Acta Psychopathologica Open Access

Keywords

Anxiety; Depression; MTMM design; Adolescent psychopathology; Emotional syndrome

Introduction

Measures of anxiety and depression in children and adolescents have repeatedly been found to correlate strongly [1-4]. Likewise, diagnosis-based generalized anxiety disorder and major depressive disorder are found to have high levels of comorbidity [5,6]. In fact, it has been argued that anxiety and depression both essentially measure negative affectivity [7]. Indeed, symptom scales have identified combined symptoms of anxiety and depression, or internalizing problems, as an important emotional syndrome in childhood and adolescence [8], and some research has supported a one-factor model of anxiety and depression [9]. Nevertheless, previous studies of the structure of anxiety and depression in children and adolescents have found inconsistent levels of construct overlap [2,3,10]. Moreover, other research suggests two-factor [11] or three- factor [12] models of anxiety and depression, and that these constructs tend to cluster in distinct groups [2,11].

Given that studies have typically relied on only one source (or “method”), such as child or parent reports, and typically not controlled for random error, previous research is limited with regard to separating true trait correlations from method effects. However, parent-child agreement in multi-informant studies is generally low, with correlations between 0.20-0.30 [13,14], which may be due to parent and child reports that measure different information, rather than invalid judgments from one informant [15].

In order to examine the construct validity of psychological measures, Campbell and Fiske [16] presented the Multitrait- Multimethod (MTMM) matrix, a correlation matrix which includes a set of traits measured by different methods, and which provides information about the convergent and discriminant validity of such traits. In addition, it is possible to examine the relative contributions of the trait- or method-specific variance, where a general goal is to achieve lower method than trait variance.

Few studies have applied the CFA/MTMM approach when assessing the construct validity of measures of anxious and depressive symptoms in children and adolescents. Cole et al. [3] showed evidence of differentiation between depression and anxiety among sixth graders, but not among third graders, and reported poor construct validity, especially among the youngest children. Phillips et al. [17] found that only self-reports of symptoms of anxiety and depression, and not parent- or peer– reports, were related to self-reported negative affectivity in fifth through twelfth grade children. The authors stressed the need for parallel self- and parent-report measures to allow comparability of different reports. Prenoveau et al. [18] examined the structural relations between anxiety and depression symptoms in adolescents and found support for a tri-level structure consisting of an overarching ‘general distress’ factor, two intermediate factors, and five narrow factors. The factors displayed convergent and discriminant validity with respect to clinician severity ratings, and the authors recommended future examinations of anxious and depressive symptoms in an unselected sample.

When conducting large-scale studies, there is often a need for short symptom scales of psychopathology [19]. Such scales are easy to administer, require fewer resources, and may contribute to better participation rates, as compared to longer scales. Scales of adolescent psychopathology are used both as brief screening questionnaires and as general indicators of mental health in epidemiological and longitudinal studies [20]. However, short symptom scales often have the disadvantage of lower degrees of discriminative and convergent validity [21].

The overall aim of this study was to examine the construct validity of self- and parent-rated scales of depression and anxiety, using brief symptom scales: the Short Mood and Feeling Questionnaire (SMFQ) [19] and the General Anxiety disorder scale (GA) from the Coolidge Personality and Neuropsychological Inventory (CPNI) [22]. The current study is, to our knowledge, the first to examine the construct validity of two short scales of symptoms of anxiety and depression in a population-based sample of young adolescents using parallel self- and mother-rated measures.

The first goal was to test the fit of measurement models across respondents and traits. The second goal was to examine agreement in mother- and self-reported measures in terms of measurement invariance and factor correlations. The last goal was to examine unique and shared information from both informants on anxious and depressive symptoms by partitioning the variance into trait, source (or method) and unique components.

Method

Participants

Data collection was conducted as a part of the Tracking Opportunities and Problems (TOPP) Study, a Norwegian population-based longitudinal study focusing on the mental health of children and their families [23]. Participants were mothers and their children reporting on the children’s behaviour at age 12- 13 years. All families from 19 geographic health care areas that visited a child health clinic in 1993 for the scheduled 18-month vaccination-visit were invited to complete a questionnaire. Of the 1081 eligible families, 939 agreed to participate (87%). The current study includes the sample from the 5th wave of data collection, with reports from 594 mothers and self-reports from 547 adolescents (age 12-13 years).

The sample was predominantly ethnic Norwegian and middle class, with 46% boys. In terms of education, 23.6% of the mothers had 11 years schooling or less, and 27.1% had a college or university education of four years or more. In terms of employment, 53.5% of the mothers worked full-time outside the home, 30.2% had part-time work, and 14% had no paid work. Economically, 2.7% reported doing poorly, 20% neither good nor bad, and 75.1% reported that they are doing well. In 14.1% of families, the index child was an only-child, and 80.8 % of the mothers were living with a spouse/partner. Analyses of sample attrition from the 1st to 5th waves showed that remaining families had significantly higher levels of education at baseline compared to the dropout sample. The remaining families at wave 5 were not significantly different at baseline from the dropout sample regarding maternal distress, family adversities, social support, and child temperament [23].

Measures

Both self and maternally reported symptoms of depression were measured with the Short Mood and Feeling Questionnaire (SMFQ) [19]. The SMFQ is a unidimensional self- and parent-reported scale designed for use in epidemiological studies of depression in children and adolescents, consisting of 13 items. Two items, about poor concentration (due to overlap with another item in the questionnaire) and restlessness (due to low reliability) were omitted. The remaining 11 items addressed the affective, cognitive and somatic components of depression found to be the best predictors of depressive status (Table 1) [19]. The child and the parent are asked to rate recent depressive symptoms on a 3-point scale (i.e. ‘not true’, ‘sometimes true’, and ‘true’). The SMFQ consists of items from the Mood and Feeling Questionnaire (MFQ), which has been translated and back-translated in another Norwegian study of adolescents (13-14 years) (Table 2) [24].

Table 1: Item content of the SMFQ and the CPNI/GA scales.

Table 2: Descriptive information for mean scores of symptoms of anxiety and depression and their inter-correlations.

Symptoms of anxiety were measured with the General Anxiety disorder scale (GA), a subscale of the Coolidge Personality and Neuropsychological Inventory for children (CPNI) [22]. The GA scale consists of 12 parent-reported items that were derived from DSM-IV criteria for three of the most typical anxiety disorders in children and adolescents; separation anxiety disorder, generalized anxiety disorder and social phobia (Table 1). Because of lack of a suitable anxiety scale with parallel parent and child versions, we constructed a self-report questionnaire based on the CPNI/GA parent-reported version [23]. The self-reported items were constructed by changing the wording of the parent-reported items (e.g. ‘My child worries too much’ into ‘I worry too much’). The items were rated on a 4-point Likert-type scale (i.e., ‘not true’, ‘seldom true’, ‘sometimes true’, and ‘always true’). The CPNI has been translated and back-translated in another Norwegian study (Table 2) [25].

Statistical analyses

Confirmatory Factor Analyses (CFA) were performed using Mplus version 5.2 [26]. Robust weighted least squares for ordinal data (i.e., WLSMV) was used due to the lack of multivariate normality [26,27]. The models were estimated with missing data using the Mplus default option. Chi square difference tests were performed using the DIFFTEST option. Model fit was evaluated with the Comparative Fit Index (CFI), and the Root-Mean-Square Error of Approximation (RMSEA). A CFI value greater than 0.95 and a RMSEA value of 0.06 or less indicate a good fit between the model and the observed data [28]. Nested models with different levels of measurement invariance were compared using a χ² test and ΔCFI, with a negative ΔCFI, preferably lower than -0.01, indicating satisfactory measurement invariance [29].

We adopted a strategy where we estimated simpler subparts of the overall model before fitting the final model. We first examined 4 separate CFA models, one for each source-trait combination. These CFA models constitute the lower row of factors in Figure 1 (labelled MR Anx, CR Anx, MR Dep, and CR Dep). Each CFA for depression is a single factor model, whereas each anxiety CFA includes three separate first-order factors of ‘separation anxiety’, ‘social phobia’ and ‘generalized anxiety’, and a second-order ‘general anxiety’ factor (Figure 1).

Figure 1: The full model of mother-(MR) and child-reported (CR) symptoms of anxiety and depression, with method factors for parent and child, and unique factors (UMA=Unique Mother-Reported Anxiety; UCA=Unique Child- Reported Anxiety; UMD=Unique Mother-Reported Depression and UCD=Unique Child-Reported depression). The anxiety measures have sub-dimensions of Separation Anxiety (SAD), Social Anxiety (SA) and Generalized Anxiety (GAD). Note that the pathways are standardized, the indicators are truncated and error terms are omitted for clarity.

Next, we examined metric invariance between mother and child reports for a particular trait, with separate models for anxiety and depression. We compared a constrained model with factor loadings for mother items constrained to equal the corresponding loadings for child items, to an unconstrained model with all factor loadings freely estimated.

We then estimated the overall model as shown in Figure 1, including source factors of mother and child (variance on anxiety and depression ratings shared by the same informant, but not including the variance shared by each informant on the same trait), and trait factors of anxiety and depression. By allowing the source factors to load onto the latent mother- and child-reported trait measures we obtained the unique trait and source variance for each respondent (Figure 1).

Several CFA/MTMM approaches have been applied when analyzing the structure of trait and method effects, but two forms of CFA specification have been predominant: the Correlated-Trait Correlated-Method (CT-CM) model [30,31] and the CT-CU model [30]. Both models have been criticized [32,33], and thus the current study used an analytic strategy that integrated aspects from both models, in order to account for some of the potential weaknesses of each approach.

Results

Descriptive statistics

Table 2 shows raw mean scores, standard deviations, inter-correlations and Cronbach’s alphas for the four mean composites of symptom scales of anxiety and depression. Cronbach’s alpha was adequate and very similar across dimensions and informants. The means of the self-reported scales were significantly higher than the mother-reports for both traits (p<0.001).

Measurement models of the four symptoms scales

Table 3 shows the fit indices of the four measurement models of mother and self-reported depression and anxiety, indicating that the items in each scale are related to the trait in question (models 1-4). The fit indices generally display a good fit for the models. Model 4 has a sub-optimal RMSEA value, but the CFI is satisfactory.

Table 3: Fit indices for models of mother- and child-reported anxiety and depression.

Agreement between mother and self-reported measures

To make sure that the different rating sources were describing the same construct, we tested whether the measurement models were metric invariant (i.e., having equal factor loadings) across mother- and child-rating. First we ran an unconstrained model; estimating the factor loadings for each rating freely. Thereafter, we ran a model with equal factor loadings across the two sources, and examined whether this model had a poorer fit to the data. The result of the comparison suggested no significant differences between the two depression models (Δχ²=3.94, Δdf=8, p=0.86, ΔCFI=-0.007) and the two anxiety models (Δχ²=13.74, Δdf=10, p=0.19, ΔCFI=-0.008), indicating metric invariance between the mother and child-reported scales. Latent measures of mother- and self-reports correlated at 0.58 for both anxiety and depression.

MTMM analysis

Figure 1 shows the full model of mother- and child-reported depression and anxiety, with four trait factors, two higher-order trait factors and two higher-order informant-specific source factors. To identify the model the factor loadings of the four mother and child-reported trait factors were set to be equal to the loadings in model 1-4. The two loadings onto each of the four second-order factors were set to be equal. The higher-order trait factors were allowed to correlate, but not the higher order source factors (Figure 1). The full model showed a satisfactory fit (model 5, Table 3). The correlation between higher-order anxiety and depression was 0.84 (95% CI: 0.74-0.98). Thus, the factors appeared to represent significantly distinguishable constructs, as evidenced by fit indices, but were also shown to be substantially overlapping as evidence by the strong correlation between higher-order anxiety and depression constructs.

Table 4 shows the proportions of explained variance for maternally and self-reported anxiety and depression, where the trait, method, and unique components sum up to 1.00 for each measure. The trait variance represents the proportion of variance that is shared with the other informant. The method variance represents the proportion that is common for both traits within each informant, but not shared with the other informant. The unique variance is the proportion of variance that is not shared with the other informant, and not common between traits. All factor loadings were significant. The unique variance for child-reported anxiety was significantly higher than for mother-reported anxiety (Δχ²=10.72, Δdf=1, p=0.001), while the unique variances for child and mother-reported depression did not differ significantly (Δχ²=0.035, Δdf=1, p=0.55).

Table 4: Variance in measures of anxious and depressive symptoms accounted for by trait, method and unique effects.

Discussion

The current study examined the construct validity of the symptom scales of mother and self-reported depression and anxiety in 12-13 years old adolescents. The methods applied separated trait variance (method variance and unique variance) and, hence, took contributions from both informants into account. Our findings suggest the full model was able to integrate anxiety and depression reports from multiple informants. First, CFA models provided evidence for the structural validity of the scales. Second, the scales had similar internal structure for both informants, indicating that the informants were describing the same observed phenomenon. Third, there was a relatively high level of mother-child agreement compared to earlier findings [13].

We found that the SMFQ demonstrated a unifactorial structure, consistent with earlier findings [20,34,35]. As such, the current findings represent a replication across culture and language. By having two different reporters for depressive symptoms, we were able to show that there were no structural differences in the depression factor for self- and mother-ratings of the SMFQ. Thapar and McGuffin [35] compared SMFQ item loadings for twin pairs and singletons aged 8-17 years and found no difference in the factor structure. Our findings extend their results by demonstrating measurement invariance between parent- and self-reports of SMFQ.

The demonstration of structural validity and metric invariance was particularly important for the CPNI/GA scale, both because the measurement model had a more complex factorial structure and because the results confirm the adequacy of the mother-report and the new self-report version of the GA scale. Except for preliminary psychometric characteristics such as scale and testretest reliability [22], the current study is the first to demonstrate the structural validity of the scores in the CPNI/GA scale.

Mother-child agreement

Our findings suggested that mother-child agreement for both anxiety and depression ratings were at a moderate level, but generally higher than most studies within the field of emotional problems [13,15,36]. However, it is important to note that this does not imply high mother-child agreement in terms of the extent to which the child suffers from anxiety or depression symptoms in general, but rather agreement on the specific topics addressed in the items. Previous research has shown that forming scales with corresponding items between parent and child results in substantially higher agreement on depressive symptoms [37]. Thus, the relatively high level of agreement in our results may indicate the impact and importance of using scales with parallel items for parent and children.

The unique variance of self-rated anxiety was greater than that of mother-rated anxiety, indicating that children reported significantly more anxiety-specific information about their symptoms compared to mothers. It is important to consider that, with only one informant, we would be unable to distinguish trait variance from systematic method variance or unique variance, and would mainly interpret the method variance as part of the measured construct in question.

Overlap between symptoms of anxiety and depression

In agreement with previous research using a CFA approach [11,38], we found the latent factors for anxiety and depression to be highly correlated. We have only found one other study [3] that has tested the construct validity of scales measuring symptoms of anxiety and depression in youth using a CFA/MTMM application, and our finding is in accordance with the overlap reported therein. Importantly, such a level of overlap suggests a strong common component of the two conditions.

The high overlap between the factors of anxiety and depression is consistent with a tripartite model of anxiety and depression [39]. The focus on the tripartite model in childhood and adolescence [12] has influenced the development of measures of positive and negative affectivity, such as the PANAS-C [17]. An important question is thus the extent to which the underlying disposition of negative affectivity influences the validity of depression and anxiety measures, and whether anxiety and depression should be assessed as separate traits. It may be that the overlap between the anxiety and depression factors in the current study is an expression of the common effect of negative affectivity.

However, a correlation between two latent variables above 0.80 does not preclude important differences between these variables. As discussed, there are potentially important context-specific symptoms of anxiety and depression that should be considered. These symptoms may be less visible but nevertheless significant, and may require longitudinal data to be manifested.

Limitations and Future Research

The results should be interpreted with respect to the following limitations. First, since we have measured anxiety and depression with symptom scales rather than diagnostic interviews, the generalizations to clinical practice are limited. Our sample was taken from a nonclinical population, and research that compares symptom-reports in clinical and non-clinical populations is needed to examine whether there are similar structures between these groups. Second, the self-report scale for anxiety symptoms was constructed to correspond with the parent-report. Even though the psychometric properties of the self-reported scale seemed adequate, it is not clinically validated and future research should undertake such validation. Third, our study has applied a variant of the correlated uniqueness model, of which several disadvantages have been noted. These include the assumptions of uncorrelated method biases, of independent trait and method effects, and that the method effects constrained to be orthogonal [33]. An implication of such restrictions may be that the amount of trait variance is overestimated, artificially enhancing convergent validity while artificially worsening the discriminant validity [32]. Fourth, content overlap between the SMFQ and the GA scales could represent a caveat. However, none of the items seem to overlap each other on a phenomenological level (Table 1). Earlier research has shown that depression and anxiety are strongly related to negative affectivity. In what ways this relation influences the traits in question is uncertain. It has been suggested that negative affectivity is more genetically driven, while environmental factors contribute more to the unique aspects of depression and anxiety [40]. Future studies should continue to pursue the selective impact of environmental and genetic factors, as this avenue of research may help us differentiate between anxiety and depression.

Conclusions and Clinical Implications

Despite its limitations, our results provide potentially useful and important clinical implications. First, our findings support the structural validity of two short scales; the SMFQ and CPNI/ GA. Short symptom scales of anxiety and depression are widely used in epidemiological and large-scale longitudinal studies, both as brief screening tools for further clinical research and as indications of where to focus preventive action on a population level. The validity of such measures can have a strong influence on how symptoms of anxiety and depression in children and adolescents are approached and interpreted, both qualitatively and quantitatively.

Second, our findings demonstrate the benefits of using equivalent scales for parent- and self-reported symptoms of anxiety and depression. Compared to earlier research on parent-child agreement of children’s emotional symptoms, the current study showed relatively high levels of agreement. Whether this finding is more relevant for anxious and depressive symptoms than other behavioural symptoms of psychopathology is unclear, and future studies are needed to replicate our findings for adolescent emotional problems and other problem behaviours.

Third, the MTMM design used in this study made it possible to partition the variance of the informants’ responses into trait, method and error variance. Thus, we obtained differentiated information important for interpreting the assessment of anxious and depressive symptoms in early adolescence. The significance of having more than one informant when using rating scales is underscored.

Fourth, we found that self-reports on anxiety included significantly more unique variance than parent-reports. This is an important finding, and future studies should identify covariates that can be related to the symptoms reported solely by the youths.

Lastly, we would like to highlight the importance of the demonstrated metric invariance between the informants in our study. Since anxiety and depression are among the most influential mental health problems in children and adolescents, it is essential to know that the core symptoms of anxiety and depression, respectively, are qualitatively similar across different informants as suggested by our results. Hence, these results can inform future refinement of clinical assessment instruments.

References

1. Brady EU, Kendall PC (1992) Comorbidity of anxiety and depression in children and adolescents. Psychol Bull 111: 244-255.
2. Cannon MF, Weems CE (2006) Do anxiety and depression cluster into distinct groups? A test of tripartite model predictions in a community sample of youth. Depress Anxiety 23: 453-460.
3. Cole DA, Truglio R, Peeke L (1997) Relation between symptoms of anxiety and depression in children: A multitrait-multimethod-multigroup assessment. J Consult Clin Psych 65: 110-119.
4. Kendall PC, Kortlander E, Chansky TE, Brady EU (1992) Comorbidity of Anxiety and Depression in Youth-Treatment Implications. J Consult Clin Psych 60: 869-880.
5. Axelson DA, Birmaher B (2001) Relation between anxiety and depressive disorders in childhood and adolescence. Depress Anxiety 14: 67-78.
6. Kovacs M, Gatsonis C, Paulauskas SL, Richards C (1989) Depressive-disorders in childhood: a longitudinal-study of co-morbidity with and risk for anxiety disorders. Arch Gen Psychiat 46: 776-782.
7. Watson D, Clark LA (1984) Negative affectivity-the disposition to experience aversive emotional states. Psychol Bull 96: 465-490.
8. Egger HL, Angold A (2006) Common emotional and behavioral disorders in preschool children: presentation, nosology, and epidemiology. J Child Psychol Psychiatry 47: 313-337.
9. Wolfe VV, Blount RL, Finch AJ, Conway SF, Ronald BL et al. (1987) Negative affectivity in children-a multitrait multimethod investigation. J Consult Clin Psych 55: 245-250.
10. Lambert SF, McCreary BT, Joiner TE, Schmidt NB, Ialongo NS (2004) Structure of anxiety and depression in urban youth: an examination of the tripartite model. J Consult Clin Psych 72: 904-908.
11. Ollendick T, Seligman L, Goza AB, Byrd DA, Singh K (2003) Anxiety and depression in children and adolescents: a factor-analytic examination of the tripartite model. J Fam Stud 12: 157-170.
12. Laurent J, Ettelson R (2001) An examination of the tripartite model of anxiety and depression and its application to youth. Clin Child Fam Psych 4: 209-230.
13. Achenbach TM, Mcconaughy SH, Howell CT (1987) Child adolescent behavioral and emotional problems- implications of cross-informant correlations for situational specificity. Psychol Bull 101: 213-232.
14. De Los Reyes A, Kazdin AE (2005) Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychol Bull 131: 483-509.
15. Karver MS (2006) Determinants of multiple informant agreement on child and adolescent behavior. J Abnorm Child Psych 34: 251-262.
16. Campbell DT, Fiske DW (1959) Convergent and Discriminant Validation by the Multitrait-Multimethod Matrix. Psychol Bull 56: 81-105.
17. Phillips BM, Lonigan CJ, Driscoll K, Hooe ES (2002) Positive and negative affectivity in children: A multitrait-multimethod investigation. J Clin Child Adolesc 31: 465-479.
18. Prenoveau JM, Zinbarg RE, Craske MG, Mineka S, Griffith JW, et al. (2010) Testing a hierarchical model of anxiety and depression in adolescents: A tri-level model. J Anxiety Disord 24: 334-344.
19. Angold A, Costello EJ, Messer SC (1995) Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int J Method Psych 5: 237-249.
20. Messer SC, Angold A, Costello EJ, Loeber R (1995) Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents: factor composition and structure across development. Int J Method Psych 5: 251-262.
21. Hodges K, Saunders WB, Kashani J (1990) Internal consistency of Dsm-Iii diagnoses using the symptom scales of the child assessment Schedule. J Am Acad Child Psy 29: 635-641.
22. Coolidge FL, Thede LL, Stewart SE, Segal DL (2002) The coolidge personality and neuropsychological inventory for children (CPNI)-preliminary psychometric characteristics. Behav Modif 26: 550-566.
23. Karevold E, Roysamb E, Ystrom E, Mathiesen KS (2009) Predictors and pathways from infancy to symptoms of anxiety and depression in early adolescence. Dev Psychol 45: 1051-1060.
24. Sund AM, Larsson B, Wichstrom L (2001) Depressive symptoms among young Norwegian adolescents as measured by The Mood and Feelings Questionnaire (MFQ). Eur Child Adoles Psy 10: 222-229.
25. Kristensen H, Torgersen S (2007) The association between avoidant personality traits and motor impairment in a population-based sample of 11-12-year-old children. J Pers Disord 21: 87-97.
26. Muthén LK, Muthén BO (1998-2012) Mplus User's Guide (7th edn.). Los Angeles, CA.
27. Flora DB, Curran PJ (2004) An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychol Methods 9: 466-491.
28. Hu LT, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 6: 1-55.
29. Cheung GW, Rensvold RB (2002) Evaluating goodness-of-fit indexes for testing measurement invariance. Struct Equ Modeling 9: 233-255.
30. Marsh HW (1989) Confirmatory factor analyses of multitrait multimethod data many problems and a few solutions. Appl Psych Meas 13: 335-361.
31. Widaman KF (1985) Hierarchically Nested Covariance Structure Models for Multitrait-Multimethod Data. Appl Psych Meas 9: 1-26.
32. Kenny DA, Kashy DA (1992) Analysis of the multitrait multimethod matrix by confirmatory factor analysis. Psychol Bull 112: 165-172.
33. Lance CE, Noble CL, Scullen SE (2002) A critique of the correlated trait-correlated method and correlated uniqueness models for multitrait-multimethod data. Psychol Methods 7: 228-244.
34. Sharp C, van Goozen S, Goodyer I (2006) Children's subjective emotional reactivity to affective pictures: gender differences and their antisocial correlates in an unselected sample of 7-11-year-olds. J Child Psychol Psychiatry 47: 143-150.
35. Thapar A, McGuffin P (1998) Validity of the shortened mood and feelings questionnaire in a community sample of children and adolescents: a preliminary research note. Psychiat Res 81: 259-268.
36. Sourander A, Helstela L, Helenius H (1999) Parent-adolescent agreement on emotional and behavioral problems. Soc Psych Psych Epid 34: 657-663.
37. Hepperlin CM, Stewart GW, Rey JM (1990) Extraction of depression scores in adolescents from a general-purpose behavior checklist. J Affect Disorders 18: 105-112.
38. Crowley SL, Emerson EN (1996) Discriminant validity of self-reported anxiety and depression in children: negative affectivity or independent constructs? J Clin Child Psychol 25: 139-146.
39. Clark LA, Watson D (1991) Tripartite model of anxiety and depression-psychometric evidence and taxonomic implications. J Abnorm Psychol 100: 316-336.
40. Kendler KS, Neale MC, Kessler RC, Heath AC, Eaves LJ, et al. (1992)Major depression and generalized anxiety disorder-same genes, (Partly) Different Environments. Arch Gen Psychiat 49: 716-722.