Acta Psychopathologica Open Access

Keywords

Pediatric; Primary care; Screening; Psychosocial problems; Parenting; Integrated care

Introduction

Prevalence rates for common psychosocial problems in children such as behavioral disturbances including disciplinary difficulties, school-related issues, and problems related to child social relationships, range from approximately 10-25 % (CDC, 2013) [1]. Despite these significant base rates, the majority of children presenting with significant behavioral concerns do not receive specialized behavioral health treatment (Burns et al., 1997; Holt and McDowell, 1998) [2, 3], leaving pediatric primary care as the de facto delivery system of psychosocial interventions for 75% children with a diagnosed behavioral health concern (NIHCM, 2009) [4].

Estimates indicate that almost 95% of American families have a primary care provider whom their children see regularly, on average 3-5 times per year (Bloom, Cohen, Freeman, 2012). Research strongly suggests that children who have behavioral concerns will likely be seen even more often, according to a recent CDC report (2011). In addition, children with diagnosable psychological disorders are more likely to utilize emergency health care services than other children (Grupp-Phelan, et al., 2009) [5].

In fact, behavioral concerns are the most common non-medical issue that parents discuss with their pediatricians (Glascoe, MacLean, Stone, 1991) [6] and account for the majority (81%) of questions that parents have for their pediatrician (McCune, Kalmanson, Fleck, Glazewski, Sillari, 1990) [7]. These data suggest a fundamental mismatch between help-seeking behavior on the part of families experiencing difficulties and the existing medical system ostensibly structured to meet their needs.

In fact, child social and behavioral problems are so common that they have been termed “the new hidden morbidity” and are a major source of healthcare costs in pediatrics (Foy, 2010). The American Academy of Pediatrics (AAP), in a statement regarding competency guidelines, called for the mandatory screening of all children and adolescents for psychological concerns in pediatric primary care settings (AAP, 2009). However, there are neither widely-established physician training programs nor visit-specific protocols to guide the process of screening for and appropriately addressing behavioral concerns in pediatrics (Drotar, 2002) [8]. Moreover, significant discrepancies have been reported between parental concerns about child psychological well-being and/or family functioning and what parents actually address with their physician (Briggs-Gowan, Horwitz, Schwab-Stone, Leventhal, Leaf, 2000; Burklow, Vaughn, Valerius, Shultz, 2001) [9,10]. Ninetyone percent of parents report that it would be appropriate to discuss family problems with their pediatrician and 78.8% report that it would be appropriate to discuss disciplinary concerns (Briggs-Gowan, et al., 2000) [9]. However, only about 8% of parents actually spontaneously address their concerns with their pediatrician and only 27-40% discusses their concerns with the pediatrician when asked specifically about behavior problems during their office visit (Glascoe, MacLean, Stone, 1991; McCue Horwitz, Leaf, Leventhal, 1998; Taaffe Young et al., 1998) [6,11,12]. Fifty-five percent of parents of children meeting criteria for a psychological disorder do not address their behavioral/emotional concerns about their child with their child’s physician (Briggs- Gowan et al., 2000) [9]. Therefore, parental underreporting is a significant impediment to the detection and treatment of behavioral problems in the primary care setting which may logically be remediated through the use of screening.

Despite the potential benefit of screening, there appear to be significant barriers to widespread implementation. For example, many questionnaires have been developed and tested for the assessment of child behavior problems. However, most of these instruments were developed for research purposes and are cumbersome and impractical in the reality of the primary care setting (Stancin & Palermo, 1997) [13]. As a result, most primary care practices do not use screening devices routinely (Jensen et al., 2011) [14]. In a survey of pediatricians, only one-third of respondents reported ever having used a screening instrument in their practice and 77% believed that such instruments “are rarely useful because (they) don’t feel competent to deal with these issues,” (Cheng, Savageau, Bigelow, Charney, 1996) [15].

As a result of failing to screen appropriately, there is considerable evidence suggesting that 46%-80% of a patient’s behavioral concerns are not known to their pediatrician (e.g., Costello et al., 1988; Sheldrink, Merchant, Perrin, 2011) [16, 17] and when detected, these concerns will not be adequately addressed (e.g. Steele, Lochrie, Roberts, 2010; Wildman & Stancin, 2006) [18,19]. It has been estimated that less than half of children identified by primary care physicians as having “mental health problems” received referrals to a tertiary care facility (Bernal, 2003) [20]. This statistic is particularly disheartening given the finding that one of the strongest predictors of a family’s decision to seek psychological services for their child is the recommendation of their primary care physician (Lavigne et al., 1998) [21]. The estimated costs associated with the known chasm between prevalence and detection rates are high for the individual child, the family, and society, given that the most effective and efficient method of service delivery is early detection and intervention (NIHCM, 2009).

Purpose

The purpose of the current research was to create and conduct preliminary evaluation of a parental self-report assessment questionnaire called the Pediatric Screening Inventory that is designed to improve on assessment as usual by: (1) improving the comprehensiveness of primary care screening; (2) being practical for use in the primary care setting (i.e. little administration time; ease of scoring); (3) assessing domains of functioning on continua, rather than being bifurcated artificially as either meeting or not meeting diagnostic criterion, and (4) subsequently increasing prevention and intervention efforts within primary care visits and referrals to other psychological or physical health specialists.

Hypotheses

It was hypothesized that:

1. The Pediatric Screening Inventory would result in increased detection of problems in child and/or family functioning in comparison to assessment as usual.

2. The Pediatric Screening Inventory would be associated with an increased rate of behavioral or parenting intervention during the primary care visit as well as an increased rate of referral.

3. The Pediatric Screening Inventory would have both greater physician and parent satisfaction than assessment as usual.

Method

Development of the Pediatric Screening Inventory

The Pediatric Screening Inventory was developed using a rational method of test construction to develop items relevant to the three domains of interest (child behavior problems, parenting practices/family environment, and physical health factors/medical treatment adherence). For child behavior problems, items were developed to assess both internalizing and externalizing types of behavioral problems (e.g., Achenbach, 2012). Specifically, four behavioral domains of interest were chosen: daily activities and behaviors, obedience/following the rules, coping with feelings, and biopsychosocial development. To construct the parenting practices/family environment portion of the assessment measure, domains of interest were chosen to parallel two reliable and well-validated parenting assessment instruments, The Family Assessment Device (Epstein, Baldwin, Bishop, 1983) [22] and the Alabama Parenting Questionnaire (APQ; Essau, Sasagawa, Frick, 2006; Shelton, Frick, Wootton, 1996) [23]. Three parenting domains were determined to be of utmost importance to assess on the PSI: self and family management (monitoring/supervision, home environment, including access to resources and safety, and parental emotional well-being), setting limits and using discipline (discipline, consistency, and behavior control), and parent-child relationship (parental involvement and use of positive parenting techniques). Finally, to increase its relevance to a medical setting, questions related to adherence to previously prescribed medical treatments and general indices of physical health were developed.

An initial item pool of 53 items encompassing all three domains of the PSI was developed. Two types of focus groups were held to gain feedback on the initial item pool. First, five focus groups with pediatric medical professionals were held (n=41). Secondly, two parent focus groups were conducted (n=26; 25 mothers and 1 father). Finally, expert reviewers were also asked to provide feedback on early drafts of the PSI (n=4). Three of the expert reviews are nationally recognized experts in behavioral pediatrics with extensive training and experience providing psychological services in pediatric primary care environments. The fourth expert reviewer is the Chief of Behavioral Pediatrics at a major university offering specialized training in the field. Feedback from all focus groups and expert reviews was incorporated into the final version of the PSI; six revisions of the PSI took place prior to beta testing. (Table 1)

Child Behaviors	Not a concern	A concern
Daily Activities and Behaviors
1.Sleeping (how much, where, when)
2.Using the toilet/toilet training
3.Eating
4.Being too active/having too much energy
5.Having “bad” habits (e.g. thumb sucking)
Obedience/Following the Rules
6.Doing as they’re told
7.Talking back
8.Using “bad” or rude words
9.Having temper tantrums
10. Hurting themselves or others
Coping with Feelings
11. Pouting
12. Not being liked by other children
13. Getting used to life changes (e.g. divorce, moving)
14. Being nervous or worried a lot
15. Being grouchy or irritable
16. Being too afraid
17. Being moody in general
18. Crying too much
Development
19. Learning
20. Getting along with others (parents, siblings, etc.)
21. Doing things other children their age do
22. Talking or communicating
23. Not growing big enough or growing too big
24. Being physically coordinated
25. Paying attention (for their age)
Parenting Behaviors	Not a concern	A concern
Managing the Family & Myself
26. Knowing where my child is
27. Having a stable place to stay
28. My own emotional well-being
29. Using drugs/alcohol in the home
30. Having an unsafe home environment
31. Having enough money
32. Finding childcare
33. Having violence or abuse in the home
34. Balancing my responsibilities (work, home)
Setting Limits and Using Discipline
35. Being too strict
36. Not knowing what to do when my child misbehaves, such as using spanking or time-outs
37. Agreeing on discipline with my child’s other parent or other caregivers
38. Setting limits
39. Losing my temper
40. Being too “easy”or lenient
41. Not always disciplining when I should
My Relationship with My Child
42. Spending time with my child
43. Knowing what to expect from my child
44. Showing affection or love to my child
45. Needing more space away from my child
Following Medical Directions
46. Following up with doctor’s appointments for my child (remembering the appointment, finding a ride, finding childcare for other children, etc.)
47. Getting my child to take their medicine even if they don’t like it
48. Knowing when my child feels sick or needs medical attention
49. Having enough money to get medical care for my child (paying for office visits, buying medicine, etc.)
50. Getting my child to follow doctors’ instructions, such as taking all of their medication, eating certain foods or doing certain activities

Most parents have some concerns about raising their family. Please check if the following common problems are a concern for you or anyone else who helps to take care of your child. This page deals with common concerns about children and the back side deals with common concerns about parenting.

Table 1: Pediatric Screening Inventory (For children ages 1-4).

Administering the PSI to a Development Sample

The second phase of this project involved piloting the PSI in the pediatric primary care setting. The piloting of the PSI took place over approximately a five month period.

Participants

Fifty-six physicians treating children and families in the greater Lansing, Detroit, and Ann Arbor, Michigan areas were invited to participate in this phase of the study. Sixteen physicians consented to participate, however, and data collection was only feasible for four physicians due to logistic constraints of their practices. Two hundred and sixteen adult parents of children aged 1-4 also participated in piloting the instrument in the context of their pediatric primary care provider’s office.

Procedures

All procedures for this study were reviewed and approved by the Internal Review Board at the University of Nevada, Reno prior to data collection. A randomized block design was employed for data gathering and analysis; physicians were the blocking factor. Physicians were randomly assigned to either use the PSI during a patient visit or to follow usual assessment procedures during the visit. Therefore, each physician participated in both the experimental and control conditions. Data were analyzed using SPSS with logistic regression and ANOVA models.

Recruitment

Parent participants were recruited to participate when they presented their child for medical care and met the criterion for inclusion. Research packets with a letter of invitation as the cover page were given to each parent as they checked in with the receptionist, consistent with HIPPA requirements. Parents indicated their consent by completing the questionnaires, rather than by completing an informed consent document, in order to maintain anonymity. Provider questionnaires were paired with parent questionnaires by using a code number which was printed on all questionnaires pertaining to a particular visit.

Experimental Condition Procedures

In the experimental condition, parents bringing their children to primary care visits (either well-child or illness/injury) were given a three page packet of research materials as they reported to the receptionist. A brief (1-3 sentences) introduction inviting parents to participate was the first page of this packet. The second page of the packet was the Background Questionnaire, created for the purpose of this study, which asked basic demographic questions. The third and final page of the packet was the PSI (double-sided), the instrument being evaluated. It is estimated that completing the Background Questionnaire and the PSI took parents no more than 5-7 minutes prior to their physician visit.

Parents were asked to give the completed research materials to their provider when they entered the examination room. It was up to the provider to decide whether they used the PSI during the visit; therefore, there was no direct experimental manipulation of any physician behaviors during the visit. At the completion of the visit, either the provider or the parent deposited the research materials in a secured marked box which was present in each of the exam rooms. It is noteworthy that none of the participating pediatric practices had a behavioral health specialist on staff at the time of the study.

Following the visit, both providers and parents were asked to complete brief questionnaires about what happened during the visit. Specifically, the first question asked whether or not any psychosocial intervention occurred during the visit. Secondly, parents were asked if any referral for specialized services was made during the visit. Third, they were asked if the PSI changed the outcome of the office visit (providers only). Finally, both parents and providers were asked to rate their satisfaction with the visit overall on a Likert-style scale from 1-100, with 100 anchoring greatest satisfaction.

Parents were given the Post-Visit Questionnaire by the receptionist as they checked-out of the clinic. Therefore, their providers did not have access to their responses. Parent Post- Visit Questionnaires were then deposited in a marked box at the receptionist’s window. For physicians, the Post-Visit Questionnaire was attached to the chart for each visit. These were then deposited by the physician in a marked box located in a convenient place in the clinic (e.g. records room, lounge, etc.). It was estimated that completing post-visit questionnaires took less than 2 minutes each for providers and parents.

Control Condition Procedures

The same procedures were followed in the Control Condition as in the Experimental Condition, except that parents were not asked to complete the PSI, therefore, physicians were not exposed to the screening data.

Results

Demographics

A total of 214 parent-provider dyads participated in the piloting of the PSI across both conditions (108 controls, 105 experimental visits). Ninety-four percent of parent respondents were the biological parents of the child they were bringing for medical care, 3.4% were adoptive parents, and the remaining respondents were grandparents, foster parents, and/or legal guardians (1.1%, .6%, and .6%, respectively). As they were functioning in the capacity of parent, accounted for the majority of the respondents bringing a child for a primary care visit, and for the sake of simplicity, all of these groups will be referred to as parents.

Eighty-four percent of parent respondents were female. The gender of the children presenting for care was relatively equal; 59% male and 41% female. The mean age of parent respondents was 33 years with a range of 20-50 years and a standard deviation of 5.3 years. The mean age of children in the study was 2.3 years (only parents of children aged 1-4 were included in the study), with a standard deviation of 1.2 years. Eighty-four percent of parent respondents reported that their children are Caucasian with 9% reporting their children are biracial and 4% reporting their children are African American. These statistics are consistent with the communities where the data were collected.

Eighty percent of parents reported English as the only language spoken in their homes and the remainder reported bilingual homes. The majority of parent respondents reported that they are currently married (86.4%); with 5% each reporting that they are either divorced or has never been married. Ninetysix percent of parent respondents reported that they are their child’s primary caregiver. When questioned as to the reasons that they were presenting their child for pediatric primary care, 40% of respondents reported that their child was sick, 31% were attending a well-child visit, 21% were coming for a follow-up visit from an illness or injury, and 6.3% were obtaining a physical.

Provider Practice Data

As described above, four physicians participated in PSI pilottesting. These subjects reported that they had been providing medical care for children on average 11.3 years (range 4-22 years). When asked how comfortable they are providing advice or guidance to parents regarding their parenting practices on a scale of 1-10 (1= not at all comfortable, 10=very comfortable), the mean rating was 7.3, “moderately comfortable” (range 6-8). When asked how comfortable they felt providing advice or guidance to parents specifically regarding appropriate child behavior or child behavior problems, the mean response of the physician participants was 6.8, or “moderately comfortable” (range 6-7). When asked how much training they had had in behavioral pediatrics on a scale of 1-10 (1=not very much, 10=in excess), the average rating was 5.3, or “sufficient training” (range 3-6). When asked whether or not they felt that their training was adequate in behavioral pediatrics on a scale of 1-10 (1=not at all, 10=very much so), the mean rating for the four physicians was 4.3 (range 3-5), a little less than “sufficiently adequate.” Finally, when asked to rate the adequacy of their knowledge of local behavioral healthcare resources on a scale of 1-10 (1=not adequate, 10=very adequate), physicians gave a mean rating of 5 (range 3-7), “moderately adequate.”

Descriptive Statistics of the PSI

Eighty-seven of the 105 parent respondents in the experimental condition completed the entire PSI. The PSI contains 50 items and each item could be rated either a concern or not a concern by the parent respondent. Seventy-eight percent of parents completing the PSI reported at least one concern, with the mean number of concerns being four, the median three, and the mode one.

The most frequent concerns reported by parents completing the PSI were balancing responsibilities (e.g., home and work), endorsed as a concern by 24.1% of respondents, followed by temper tantrums (23%), and obeying, financial stability, and consistency of discipline (20.7% each). Concerns relating to domestic violence, showing affection to their child, and following up with medical appointments were not endorsed by any parent participants.

When parents were asked following their visit whether or not using the PSI made it easier to talk about their concerns during their visit, 58.7% reported that it was helpful. When asked if the PSI reminded them to initiate discussions or ask questions about their child’s behavior during the visit, 61.3% reported that it had. When parents were asked if they would like it if their pediatric primary care provider always used the PSI when they came to an appointment, responses were approximately evenly distributed, with 48.6% responding that they would like that practice and 51.4% responding that they would not.

When providers were asked whether or not the PSI had any impact on the interventions they provided, participants indicated that only in 7.6% of visits that the PSI had made a difference in their behavior.

Hypothesis Testing

Hypothesis 1: Using the Pediatric Screening Inventory would result in increased detection of problems in child and/or family functioning in comparison to assessment as usual.

Hypothesis 1 was tested using a logistic regression model. The independent variables in this model were the categorical variable of condition (experimental vs. control) and the categorical variable of physician, which was coded to test for physician differences. The dependent variable was whether or not a behavioral concern was detected during the visit. Behavioral problems were detected 68 times (65% of visits) in the experimental condition and 26 times (24% of visits) in the control condition. A significant main effect for condition (sig=.000, p<.001) was found with no significant physician effect and no significant interaction. The odds of detecting a behavioral concern in the experimental condition were approximately twice that of the control condition. By these criteria, Hypothesis 1 was supported.

Hypothesis 2: Use of the PSI would be associated with (a) an increased rate of behavioral or parenting intervention during the primary care visit and (b) an increased rate of referral for services outside of the primary care setting.

Hypothesis 2 was tested using a logistic regression model. The independent variables in this model were the categorical variable of condition (experimental vs. control) and the categorical variable of physician, which was coded to test for physician differences. The dependent variables were whether or not a (a) behavioral intervention and/or a (b) referral occurred during the visit. Behavioral interventions occurred 42 times (40% of visits) in the experimental condition and 26 times (24% of visits) in the control condition. Referrals for specialty care occurred 7 times in the experimental condition (7% of visits) and 10 times (9% of visits) in the control condition.

When the logistic model for intervention was tested, a significant main effect was found for condition (sig=.011, p<.05), with no significant physician effect and no interaction. The odds of a behavioral intervention occurring in the experimental condition were 1.6 times more likely than in the control condition. When the logistic model for referrals was tested, no significant effects were found for condition or physician and there were no significant interactions. By these criteria, Hypothesis 2 was supported for interventions, but rejected with regard to referrals.

Hypothesis 3: The PSI would be associated with greater physician and parent satisfaction than assessment as usual.

To test this hypothesis, satisfaction was rated as a continuous variable on a Likert scale from 1-100 (100=very satisfied). Of a total of 170 physician satisfaction ratings, 161 ratings were “100.” The mean rating of satisfaction was 97.8 and the mode and median were both 100. Likewise, of a total of 161 parent satisfaction ratings, 138 ratings were “100.” The mean rating of satisfaction was 93.4 and both the mode and median were 100. Because of these ceiling effects, no significant differences were found between condition and physician and this hypothesis was rejected.

Discussion

The present study attempted to develop and evaluate a screening device to detect parental concerns regarding child behavior, parenting, and following medical directions in the context of routine pediatric primary care. The resulting questionnaire is a brief 50 item parental self-report measure entitled the Pediatric Screening Inventory (PSI).

When providers participating in the piloting of the instrument were asked to rate their comfort with providing behavioral guidance, they reported that they were only moderately comfortable doing so. When asked to rate the sufficiency of their training in behavioral pediatrics, physicians reported that they had received less than sufficient training on average, akin to the early findings of Jellinek (1982) [24] and Costello (1986) [25]. Also consistent with previous findings, physicians reported only moderately adequate knowledge of local behavioral healthcare resources. Taken together, these data suggest that pediatricians are not well-prepared to meet the behavioral needs of their patients and could benefit from collaboration with a behavioral health provider within the practice.

The majority of parents who completed the PSI (78%) indicated that they had a least one concern regarding their child’s behavior, their parenting, and/or following medical directions. Notably, some of the more prevalent areas of concern reported by parents in this study (e.g., balancing responsibilities, financial stability) have not been previously examined in pediatric primary care, indicating that the PSI may have provided an impetus for parents to communicate genuine concerns relevant to the well-being of their child that would have otherwise not been known by their medical provider.

In this vein, a little more than half of parents surveyed indicated that using the PSI made it easier for them to remember and initiate discussions about their behavioral concerns. Correspondently, about half of parents reported that they would approve of their pediatrician using the instrument routinely. Presumably these were the same parents for whom the instrument was a useful bridge between their concerns and communicating their concerns to their child’s doctor. However, when physicians were asked to report whether the PSI had changed their clinically relevant behavior during visits, the vast majority reported that it had not.

These findings suggest that while the PSI may be perceived as a useful tool by parents, the use of the PSI may not lead to any changes in provider behavior. Ultimately, this disconnection between assessment and treatment could lead to increased frustration in parents and providers, as parents would have presented concerns to their child’s physician but the physician was either unable or unwilling to address these concerns, given practice and training constraints. While this question was not examined in this study, it is also possible that this apparent breach may actually serve a protective function if it were the case that increased detection may lead to increased and potentially iatrogenic treatments being applied by clinicians who are not sufficiently trained and/or skilled to do so (e.g., pediatricians not trained in behavioral interventions failing to administer empirically-supported treatments in favor of inappropriately prescribing psychotropic medications). Furthermore, the question remains when (e.g., sick visits, well-child visits only, etc.) and how often (i.e., yearly, every visit) the PSI should be used. In addition, future research should investigate the extent to which physician behavior changes when a collocated behavioral health consultant is present in the pediatric practice.

Overall, the PSI improved detection of behavioral concerns significantly over assessment as usual. Even in the control condition, however, the rates of detection exceeded those previously reported in the literature when parents were not prompted to report concerns (e.g., McCue Horowitz et al., 1998; Taaffe Young et al., 1998) [26,12]. What is not known is how physicians in the control condition assessed for behavioral concerns. Future research also needs to investigate the sensitivity and specificity of the PSI as it may be the case that some of the identified behaviors were false positives. Because of the randomized block design, carryover effects are also likely, which suggests that perhaps physicians in control condition visits were more likely to assess for behavioral concerns because of having also recently participated in experimental visits. These effects may, in part, explain why detection in the control condition was higher than predicted.

Overall, the rate of detection in the experimental condition matches reported rates in previous research, while the control condition (though comparable to other detection studies) falls far short of adequately detecting behavioral concerns in comparison to the experimental condition and known prevalence rates. These preliminary findings suggest that the PSI may improve current clinical practices by greatly improving physician awareness of behavioral concerns of their patients’ parents.

Similar to detection, rates of intervention occurred at significantly higher levels in the experimental condition than in the control condition. However, in the experimental condition, far more behavioral problems were detected than were addressed, while in the control condition the rates of detection and intervention matched. A question raised by this finding is whether the PSI was accurately detecting behavioral concerns (those warranting intervention) in the experimental condition, but barriers in the visit impeded intervention occurring at a higher rate. For example, parents may have been identifying concerns through the use of the PSI, but factors such as limited time during the visit and/or physician reluctance to treat may have interfered with an intervention actually taking place. Additional research would be necessary to determine what variables may have interfered with physicians’ ability to intervene when behavioral problems were detected, but these findings may have significant implications for the system of pediatric primary care. For example, if it is the case that insufficient training and/or time during visits are factors that interfere with the delivery of appropriate behavioral interventions; this may suggest a need for a more radical change in the structure of typical pediatric primary care. One solution to the dilemma of having inadequate resources in primary care to provide appropriate behavioral care would be to include in the treatment team a co-located pediatric behavioral health specialist, such as a specially-trained psychologist.

It is also interesting to note that the intervention findings do not support the physicians’ reports that their behavior was unaltered by using the PSI, given that rates of intervention were much higher in the experimental condition. Was it the case that physician behavior was influenced by the PSI and yet they did not perceive it to be so? Physicians may have increased their rate of intervention because of the stimulus cues inherent to participating in the research, without the PSI serving as a specific cue. It is also plausible that physicians consulted the PSI but perceived that they would have been able to detect and address the same concerns with usual methods of assessment, leading to their self-report that the PSI did not impact their intervention.

Unfortunately, this question cannot be answered from the current data and will require additional scientific inquiry.

To summarize the findings on the ability of the PSI to increase the rates of behavioral intervention in pediatric primary care, the outcomes of the statistical tests for significance indicate that the PSI condition far exceeded the control condition in rates of intervention. However, this apparent change in behavior may or may not be attributed to the use of the PSI, particularly given the reports of physicians that the PSI did not impact their interventions.

Across both the experimental and control conditions, referrals for specialty care occurred very rarely. Furthermore, no differences were noted between the two groups on this variable. These findings are particularly interesting in the context of the findings previously described indicating that relatively high levels of behavioral concerns and low levels of behavioral intervention were reported. One would expect that if there was a gap between clinical concerns and intervention during the visit that referrals would fill that gap, however, this was not borne out empirically. One possible explanation for the low level of referrals was the physicians’ self-report of only moderately adequate knowledge of local behavioral health resources. If this was the case, then training in local resources should theoretically rectify the problem. However, if there were other reasons that physicians were not referring their patients for additional specialty care, such as insurance limitations, a belief that behavioral interventions are ineffective, or a dearth of available resources in the community, possible solutions may be much more difficult to develop. Moreover, across both conditions (experimental and control) there appeared to be a ceiling effect in the ratings of physician and parent satisfaction. Satisfaction ratings were uniformly much higher than anticipated. In fact, this invariance in the data precluded meaningful statistical analysis of the satisfaction data.

Limitations of the Current Study

Although the sample size was sufficient for preliminary testing of the PSI as evidenced by the significant findings in two important areas (detection and intervention), it was not sufficiently large to draw conclusions about the utility of the PSI or the generalizability of the data to other populations of providers or parents. Specifically, only a small number of providers participated in this study, all of whom shared the same two offices and self-selected to participate based on their self-described interest in behavioral pediatrics. It is quite possible that lower rates of assessment and intervention would be found in other samples of pediatric providers. Furthermore, all of the participating physicians were pediatricians; however, many physicians providing care to children and families are general practitioners and as such would have different training histories and may have different practice patterns. Therefore, the findings with regard to physician behavior have only limited generalizability.

At this stage of instrument development, it would be premature to recommend that the PSI be disseminated for use in pediatric primary care. However, the results were promising and no ill effects related to using the PSI (such as decreased satisfaction) were reported. Based on the findings reported above, it is reasonable to suggest that the PSI continue to be used in pediatric clinics in the context of continued instrument development.

There are several logical steps which need to be taken in order to continue to develop the PSI. First and foremost, the psychometric properties of the PSI need to be examined. First, the PSI needs to be administered to a larger and more diverse sample of parents. In doing so, it would be helpful to establish test-retest reliability by administering the instrument to the same set of parents twice with a one-week time interval in between (enough time to reduce carry-over effects but a small enough time frame that we would not expect meaningful changes in concerns). Following preliminary analysis of reliability, each of the 50 items should be carefully examined independently and in relationship to the other items, likely utilizing exploratory factor analysis. Items which are endorsed infrequently or correlate highly with other items should be evaluated for possible exclusion. In addition, the factor analytic process should also examine the extent to which the items correspond to the three main areas of interest on the PSI.

Conclusions

In summary, support was found for the two main hypotheses in this study. The PSI may be useful in improving the detection of behavioral problems and the occurrence of behavioral interventions in pediatric primary care. No evidence was found to support the PSI increasing the rate of referrals for specialty care nor that the use of the instrument improves either parent or physician satisfaction with pediatric primary care visits. However, its best use may be in integrated primary care in which behavioral health consultants are present. Collectively, these data suggest that the PSI may have some utility as a new screening tool in pediatric care and additional research is warranted to further explore the psychometrics properties and utility of the PSI.

References

1. Centers for Disease Control and Prevention (2013) Mental health surveillance among children – United States, 2005—2011. MMWR 62:1-35.
2. Burns BJ, Costello EJ, Erkanli A, Tweed DL, Farmer EMZ et al (1997) Insurance coverage and mental health service use by adolescents with serious emotional disturbance. Journal of Child and Family Studies 6: 89-111.
3. Holt J & McDowell MJ (1998) Developmental-behavioural problems in general paediatrics. J Paediatr Child Health 34: 245-249.
4. National Institue for Health Care Managment (2009) Strategies to Support the Integration of Mental Health Into Pediatric Primary Care 1-36.
5. Grupp-Phelan J, Mahajan P, Foltin GL, Jacobs E, Tunik M, et al. (2009) Referral and Resource Use Patterns for Psychiatric-Related Visits to Pediatric Emergency Departments. PediatrEmerg Care 25: 217-220.
6. Glascoe FP, MacLean WE,Stone,WL (1991) The importance of parents’ concerns about their child’s behavior. ClinPediatr30: 8-11.
7. McCune L, Kalmanson B, Fleck MB, Glazewski B, Sillari J (1990)An interdisciplinary model of infant assessment. In S.J. Meisels and J.P. Shonkoff (Eds.), Handbook of early childhood intervention, New York: Cambridge University Press 219-245.
8. Drotar D (2002) Behavioral and emotional problems in infants and young children: Challenges of clinical assessment and intervention. Infants & Young Children 14: 1–5.
9. Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, Leventhal JM, Leaf PJ (2000)Mental health in pediatric settings: Distribution of disorders and factors related to service use. J Am Acad Child Adolesc Psychiatry. 39: 841-849.
10. Burklow KA, Vaughn LM, Valerius KS, Schultz JR (2001) Parental expectations regarding discussions on psycho-social topics during pediatric office visits. ClinPediatr40: 555–562.
11. McCue Horwitz S, Leaf PJ,Leventhal JM (1998) Identification of psychosocial problems in pediatric primary care: Do family attitudes make a difference? Arch PediatrAdolesc Med 152: 367-371.
12. Taaffe Young K, Davis K, Schoen C, Parker S (1998)Listening to parents: A national survey of parents with young children. Arch PediatrAdolesc Med 152: 255-262.
13. Stancin T & Palermo TM (1997)A review of behavioral screening practices in pediatric settings: Do they pass the test? J DevBehavPediatr18: 183-94.
14. Jensen PS, Goldman E, Offord D, Costello EJ, Friedman R, et al. (2011) Overlooked and Undersevered: "Action Signs" for Identifying Children with Unmet Mental Health Needs. Pediatrics 128: 970-979.
15. Cheng TL, Savageau JA, Bigelow C, Charney E (1996) Assessing mothers' attitudes about the physician's role in child health promotion. Am J Public Health86: 1809-1812.
16. Costello EJ, Edelbrock C, Costello AJ, Dulcan MK, Burns BJ, et al. (1988) Psychopathology in primary care: The new hidden morbidity. Pediatrics 82: 415-424.
17. Sheldrick RC, Merchant S, Perrin EC (2011) Identification of Developmental-Behavioral Problems in Primary Care: A Systematic Review. Pediatrics 128: 356-363.
18. Steele MM, Lochrie AS, C RM (2010) Physician identification and management of psychosocial problems in primary care. J ClinPsychol Med Settings 17: 103-115
19. Wildman BG, Kinsman AM, Logue E, Dickey DJ, Smucker WD (1997) Presentation and management of childhood psychosocial problems. J FamPract 44: 77-84.
20. Bernal P (2003) Hidden morbidity in pediatric primary care. Pediatr Ann 32: 413–418.
21. Lavigne JV, Binns HJ, Arend R, Rosenbaum D, Christoffel KK, et al. (1998) Psychopathology and health care use among preschool children: A retrospective analysis. J Am Acad Child Adolesc Psychiatry 37: 262-270.
22. Epstein NB, Baldwin LM, Bishop DS (1983)The McMaster Family Assessment Device. Journal of Marital and Family Thereapy 9: 171-180.
23. Essau CA, Sasagaw S, Frick PJ (2006) Psychometric properties of the Alabama Parenting Questionnaire. Journal of Children and Family Studies 15: 597-616.
24. Jellinek MS (1982) Sounding board. The present status of child psychiatry in pediatrics. N Engl J Med 306: 1227-1230.
25. Costello EJ (1986) Primary care practice and child psychopathology: A review of diagnostic, treatment, and referral practices. Pediatrics 78: 1044-1051.
26. McCue Horwitz S, Leaf PJ, Leventhal JM (1998) Identification of psychosocial problems in pediatric primary care: Do family attitudes make a difference? Arch PediatrAdolesc Med. 152: 367-371.