Journal of Addictive Behaviors and Therapy Open Access

Keywords

Children; Youth; Substance use; Depression; Anxiety; Suicidality; Brief intervention; Treatment; Screening

Introduction

The rate of substance misuse may be increasing [1], with youth between the ages of 15 and 24 yrs exhibiting the highest rate of substance misuse [2]. Unfortunately, alcohol and/or substance use during adolescence increases the likelihood of developmental delays in both social and academic capacities, particularly those associated with cognition, motivation, and impulse control [3-5]. In addition, short and long-term consequences beyond mental health problems include decreased school achievement, increased victimization and suicide attempts, and subsequent mortality [6]. Previous Canadian studies have suggested that at least 25% of students in Grades 6-12 (ages 11-18) use alcohol, of whom nearly half binge drink (defined as consuming more than 5 alcoholic drinks during one occasion) [7]. This percentage increases with age, with 86% of Grade 12 students (aged 17-18) consuming alcohol, and 50% of students having used an illicit drug in the previous 12 months [7]. These findings support the need for effective tools to reduce abuse of alcohol and drugs in youth populations.

Amidst a variety of proposed interventions [8,9], one suggested approach for youth is to utilize a combination of universal Screening, Brief Intervention for those at high risk, and Referral to Treatment (SBIRT) when any such requirement is identified [10]. In principal, SBIRT may be an effective approach that can be used in sites such as primary care or schools, and can be used as stand-alone treatment (screening and brief intervention components) for adolescent substance use, or in conjunction with other treatment approaches, such as cognitivebehavioural therapy [11]. SBIRT in youth also has potentially important public health benefits since early identification and treatment of individuals engaging in high risk substance use may delay, and optimally prevent, the onset of substance use problems [10,12]. Although SBIRT has the potential to be an inclusive program for the prevention and intervention of adolescent substance use, implementing this program in the most appropriate settings, primarily within schools and possibly also in primary care, has proved challenging [10,11]. For this reason, determining the best approach and setting for substance use prevention, and intervention, in youth attending schools remains uncertain. Several key issues arise when considering the most appropriate methods to utilize SBIRT most effectively, including the choice of screening tools, the nature of brief interventions, and the best methods for subsequent referral to treatment.

Components of SBIRT

Screening tools used to assess risk of developing a substance use disorder

While there are several screening tools available for youth, most focus on alcohol use and not overall substance use. Three of the most widely used alcohol-centred screening tools are the AUDIT (The Alcohol Use Disorders Identification Test), RAPS-QF (Rapid Alcohol Problems Screen-Quantity Frequency), and the CAGE questionnaire (Cut, Annoyed, Guilty, and Eye-Opener) [13]. In addition, the POSIT (Problem Oriented Screening Instrument for Teenagers) is used to evaluate 10 dimensions of health, including substance use [14]. Nonetheless, it is important to note these screening tools, with the exception of the POSIT, were not specifically designed for use in youth, and some of the components included in these questionnaires are not readily transferable to youth.

In contrast to these alcohol-centred screening tools there are tools designed to measure substance abuse more widely, including the CRAFFT (named after the focus of each of the 6 questions-Car, Relax Alone, Forget, Friends, and Trouble) [15] and the ASSIST (Alcohol, Smoking and Substance Involvement Screening Test) [16]. When comparing these two, ASSIST is a screening tool not specifically designed for youth populations, consists of more than 80 questions, and has a more complex scoring procedure than the CRAFFT [17]. The CRAFFT is a 6 item screening tool that can assess lifetime and current substance misuse, and is specifically designed for youth populations [18]. The questions are answered dichotomously (yes/no), and each positive answer is scored as one (1) point, with a maximum score of 6. Individuals who score ≥ 2 are likely to be at elevated risk of developing a substance use disorder [18,19]. For these reasons, the CRAFFT has been widely used in adolescent populations [19] and has discriminant properties that can assist clinicians with their assessment of severity of substance use [20,21]. Studies have also examined its effectiveness in those youth presenting to Emergency Rooms of hospitals, although there is low predictive value for abuse and dependence overall when used in such situations [22]. This may be due to reluctance to disclose misuse of substances in acute care settings. More success from screening using CRAFFT has been found in primary care in some studies [23], and it has been recommended for this purpose [24] although it isn’t widely used [25]. Further, without the use of appropriate tools, such as the CRAFFT, the severity of the substance use problem is frequently underestimated [26]. Indeed, there is evidence that the CRAFFT is a useful gauge of substance misuse and has been recommended for routine use with adolescents in the 2011 guidelines presented by the American Academy of Pediatrics [5,27]. Barriers to more widespread use in primary care include insufficient training, knowledge of screening tools, and treatment resources, followed by attention to competing medical issues and insufficient time [28].

Brief interventions used as initial treatment for substance use disorder

Brief interventions are usually described as time-limited sessions, often administered in person with the principal goal to motivate an individual to progress from pre-contemplating change (no concern or thoughts regarding substance using behaviour) to actually contemplating change (beginning to realize potential risks or consequences) [29]. One of the most widely used brief interventions in SBIRT is motivational interviewing and enhancement therapy [30,31]. Motivational interviewing is described as allowing the adolescent and interviewer to establish a working alliance through unconditional positive regard (avoiding judgment and displaying acceptance), empathy, and support in order to examine the adolescent’s feelings of uncertainty, or ambivalence, toward changing substance misuse behaviour [32,33]. The inclusion of an assessment component to motivational interviewing creates the counselling approach known as motivational enhancement therapy [32,34]. This encourages adolescents to move toward harm reduction or abstinence from substance use [35,36]. Traditionally, SBIRT uses motivational interviewing and motivational enhancement therapy for the brief intervention; however; other therapeutic interventions have been used, such as cognitive behavioural therapy (CBT) [37-39]. Despite many findings of the effectiveness of SBIRT in adult populations [30,40-43], there is some evidence of ineffectiveness of SBIRT on adult drug use [6,44]. Further, there is a paucity of evidence regarding the potential positive impact of SBIRT as an integrative program for youth [27]. In the only review to date of 15 relevant studies [45], six targeted adolescent alcohol use only, another 3 targeted only marijuana use, one study targeted alcohol, cocaine and ecstasy, while one study examined the impact on a complex high needs youth population. The remaining 4 studies included some control groups, and of the 15 studies included in the metaanalysis, 7 examined components of SBIRT in school-based settings [46-52]. The 7 SBIRT studies had significant methodological issues, including small sample sizes, different age groups being studied, different screening tools being used and different interventions being offered. It is perhaps not surprising that the findings between studies varied widely in terms of effectiveness [45]. Another recent review focussing on SBIRT identified similar findings and concerns, although that review focused on implementation of SBIRT within primary care settings [53]. Currently, there is no clear consensus regarding the most efficacious brief intervention to use with youth.

Referral to treatment pathways

Although SBIRT demonstrates potential as a comprehensive public health approach to the prevention and early identification of substance use problems, there is limited evidence of formal evaluation of this approach. Additionally, while individual components, such as screening and brief intervention have been assessed, a comprehensive program has not been adequately studied in youth and there remains significant lack of information necessary to determine the key elements of any such program [54]. In addition, there is very limited information on the outcomes of adolescents who are subsequently referred for treatment, either to primary care or to specialty addiction clinics. Furthermore, there are major concerns regarding the cost implications of treating newly identified individuals following screening [6,55,56]. All of these factors represent potentially significant barriers to developing effective SBIRT approaches in youth.

Rationale for present study

Taken together the literature suggests there is a need for wellstudied interventions that can potentially reduce substance misuse in youth. Ideally, these could be administered within school settings as this is likely to impact the largest number of youth. However, it is difficult to carry out randomized controlled trials (RCTs), the proposed gold standard, to examine possible impacts. Nonetheless, it is possible that SBIRT may offer help provide a solution, but currently there is limited evidence to support this approach. In the present publication we utilize an observational approach to examine the effectiveness of SBIRT that was contained within a larger school-based program, termed Empowering a Multimodal Pathway towards Healthy Youth (EMPATHY). Previously we have published baseline and 3 month data that show the large number of students that use drugs and alcohol; however, findings were based on a created screening tool that is not yet validated for use [57]. Here we present data on a 15 month follow-up of a larger cohort, specifically examining CRAFFT scores for students who took part in the EMPATHY program.

Methods

Program location, timing, and participants

The current data are from a program in Red Deer, Alberta, Canada, a large urban city with a population of approximately 100,000 people. The observational study was carried out in all of the 9 schools educating those aged 11-18 (Grades 6-12) located within a single school district. Schools included; 3 Middle Schools for those aged 11-14 (Grades 6-8); 3 schools which were Kindergarten-Grade 8 for those aged 5-14, but only those in Grades 6, 7 and 8 at these schools were included in the study; also included were 1 special school for those aged 15-18 (Grades 9-12); and 2 High Schools for those aged 15-18 (Grades 9-12). Here we present the longer-term outcomes, which consisted of data collected by the school district at 4 separate assessment time-points from February 2014 until June 2015.

Multimodal approach

This study utilized several novel approaches, which were detailed in a previous publication [57]. In brief, the EMPATHY program collected data on electronic tablets linked to a private school intranet for screening, on-going assessments, and intervention. All staff hired for the program had experience working with youth but were deliberately chosen not to be highly qualified individuals (thus excluding registered psychologists, for example). This was to determine if the EMPATHY program could be successful with staff who could potentially allow the program to scale up, recognizing the potential shortage (and cost) of more highly trained individuals. These individuals were termed “Resiliency Coaches”, and each was attached to a specific school, but was not therapists, and did not act in this role. When significant concerns were noted during screening, students and their families were contacted. For the most significant situations this was within 48 h of screening in almost every case, and as rapidly as a few hours after screening, if required.

Screening

All screening was carried out on dedicated electronic tablets during a standard classroom session. Students logged on using only their student IDs. Electronic data collection complied with all privacy and security requirements. The individual scales were presented to students in a randomized order, and no data were stored on the tablets as they were directly linked to the school intranet. The data were stored in a dedicated and secure database and was immediately available to the study staff to those flagged as “at risk”. It should be noted that students were identifiable to study staff only by a unique study number assigned when the screening began, and if the student was flagged as high risk, then the study number was communicated to the school staff who could determine the student identity. This was the only time that information about individual student results and scores was available to school staff. Apart from these specific instances, information about individual student results and scores was not available to school staff.

Screening tools

All of the screening tools were brief and easily adapted to the “app” used on the electronic tablet. To determine use of drugs, alcohol, and tobacco we used 11 questions in a novel scale (DAT) (Table 1). Of these 11 questions, 9 were from CRAFFT but it should be noted that in scoring the CRAFFT only 6 questions are actually used (questions 4-9 in Table 1, shown in bold), with the 3 initial CRAFFT screening questions (questions 1-3 in Table 1) not being used for scoring the CRAFFT. Previously we have reported baseline findings from the overall 11 item DAT scale [57], but in the present publication we have extracted the scores for just the 6 questions that form the CRAFFT to allow more direct comparison to previous research findings. It is also very important to note that the CRAFFT questions ask about use “during the past 12 months”. Therefore, although we collected data at 3 months, 7 months, and 15 months, the only time at which statistical comparisons were made were between Baseline ratings and at 15 months (Assessment #1 and Assessment #4); and between 3 months and 15 months (Assessment #2 and Assessment #4). In addition to the CRAFFT, four screening tools were used to assess risk of developing a mental illness, such as the Patient Health Questionnaire (PHQ-9/ PHQ-A) to screen for depression and/or suicidality; the Hospital Anxiety and Depression Scale (HADS) to screen for anxiety; the Rosenberg Self-Esteem Scale; and KIDSCREEN-10 to screen for quality of life. The findings from these additional screens in relation to CRAFFT scores are outside of the scope current paper and, as such, will be reported elsewhere.

Source of question	Question Number	Individual Questions	Scoring Range for each question
CRAFFT screening	1	During the past 12 months, did you drink any alcohol (more than a few sips)?	0 or 1
CRAFFT screening	2	During the past 12 months, did you smoke any marijuana or hashish?	0 or 1
CRAFFT screening	3	During the past 12 months, did you use anything else to get high?	0 or 1
CRAFFT (C)	4	During the past 12 months, have you ever ridden in a CAR driven by someone (including yourself) who was "high" or had been using alcohol or drugs?	0 or 1
CRAFFT (R)	5	During the past 12 months, do you ever use alcohol or drugs to RELAX, feel better about yourself, or fit in?	0 or 1
CRAFFT (A)	6	During the past 12 months, do you ever use alcohol or drugs while you are by yourself, or ALONE?	0 or 1
CRAFFT (F)	7	During the past 12 months, do you every FORGET things you did while using alcohol or drugs?	0 or 1
CRAFFT (F)	8	During the past 12 months, do your FAMILY or FRIENDS ever tell you that you should cut down on your drinking or drug use?	0 or 1
CRAFFT (T)	9	During the past 12 months, have you ever gotten into TROUBLE while you were using alcohol or drugs?	0 or 1
	10	During the past 12 months, did you smoke tobacco products?	0 or 1
	11	During the past 12 months, did you use smokeless tobacco products?	0 or 1
		Unmodified maximum possible raw score=	11
		Maximum possible CRAFFT score=	6

Table 1: List of questions asked in Drugs, Alcohol, and Tobacco (DAT) scale.

Identification of high risk group (Top 10%)

Following screening we identified a “High Risk” group. These were either that who were actively suicidal, or who were in the highest 10% of combined scores for their grade at their school [57]. This combined score was termed the EMPATHY scale score, and each of the 5 scales was given equal weight in the final score (i.e., each scale, including DAT scores, comprised 20% of the total score, which was normalized to be out of 100). Identifying the 10% with the highest overall scores was based on previous studies suggesting that up to 10% of students aged 10-18 may have significant symptoms of depression [58,59] and also that youth who have “sub-threshold” depression may also be at raised risk of self-harm and suicide attempts [60-62]. This group may also have increased use of drugs and alcohol [63].

Brief interventions offered in school

For those identified as being in the High-Risk group, specific additional parental and student written consent was required for the brief interventions. These were existing internet-based CBT programs, either one focused on depression (“This Way Up”) [64], or for those who had potential addiction issues a program more focused on addictions was offered (“Breaking Free”) [65], although all youth could choose which of the two they preferred. These programs were administered in a “guided” manner, in which the Resiliency Coach would ask the student how they found each section, and ask general questions, but not try and provide therapy themselves. Previous studies have suggested that the number of students who take up such opportunities can be limited for a variety of reasons [66], so we were uncertain how many students in the High-Risk group would actually use the brief interventions offered.

Additionally, a universal preventative CBT program, called OVK after its Dutch name (Op Volle Kracht-translated as “At Full Power”) [67-70], was given to some of the younger students. There were 16 OVK sessions offered during class time focused on understanding the connection between thoughts, feelings, and behaviours, while providing instruction on social norms and social competence behaviours (i.e., assertiveness, resistance training, and stress relief methods). This was completed during regular classroom time usually devoted to “health” topics. During the first school year of the EMPATHY program from February-June 2014 only 8 CBT sessions were given to all students in Grades 7 and 8 (ages 12-13). In the second school year, from September 2014-June 2015 the full 16 sessions (including 8 CBT) were given to all students in Grades 6-8 (ages 11-13). Those administering the program were specifically trained by individuals involved in developing the OVK program [57].

Referral to treatment

Close communication between the schools and both the local primary care practice, and the local specialty youth psychiatric services, increased shared decision making and the appropriateness of referrals. This was both in terms of which students were referred and where students were most appropriately referred to. In the present study we worked closely with both providers of primary care (both primary care physicians, nurses, and other therapists working within these clinics) as well as with specialist child and youth psychiatric services. For primary care there was specific training on the use of CBT in youth, and potential uses of medication when appropriate. However, in both instances treatment was the current standard of care they were currently using and there were no forms of treatment specifically mandated in this program. Careful tracking of all referrals to both primary care and specialist mental health care during the program were made, as well as a commitment by the study team that (if required) additional staff resources could be made available to help with any additional needs following identification of youth at need.

Statistical analysis

Cross-sectional tests: To test the equality of means from two independent groups, we used the two-tailed Wilcoxon rank-sum test (also known as the Mann Whitney U test) to test hypotheses on the differences between mean scores for different stages of screening, since our data displayed a non-normal distribution. The Mann Whitney U test is a nonparametric test which is often reported to test differences in medians [71,72]. This may prove problematic since comparing two groups could yield the same median, but demonstrate a Mann Whitney U test that is significant [71,72]. Essentially, if the sample distribution of each group is similar, the location shift will move both medians and means by the same amount [71,72]. This indicates that the Mann Whitney U test is a statistical process that can account for differences in means [71,72]. As such, the results reported in the present paper describe differences in means rather than medians. The test was performed on all students screened. In addition, Chi-square (or Fisher’s Test when n<10) was used to test the difference between expected and observed frequencies of the number of students at risk of a substance use disorder. Effect sizes for Chi-Square tests were calculated using the Phi- Coefficient , and effect sizes for the non-parametric Mann Whitney U Test (Wilcoxon rank-sum test) were calculated using Pearson’s Coefficient (r). Statistical significance varied for each set of data with the application of the Bonferroni correction for multiple comparisons.

Longitudinal tests: For those students who completed all 4 ratings, the statistical method was a paired design, in which each student who completed both baseline ratings and followup ratings was their own control. Similarly, since the data showed evidence of non-normality, a non-parametric test was carried out to compare the differences between the mean scores at baseline and each follow-up. This involved the twotailed Wilcoxon match pair signed-rank test, unless otherwise specified. In addition, McNemar’s Test was used to calculate the difference between expected and observed frequencies in longitudinal study groups. Calculating the effect size for McNemar’s test using the odds ratio to subsequently determine the Cohen’s d value was problematic due to a discordant value, therefore effect sizes are not reported for these results. The effect sizes are reported through Pearson’s Coefficient (r) for results calculated using the Wilcoxon Signed Rank Test. Statistical significance varied for each set of data with the application of the Bonferroni correction for multiple comparisons.

A power analysis had been completed prior to the initial EMPATHY study [57], which determined that the study was adequately powered at the current sample sizes.

Statistical analysis was carried out on an “intention to treat” basis utilizing R, version 3.1.0 (R Foundation for Statistical Computing, Vienna, Austria).

Results

From a total of 6,227 students, who were assessed at least once (Figure 1), 15,830 assessments were completed. Assessment #1 (Baseline) occurred during February and March 2014 (n=3244); Assessment #2 was the 3 month follow-up screening which occurred during May and June 2014 (n=3229); Assessment #3 was the 7 month follow-up screening which occurred during September and October 2014 (n=4860); and Assessment #4, the final 15 month follow-up assessment, occurred during the period April to June 2015 (n=4497). The timing of the follow-up screenings was based around the school year, with Assessments #1 and #2 in school year 1, and Assessments #3 and #4 occurring in school year 2, with all schools being on vacation in July and August 2014. Note that only 5 schools took part in school year 1; three Middle Schools and 2 High Schools. Of these students, a total of 6,227 students completed at least one assessment, 4,917 completed at least 2 assessments, and 2,796 completed at least 3 assessments (Figure 1). A total of 1,884 students completed all 4 assessments over both years, and this group consisted only of students in Grades 6-11 from the first school year: they could not be in either Grade 6 in school year 2 (as they would be new to the system) or Grade 12 in school year 1 (as they would have graduated before school year 2).

Figure 1: CONSORT 2010 Flow Diagram.

In the present study we carried out a cross-sectional statistical analysis comparing the Baseline group (n=3224), who were in 5 separate schools, with the total sample at 15 months (n=4497), who were at 9 separate schools. We also performed a longitudinal analysis of the data comparing Baseline findings to 15 month findings in the 1,884 students who completed all assessments and were therefore their own controls. Note that due to the significant amount of data, the key findings are shown in the Tables in the paper, with additional information being shown in Supporting Information tables (show with an ‘S’ in front of the table number).

CRAFFT Substance use scores: Cross-sectional results

As anticipated, with increasing age there was a clear increase in the percentage of students who scored ≥ 2 on the CRAFFT, indicating an increased likelihood of substance abuse as students got older (Table S1). This showed an increase particularly when students entered High School (increasing from 4% at Baseline in Grade 8 to 15% at Baseline in Grade 9, mean age 13.3 and 14.3 yrs old respectively), and this percentage increased again in Grade 11 to 24% at Baseline (mean age 16.4 yrs old), and even further to 31% in Grade 12 (mean age 17.4 yrs old). This increase was consistent at all Assessments and was seen even more clearly with those who scored in the range of 3-6 on the CRAFFT (Table S1).

Upon closer examination of mean CRAFFT scores and number of students at risk of developing a substance use disorder by individual grade, a change in mean scores is not evident (Table S2). Despite this, a highly significant decrease in the number of students at risk is observed for Grades 7,9,11 and 12 when comparing Baseline to 15 month follow-up (Assessment #4) p<0.001, as well as comparing the 12 month interval between Assessment #2 and Assessment #4, p<0.001 (Table S2). The size of the difference is most notable for Grade 9 ( = 0.23), demonstrating a small to medium effect size over the 15 month interval from Baseline to Assessment #4 (Table 3). Small effect sizes are noted for Grades 7,11 and 12 (Table S2).

	Baseline Assessment #1 (B) (n=3,244) Year 1	3-month Assessment #2 (A2) (n=3,228) Year 1	7-month Assessment #3 (A3) (n=4,856) Year 2	15-month Assessment #4 (A4) (n=4,496) Year 2
Mean Substance Use Score (±SD)	3.00 (± 1.05) (95%CI = 2.90 – 3.10)	3.01 (± 1.11) (95%CI = 2.90 – 3.12)	3.14 (± 1.14) (95%CI = 3.02 – 3.26)	3.09 (± 1.07) (95%CI = 2.97 – 3.21)
Risk of Substance Use Disorder (n)	443	404	320	291 ***+++ B – A4 =0.120 A2 – A4 =0.120
Proportion at risk (%)	0.137	0.125	0.066	6.50%
Males at risk (n)	216	179 ++	150	128 ***+++ B – A4 =0.127 A2 – A4 =0.093
Proportion of males at risk (%)	0.49	0.444	0.469	0.449
Females at risk (n)	225	224	170	157***+++ B – A4 =0.115 A2 – A4 =0.114
Proportion of females at risk (%)	0.51	0.556	0.531	0.551
1Decreased scores show improvement 2Note that scores for risk of substance misuse varied from 2-6 3Statistical significance is set at α=0.001 after application of Bonferroni correction to account for multiple comparisons 4Effect size for Chi-square is depicted by  (Phi Coefficient): 0.1 (small), 0.3 (medium), 0.5 (large) 5Effect size for Mann Whitney (Wilcoxon Rank Sum Test) is depicted by r (Pearson r Coefficient): 0.1 (small), 0.3 (medium), 0.5 (large)

Table 2: Changes in substance misuse scores for entire cross-sectional cohort^1,2,3,4,5.

	Baseline Assessment #1 (B) (n, n%)	3-month Assessment #2 (A2) (n, n%)	7-month Assessment #3 (A3) (n, n%)	15-month Assessment #4 (A4) (n, n%)
CAR	267, 60.3%	239, 59.2%	225, 70.3%	198, 68.0%
RELAX	191, 43.1%	174, 43.1%	156, 48.8%	135, 46.4%
ALONE	92, 21.8%	68, 16.8%	60, 18.8%	40, 13.7%
FORGET	274, 61.9%	222, 55.0%	205, 64.0%	179, 61.5%
FRIENDS/FAMILY	251, 56.7%	204, 50.5%	204, 63.8%	174, 59.8%
TROUBLE	253. 57.1%	218, 54.0%	183, 57.1%	173, 59.5%

Table 3: Number of affirmative responses on the CRAFFT Substance Use Screening Tool for the cross-sectional cohort.

Table 4 displays aggregated scores for all Grades and a highly statistically significant reduction in the number of students at risk of a substance use disorder from Baseline to 15 months, as well as the 12 month interval between Assessment #2 and #4. The effect sizes for the decreased number of students at risk of a substance use disorder over the course of the 15 month EMPATHY program are small, ranging from ¸=0.09-0.13 (Table 2).

	Baseline Assessment #1 (n=1,884) Year 1	3-month Assessment #2 (n=1,884) Year 1	7-month Assessment #3 (n=1,884) Year 2	15-month Assessment #4 (n=1,884) Year 2
Mean Substance Use Score (± SD)	2.90 (± 1.00) (95%CI=2.74–3.06)	2.13 (± 1.46) (95%CI=1.90–2.36)	2.32 (± 1.56) (95%CI=2.07–2.57)	2.29 (± 1.67)*** (95%CI=2.03–2.55) r = 0.247
Risk of Substance Use Disorder (n)	154	101	100	99 ***
Proportion at risk (%)	0.082	0.054	0.053	0.053
Males at risk (n)	78	54	51	50 ***
Proportion of males at risk (%)	0.506	0.535	0.51	0.505
Females at risk (n)	76	47	49	49 ***
Proportion of females at risk (%)	0.494	0.465	0.49	0.495
1Decreased scores show improvement 2Note that scores for risk of substance misuse varied from 2-6 3Statistical significance is set at α=0.001 after application of Bonferroni correction to account for multiple comparisons 4Effect size for Wilcoxon Signed Rank Test is depicted by r (Pearson r Coefficient): 0.1 (small), 0.3 (medium), 0.5 (large) *** p<0.001 compared to Baseline

Table 4: Changes in substance misuse scores for longitudinal cohort^1,2,3,4.

It is interesting to note that, while there was a decrease in the number of students at risk, there was no overall decrease in mean CRAFFT scores over time that was statistically significant with the mean being 3.0 ± 1.05 (95%CI = 2.90–3.10) at Baseline and 3.1 ± 1.07 (95%CI = 2.97–3.21) at 15 months (Table 2 and Figure 2).

Figure 2: Mean CRAFFT Score for students at risk of a substance use disorder over 15 months.

In addition, we found that there were slightly more males than females who scored at least ≥ 2 on the CRAFFT (Table 2). This ratio changed significantly at Assessment #2 and Assessment #4 (Table 2) and was also reflected in those with the highest CRAFFT scores (Table S1). Changes in the number of males and females from Baseline to 15 months, as well as the 12 month interval between Assessment #2 and Assessment #4 were found to be statistically significant (Table 4). Figure 2 shows mean CRAFFT scores at each Assessment for all students (Baseline - n=3,244; Assessment #2 - n=3,229; Assessment #3 - n=4,860; Assessment #4 - n=4,497). There were no statistically significant differences between the mean score at Baseline and at 15 months.

It was of interest to determine if the range of behaviours changed over time following interventions. This is shown in Table 5, where the number of students answering in the affirmative (i.e., agreed with the question) for each cohort was examined to determine if the rates of answers to specific items varied between Assessments. For the individual questions of the CRAFFT, described fully in Table 1, there are noteworthy changes observed over time; however, these changes in affirmative responses were not statistically significant (3).

	Baseline Assessment #1 (n, n%)	3-month Assessment #2 (n, n%)	7-month Assessment #3 (n, n%)	15-month Assessment #4 (n, n%)
CAR	76, 49.4%	74, 48.1%	81, 52.6%	80, 51.9%
RELAX	55, 35.7%	45, 29.2%	52, 33.8%	58, 37.7% +++
ALONE	34, 22.1%	22, 14.3%	27, 17.5%	14, 9.1%***
FORGET	95, 61.7%	64, 41.6%	65, 42.2%	64, 41.6%***
FRIENDS/FAMILY	92, 59.7%	61, 39.6%	68, 44.2%	65, 42.2%***
TROUBLE	95, 61.7%	62, 40.3%	64, 41.6%	71, 46.1%***
1Statistical significance is set at α=0.003 after application of Bonferroni correction to account for multiple comparisons *** p<0.001 compared to Baseline +++ p<0.001 comparing Assessment #2 to Assessment #4

Table 5: Number of affirmative responses on the CRAFFT Substance Use Screening Tool for the longitudinal cohort¹.

CRAFFT Substance use scores: Longitudinal results

As with the cross-sectional analysis, there was a clear increase in the percentage of students who scored ≥ 2 on the CRAFFT as students got older, indicating an increased likelihood of substance abuse with age in the longitudinal cohort (Table S3). This showed an increase particularly when students entered High School (increasing from 4 % at Baseline in Grade 8 to 12% at Baseline in Grade 9, mean age 13.3 and 14.3 yrs old respectively), and this percentage increased again in Grade 11 to 20% at Baseline (mean age 16.4 yrs old). This increase was consistent at all Assessments and was seen even more clearly with those who scored in the range of 3-6 on the CRAFFT (Table S3).

Similar to the cross-sectional cohort, a highly significant decrease in the number of students at risk of a substance use disorder is observed for students in Grades 9 and 10 at Baseline (Table S4). In contrast to the results in the cross-sectional cohort, a highly statistically significant decrease in mean CRAFFT scores is observed for Grade 9 (shifting to Grade 10 in Year 2), p<0.001 (Table S4). The effect size of this change in mean score for Grade 9 is noted to be medium to large (r = 0.34) (Table S4).

Although the focus of this study is to examine changes in substance use scores, some students in the longitudinal cohort scored at risk of potential comorbid conditions, such as suicidality, depression, and/or anxiety and were classified as having an EMPATHY score in the Top 10% within their Grade and school. As such, they were invited to participate in the Selective Brief Intervention (online CBT,“This Way Up”). Out of the 154 students with a CRAFFT score ≥ 2 at Baseline, 35 students (23%) scored in the Top 10% and were offered the intervention (Table S5). Five students (14%) participated in the online CBT program; however, the decrease in mean CRAFFT scores did not reach statistical significance (Table 8). Further, although a decrease in mean CRAFFT scores is observed for the remaining 30 students (86%) who did not participate in the intervention, this decrease was also not statistically significant (Table S5).

Interestingly, the remaining students in the cohort, who were not classified as being at risk of comorbid conditions and thus were not in the Top 10% (n=119; 7%), demonstrated a reduction in mean CRAFFT scores that is highly statistically significant (Table S6, ***p<0.001). In addition, the effect size of this change in mean scores was observed to be small to medium (r = 0.28) (Table S6).

Students in Grades 7 and 8 in Year 1, and Grades 6–8 in Year 2 participated in the Universal Prevention program OVK. Some of these students were found to have EMPATHY score in the Top 10% (n=6) and were also invited to participate in the Selective Intervention (“This Way Up”). Table S7 indicates that out of the 6 students offered the Selective Brief Intervention, 2 students participated in the program. Although there is a clear reduction in mean CRAFFT scores in both groups-individuals who participated in the intervention vs. those who did not-the results are not statistically significant (Table S7).

Further, students in Grades 6-8 who were at risk of a substance use disorder but did not score in the Top 10% (n=10) also demonstrated a noteworthy reduction in mean CRAFFT score; however, as with previous results, this reduction was not statistically significant (Table S8).

Table 4 displays aggregated scores for all Grades and a highly statistically significant decrease over time in the percentage of students who scored ≥ 2 on the CRAFFT. This clear reduction in the number of students at risk of a substance use disorder is statistically significant from Baseline to 15 months (Table 4). However, unlike the cross-sectional cohort, a statistically significant reduction in the number of students at risk of a substance use disorder was not seen in the 12 month interval between Assessment #2 and #4 (Table 4).

There was a statistically significant decrease in the mean CRAFFT scores over time, from a mean of 2.90 (± 1.00; 95%CI = 2.74–3.06) at Baseline to 2.29 (± 1.67; 95%CI = 2.03–2.55) at 15 months with a small to medium effect size (r=0.25) (Table 4). While a reduction is noted, it appears that most of this change was seen in the first 3 months (Figure 3).

Figure 3: Mean CRAFFT Score for students with all 4 screening points over 15 months.

Table 4 also shows the percentage of males and females at risk of a substance use disorder. It can be seen that there were no statistically significant changes in the sex ratio within each Assessment (as determined by the percentage of each sex that was at risk). However, changes in the actual number of male and the number of female students decreased significantly in both sexes from Baseline to 15 months, p<0.001. These results were not observed in the 12 month interval between Assessment #2 and Assessment #4.

As with the cross-sectional cohort, it was of interest to track possible changes in substance use behaviour through affirmative responses on the CRAFFT Substance Use Screening Tool. There were statistically significant changes over time for all questions, with the exception of riding in a car operated by someone under the influence of a substance or using substances to relax or fit in, between Baseline and 15 months (Table 5).

Discussion

The present observational study indicates the potential usefulness of a screening, brief intervention, and referral to treatment (SBIRT) program in youth populations in a school context. A particular strength of this observational study is that it was carried out across a complete school district and involved a large number of students. We found high rates of substance abuse, particularly in High Schools, although during the program the actual number of students requiring referral for any mental health reason (including addictions) was only 2% (65 students) in the first 3 months of the program [57]. This is somewhat reassuring, in that one of the significant potential barriers to a more widespread implementation is the fear that by screening large populations an undue workload will be placed on existing services. In the present study this was mitigated by the selective brief intervention given in schools (having “guided” internetbased CBT). This may be an approach that will also prove useful in other programs; however, our results indicate that screening alone may be a potential catalyst to changing the perception of normative behaviour within schools, subsequently decreasing the prevalence and severity of substance misuse. Although our results have not shown a decrease in substance using behaviour by those experiencing a brief intervention (OVK as the universal intervention for Grades 6-8; This Way Up as the selective brief intervention for those with an EMPATHY score in the Top 10%), it would be premature to discredit the potential value brief interventions may have since the number of students who participated was very small. There are various reasons why adolescents at risk may not engage in programs addressing substance misuse, including fear of exposure to parents and subsequent repercussions; developmental recalcitrance; and denial of consequences related to risky behaviour [27,66,73]. Specifically, in our cohort, we found obtaining parental consent to be a consistent barrier preventing students from participating in the brief intervention. Future research focusing on programs that do not require parental consent may provide further insight into both behavioural and attitudinal components of adolescent engagement, as well as an ecological view of potential barriers external to the adolescent (i.e., family, community, and agency).

Of interest was the finding that specific answers to the different CRAFFT questions changed significantly over time, and it appears that not all behaviours are equally impacted by the present program. Knowing this, future programs may want to specifically address those areas that changed the least such as the continued riding in a car with somebody who was “high” or using drugs or alcohol to relax.

In the present study, 14% of students between the ages of 11 and 18 scored ≥ 2 on the CRAFFT, indicating a possible increased risk of future substance abuse. The results also confirm that rates of substance misuse may increase dramatically over 13 yrs of age, in accord with previous studies [74,75]. This is clearly of concern given the increasing evidence of vulnerability of the developing brain to impacts of alcohol and drug use and, consequently, possible impact on behaviours related to motivation and impulsivity [76,77].

Specifically, our results confirm previous findings that students in Grades 9-12 (ages 14-17) engage in more high-risk substance use than younger school-age groups, and also show that the rate of substance misuse may double between Grade 9 (14 yrs old) and Grade 12 (17 yrs old). Linked to this, we observed a marked increase in substance abuse in the year following Middle School (Grade 8) to the first year of High School (Grade 9). It should be noted that in this specific school district (and similar to the United States) High School includes 4 grades. It is not certain if the same increase at the same time would have been seen in those school districts in which High School is only 3 yrs (Grades 10-12). Also of interest was the finding that, while males exhibited a higher risk, the size of this gender difference was not as large as reported in previous studies [78-80]. We are not certain why this may be, but it will be important to determine in future large studies, as it may support adult studies suggesting that that the risk of alcohol and drug use for women may be increasing.

On a more positive note, the present data also showed a statistically significant and clinically meaningful reduction in the rate of those scoring ≥ 2 on the CRAFFT. This rate decreased to only 7% 15 months following the implementation of the EMPATHY program. This is approximately 50% of the Baseline rate in this population, which suggests that, at the very least, the screening component of an SBIRT program in schools can meaningfully impact risk of drug and alcohol abuse. This was the primary goal of the study, with the size of the improvement being greater than anticipated [57].

There are limitations to this observational study. Among these, for ethical reasons, this program could not incorporate a control group since doing so would have required randomization of students who were actively suicidal to non-intervention groups, which is clearly unethical [57]. Additionally, in terms of the misuse of substances, the present study relied on self-report scales, and so there was no objective measurement of substance use. However, previous studies have supported the relative reliability of youth self-reports of substance use and other high-risk behaviours [81-83]. A further potential study limitation was that the treatment received after referral was non-standardized, being treatment as usual by the individual physician or care team, both in primary care and in specialist care.

In conclusion, in this publication we present observational data on a large community sample of students aged 11–18 yrs old and report high rates of substance misuse, which increases with age. We also found a major decrease in the risk of abuse following the EMPATHY program, which entailed a form of SBIRT. These findings also support the potential utility of such approaches in other programs and we hope that this research provides support for future endeavours and effective knowledge translation. The importance of effective preventative programs for substance abuse should not be minimized, and the present study findings support proposals that SBIRT may be an appropriate approach for school-aged children and youth, particularly when incorporated into more comprehensive and multimodal programs.

Declarations

Ethics approval and consent to participate

It is important to note that the EMPATHY program and our observational study data reported in this publication, was from a standard school curriculum and was not a research program. It is important to clarify why, and to emphasize that, as clarified with the University of Alberta Ethics Review Board, because the screening process was not part of a research study (since this was part of existing curriculum and applied to all students) there was no informed consent requirement. However, analysis of data collected, did require research ethics approval (which was appropriately obtained), as did any individual additional interventions (which was also appropriately obtained).

The school district was clear that programs that they implement as part of their curriculum on district-wide basis did not need specific parental consent. This was based on the fact that education for any subject matter within these schools, even areas that may be considered controversial (such as discussions about sexual matters or sexual orientation, or ethical matters such as physician-assisted suicide) do not require parental consent. Thus, they did not believe that teaching students on a universal, district-wide basis about mental health issues required a different process than any other subject. Similarly, they asserted that assessing students on a range of topics is done frequently, again, on a universal, district-wide basis, and the assessments in the screening were not something for which specific consent was required and antithetical to their processes. Consistent with their normal processes, they sent a general information letter to all parents about the new program, informing the parents/guardians that the school district was introducing a school district-wide “Resiliency Project” (they didn’t want to use the term “EMPATHY program” as they had recently introduced a different program that had a similar name). The University of Alberta, as well as the Strategic Clinical Network for Addiction and Mental Health of Alberta Health Services, helped design the most appropriate screening tool and intervention program, train teachers about assessment, provide close liaison to clinical services in the region when required, and evaluate the effectiveness of the program. In discussions, the school set a threshold for the combined scores (termed the “EMPATHY score” [57]) to trigger invitation to the “trial” or targeted intervention; this threshold varied by age and by school. This component included a guided internet-based CBT approach. Since this additional support was not part of regular teaching, prior to receiving this additional internet-based CBT, parents or guardians of students provided written consent on behalf of the youth and the student provided written assent, both documents and the process were approved by the University of Alberta Health Ethics review board.

The signed consent and assent forms were kept as part of the trial documentation, in a secure, private location. The data collected by the school as part of their regular classroom assessment was made available for analysis by the research team, but only on an anonymized basis (excepting the Alberta Health Number which, while having no personal identification components, would allow subsequent determination of any attendance at ER for suicidal behaviour). This approach was approved by the Health Research Ethics Committee of the University of Alberta on December 5th, 2013, ethics protocol number Pro00041063. Amendments to the original protocol and consent letters were approved in January of 2014, with all changes to the informed consent letter subsequently approved prior to the start of the program. The first student was screened in February 2014 and the follow-up screening was completed by June 2015. No further student follow-up will be carried out subsequently since program funding was terminated.

This study was registered with ClinicalTrials.gov Identifier: NCT02169960. Although an application for this was completed at the time of the ethics approval, due to an unfortunate administrative oversight, the actual submission to the registration database did not occur until July 2014. This omission was corrected as soon as it was recognized, and was noted in our original publication [57].

From a scientific viewpoint, the most rigorous findings would have come from a randomized controlled study. Indeed, such an approach has been carried out in studies looking at depression rates in students and, in theory; randomization can be carried at the class level or school level. However, this was a school district-wide program that was specifically focused on decreasing the number of youth who were actively suicidal, with assessments of students to measure this. To have considered a program (or study) in which a group of students was assessed, found to be suicidal, but then not offered an intervention would clearly have been unethical. For this reason we determined that the most appropriate methods to analyze the data collected by the school district was by examining data from both a cross-sectional analysis of all students who completed at least one assessment, as well as using a longitudinal analysis for those students who completed all 4 assessments (each student as their own control). This approach to analysis of the data was approved by the Health Research Ethics Committee with subsequent modification in April 2016 to approve secondary analysis of the information obtained from the EMPATHY program (presented here in the present publication) with the ethics protocol number Pro00061164.

Consent for Publication

All parties involved gave full consent for publication.

Availability of Data

The anonymized database is available for other researchers. For access please contact the corresponding author. Prior to release researchers must be approved by Red Deer Public Schools.

Competing Interests

None of the authors have any competing interests in the manuscript.

Funding

The primary funding source was Alberta Health through a specific grant. Funding-in-kind, to support staff, was also provided by Red Deer Public Schools and Alberta Health Services.

Authors Contributions

DMH wrote the first draft and carried out the work as part of her submission for a PhD in Psychiatry. All authors contributed to manuscript review and revisions. MB, VYMS, AA, JG, SH, CP, PL, and KR were all involved in the design and data collection. IC, SC, and CM were all involved in the data analysis. RCE was involved in supporting development of the tools used in the study. PHS was involved in all of the above aspects and was the primary grant holder.

Acknowledgements

This study would not have been possible without the assistance of several individuals at the start of this study (many of whom have now moved positions). These include those in the Red Deer community and in Red Deer Public Schools (Elizabeth Fargey, Phil Penner), in Alberta Health Services (Michael Trew, Dwight Hunks, Noreen McCallum, Marti Rustulka, Tanya Drescher, Chris Wilkes), in the Red Deer Primary Care Network (Lorna Milkovich), at the Central Alberta Child and Family Services Authority (David Tunney), and within Justice and Solicitor General Community Corrections (Kim Fay). We also had some institutional support in Alberta including that of the Science, Policy Practice Network, a multi-sectoral partnership supported in part by the Norlien Foundation of Alberta. We also want to thank Rowella Kuijpers and Karlijn Kindt for training the study team in the correct methodology for administration of the OVK program.

References

1. https://www.statcan.gc.ca/pub/82-624-x/2013001/article/11855-eng.htm.
2. Patton R, Deluca P, Kaner E, Newbury-Birch D, Phillips T, et al. (2014) Alcohol screening and brief intervention for adolescents: The how, what, and where of reducing alcohol consumption and related harm among young people. Alcohol Alcohol 49: 207-212.
3. Carney T, Myers B (2012) Effectiveness of early interventions for substance-using adolescents: findings from a systematic review and meta-analysis. Subst Abuse Treat Prev Policy 7: 25.
4. Curtis BL, McLellan AT, Gabellini BN (2014) Translating SBIRT to public school settings: An initial test of feasibility. J Subst Abuse Treat 46: 15-21.
5. Levy SJ, Williams JF (2016) Substance Use Screening, Brief Intervention, and Referral to Treatment. Pediatrics 138: e20161211.
6. Mitchell SG, Schwartz RP, Kirk AS, Dusek K, Oros M, et al. (2016) SBIRT Implementation for Adolescents in Urban Federally Qualified Health Centers. J Subst Abuse Treat 60: 81-90.
7. www.phac-aspc.gc.ca/hp-ps/dca-dea/publications/hsbc-mental-mentale/risks-risques-eng.php.
8. Toumbourou JW, Hemphill SA, Tresidder J, Humphreys C, Edwards J, et al. (2007) Mental health promotion and socio-economic disadvantage: lessons from substance abuse, violence and crime prevention and child health. Health Promot J Austr 18: 184-190.
9. Jensen CD, Cushing CC, Aylward BS, Craig JT, Sorell DM, et al. (2011) Effectiveness of motivational interviewing interventions for adolescent substance use behavior change: a meta-analytic review. J Consult Clin Psychol 79: 433-440.
10. Yuma-Guerrero PJ, Lawson KA, Velasquez MM, von Sternberg K, Maxson T, et al. (2012) Screening, brief intervention, and referral for alcohol use in adolescents: a systematic review. Pediatrics 130: 115-122.
11. Sterling S, Valkanoff T, Hinman A, Weisner C (2012) Integrating substance use treatment in adolescent health care. Curr Psychiatry Rep 14: 453-461.
12. Vendetti J, McRee B, Hernandez A, Karuntzos G (2013) Screening, brief intervention, and referral to treatment (SBIRT) implementation models and work flow processes: commonalities and variations. Addict Sci Clin Pract 8: A79.
13. Kelly TM, Donovan JE, Chung T, Cook RL, Delbridge TR (2004) Alcohol use disorders among emergency department - treated older adolescents: A new brief screen (RUFT-Cut) using the AUDIT, CAGE, CRAFFT, and RAPS-QF. Alcohol Clin Exp Res 28: 746-753.
14. Rahdert E (1991) The Problem Oriented Screening Instrument for Teenagers (POSIT). NIDA.
15. Knight JR, Shrier LA, Bravender TD, Farrell M, Vander Bilt J, et al. (1999) A new brief screen for adolescent substance abuse. Arch Pediatr Adolesc Med 153: 591-596.
16. Group WAW (2002) The alcohol, smoking, and substance involvement screening test (ASSIST): development, reliability, and feasibility. Addiction 97: 1183-1194.
17. https://www.who.int/substance_abuse/activities/assist_test/en/
18. Pilowsky DJ, Wu LT (2013) Screening instruments for substance use and brief interventions targeting adolescents in primary care: a literature review. Addict Behav 38: 2146-2153.
19. Harris SK, Louis-Jacques J, Knight JR (2014) Screening and brief intervention for alcohol and other abuse. Adolesc Med State Art Rev 25: 126-156.
20. Dhalla S, Zumbo BD, Poole G (2011) A review of the psychometric properties of the CRAFFT instrument: 1999-2010. Curr Drug Abuse Rev 4: 57-64.
21. Whiteside U, Cronce JM, Pedersen ER, Larimer ME (2010) Brief motivational feedback for college students and adolescents: a harm reduction approach. J Clin Psychol 66: 150-163.
22. https://phprimer.afmc.ca/Part2MethodsStudyingHealth/Chapter6MethodsMeasuringHealth/Interpretingtestsonindividuals.
23. Knight JR, Harris SK, Sherritt L, Van Hook S, Lawrence N, et al. (2007) Prevalence of positive substance abuse screen results among adolescent primary care patients. Arch Pediatr Adolesc Med 161: 1035-1041.
24. Whittle AE, Buckelew SM, Satterfield JM, Lum PJ, O'Sullivan P (2015) Addressing Adolescent Substance Use: Teaching SBIRT and MI to Residents. Subst Abus 36: 325-331.
25. Sterling S, Valkanoff T, Hinman A, Weisner C (2012) Integrating substance use treatment into adolescent health care. Current psychiatry reports 14: 453-461.
26. Wilson CR, Sherritt L, Gates E, Knight JR (2004) Are clinical impressions of adolescent substance use accurate? Pediatrics 114: e536-e540.
27. Ozechowski TJ, Becker SJ, Hogue A (2016) SBIRT-A: Adapting SBIRT to Maximize Developmental Fit for Adolescents in Primary Care. J Subst Abuse Treat 62: 28-37.
28. Van Hook S, Harris SK, Brooks T, Carey P, Kossack R, et al. (2007) The “Six T’s”: barriers to screening teens for substance abuse in primary care. J Adolesc Health 40: 456-461.
29. Carney T, Myers BJ, Louw J, Okwundu CI (2014) Brief school-based interventions and behavioural outcomes for substance-using adolescents. Cochrane Database Syst Rev 2: CD008969.
30. Madras BK, Compton WM, Avula D, Stegbauer T, Stein JB, et al. (2009) Screening, brief interventions, referral to treatment (SBIRT) for illicit drug and alcohol use at multiple healthcare sites: comparison at intake and 6 months later. Drug Alcohol Depend 99: 280-295.
31. Cole B, Clark DC, Seale JP, Shellenberger S, Lyme A, et al. (2012) Reinventing the reel: an innovative approach to resident skill-building in motivational interviewing for brief intervention. Subst Abus 33: 278-281.
32. Miller WR, Rose GS (2009) Toward a theory of motivational interviewing. Am Psychol 64: 527-537.
33. Rubak S, Sandbaek A (2005) Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract 55: 305-312.
34. Hettema J, Steele J, Miller WR (2005) Motivational interviewing. Annu Rev Clin Psychol 1: 91-111.
35. Angres DH1, Bettinardi-Angres K (2008) The disease of addiction: origins, treatment, and recovery. Dis Mon 54: 696-721.
36. Tatarsky A (2003) Harm reduction psychotherapy: extending the reach of traditional substance use treatment. J Subst Abuse Treat 25: 249-256.
37. Copeland J, Swift W, Roffman R, Stephens R (2001) A randomized controlled trial of brief cognitive-behavioral interventions for cannabis use disorder. J Subst Abuse Treat 21: 55-66.
38. Baker A, Lee NK, Claire M, Lewin TJ, Grant T, et al. (2005) Brief cognitive behavioural interventions for regular amphetamine users: a step in the right direction. Addiction 100: 367-378.
39. Kelleher E (2014) Is there more to psychosocial treatments in addiction than brief interventions? Ir J Psychol Med 31: 61-68.
40. Bernstein J, Bernstein E, Tassiopoulos K, Heeren T, Levenson S, et al. (2005) Brief motivational intervention at a clinic visit reduces cocaine and heroin use. Drug Alcohol Depend 77: 49-59.
41. Ondersma SJ, Chase SK, Svikis DS, Schuster CR (2005) Computer-based brief motivational intervention for perinatal drug use. J Subst Abuse Treat 28: 305-312.
42. Ondersma SJ1, Svikis DS, Schuster CR (2007) Computer-based brief intervention a randomized trial with postpartum women. Am J Prev Med 32: 231-238.
43. Collaborative AESR (2010) The impact of screening, brief intervention and referral for treatment in emergency department patients’ alcohol use: a 3-, 6-and 12-month follow-up. Alcohol Alcohol 45: 514-519.
44. Saitz R (2014) Screening and brief intervention for unhealthy drug use: little or no efficacy. Front Psychiatry 5: 121.
45. Puskar K, Gotham HJ, Terhorst L, Hagle H, Mitchell AM, et al. (2013) Effects of Screening, Brief Intervention, and Referral to Treatment (SBIRT) education and training on nursing students' attitudes toward working with patients who use alcohol and drugs. Subst Abus 34: 122-128.
46. Grenard JL, Ames SL, Wiers RW, Thush C, Stacy AW, et al. (2007) Brief intervention for substance use among at-risk adolescents: a pilot study. J Adolesc Health 40: 188-191.
47. McCambridge J, Strang J (2004) The efficacy of single-session motivational interviewing in reducing drug consumption and perceptions of drug-related risk and harm among young people: results from a multi-site cluster randomized trial. Addiction 99: 39-52.
49. McCambridge J, Strang J (2005) Deterioration over time in effect of Motivational Interviewing in reducing drug consumption and related risk among young people. Addiction 100: 470-478.
50. Walker DD, Roffman RA (2006) Motivational enhancement therapy for adolescent marijuana users: A preliminary randomized controlled trial. J Consult Clin Psychol 74: 628-632.
51. Winters KC, Fahnhorst T, Botzet A, Lee S, Lalone B (2007) Brief intervention for drug-abusing adolescents in a school setting. Psychol Addict Behav 21: 249-254.
52. Marsden J, Stillwell G, Barlow H, Boys A, Taylor C, et al. (2006) An evaluation of a brief motivational intervention among young ecstasy and cocaine users: no effect on substance and alcohol use outcomes. Addiction 101: 1014-1026.
53. Peterson PL, Baer JS, Wells EA, Garrett S (2006) Short-term effects of a brief motivational intervention to reduce alcohol and drug risk among homeless adolescents. Psychol Addict Behav 20: 254-264.
54. Beaton A, Shubkin CD, Chapman S (2016) Addressing substance misuse in adolescents: a review of the literature on the screening, brief intervention, and referral to treatment model. Curr Opin Pediatr 28: 258-265.
55. Sowers WE1, Rohland B (2004) American Association of Community Psychiatrists' principles for managing transitions in behavioral health services. Psychiatr Serv 55: 1271-1275.
56. Madson MB1, Mohn RS, Zuckoff A, Schumacher JA, Kogan J, et al. (2013) Measuring client perceptions of motivational interviewing: factor analysis of the Client Evaluation of Motivational Interviewing scale. J Subst Abuse Treat 44: 330-335.
57. Barnett E1, Sussman S, Smith C, Rohrbach LA, Spruijt-Metz D (2012) Motivational Interviewing for adolescent substance use: a review of the literature. Addict Behav 37: 1325-1334.
58. Silverstone PH, Bercov M, Suen VYM, Allen A, Cribben I, et al. (2015) Initial findings from a novel school-based program, EMPATHY, which may help reduce depression and suicidality in youth. PLoS One 10: e125527.
59. Costello EJ1, Egger H, Angold A (2005) 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. J Am Acad Child Adolesc Psychiatry 44: 972-986.
60. Goldman S (2012) Developmental epidemiology of depressive disorders. Child Adolesc Psychiatr Clin N Am 21: 217-235.
61. Carli V, Hoven CW, Wasserman C, Chiesa F, Guffanti G, et al. (2014) A newly identified group of adolescents at "invisible" risk for psychopathology and suicidal behavior: findings from the SEYLE study. World Psychiatry 13: 78-86.
62. Balázs J, Miklósi M, Keresztény A, Hoven CW, Carli V, et al. (2013) Adolescent subthreshold-depression and anxiety: psychopathology, functional impairment and increased suicide risk. J Child Psychol Psychiatry 54: 670-677.
63. Bertha EA, Balazs J (2013) Subthreshold depression in adolescence: a systematic review. Eur Child Adolesc Psychiatry 22: 589-603.
64. Nardi B, Francesconi G, Catena-Dell'osso M, Bellantuono C, et al. (2013) Adolescent depression: clinical features and therapeutic strategies. Eur Rev Med Pharmacol Sci 17: 1546-1551.
65. Calear AL, Christensen H (2010) Review of internet-based prevention and treatment programs for anxiety and depression in children and adolescents. Med J Aust 192: S12-S14.
66. Postel MG, de Haan HA, De Jong CA (2008) E-therapy for mental health problems: a systematic review. Telemed J E Health 14: 707-714.
67. Sawyer MG, Pfeiffer S, Spence SH, Bond L, Graetz B, et al. (2010) School-based prevention of depression: a randomised controlled study of the beyondblue schools research initiative. J Child Psychol Psychiatry 51: 199-209.
68. Tak YR, Van Zundert RM, Kuijpers RC, Van Vlokhoven BS, Rensink HF, et al.(2012) A randomized controlled trial testing the effectiveness of a universal school-based depression prevention program 'Op Volle Kracht' in the Netherlands. BMC Public Health 12: 21.
69. Wijnhoven LA, Creemers DH, Vermulst AA, Scholte RH, Engels RC, et al. (2014) Randomized controlled trial testing the effectiveness of a depression prevention program ('Op Volle Kracht') among adolescent girls with elevated depressive symptoms. J Abnorm Child Psychol 42: 217-228.
70. Kindt KC, van Zundert R, Engels RC (2012) Evaluation of a Dutch school-based depression prevention program for youths in highrisk neighborhoods: study protocol of a two-armed randomized controlled trial. BMC Public Health 12: 212.
71. Kindt KC, Kleinjan M, Janssens JM, Scholte RH (2014) Evaluation of a school-based depression prevention program among adolescents from low-income areas: a randomized controlled effectiveness trial. Int J Environ Res Public Health 11: 5273-5293.
72. https://www.healthknowledge.org.uk/public-health-textbook/research-methods/1b-statistical-methods/parametric-nonparametric-tests.
73. Swinscow TDV, Campbell MJ (1997) Statistics at Square One. Oxford: BMJ Books.
74. Dakof GA, Tejeda M, Liddle HA (2001) Predictors of engagement in adolescent drug abuse treatment. J Am Acad Child Adolesc Psychiatry 40: 274-281.
75. Botvin GJ, Griffin KW (2007) School-based programmes to prevent alcohol, tobacco and other drug use. Int Rev Psychiatry 19: 607-615.
76. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, et al. (2010) Lifetime prevalence of mental disorders in U.S. adolescents: results from the National Comorbidity Survey Replication--Adolescent Supplement (NCS-A). J Am Acad Child Adolesc Psychiatry 49: 980-989.
77. Chambers RA, Taylor JR, and Potenza MN (2014) Developmental neurocircuitry of motivation in adolescence: a critical period of addiction vulnerability. Am J Psychiatry 160: 1041-1052.
78. Swendsen J, Burstein M, Case B, Conway KP, Dierker L, et al. (2012) Use and abuse of alcohol and illicit drugs in US adolescents: Results of the National Comorbidity Survey–Adolescent Supplement. Arch Gen Psychiatry 69: 390-398.
79. Becker JB, Hu M (2008) Sex differences in drug abuse. Front Neuroendocrinol 29: 36-47.
80. Holloway K, Bennett T (2007)  Gender differences in drug misuse and related problem behaviors among arrestees in the UK. Subst Use Misuse 42: 899-921.
81. Vigna-Taglianti F, Vadrucci S, Faggiano F, Burkhart G, Siliquini R, et al. (2009) Is universal prevention against youths' substance misuse really universal? Gender-specific effects in the EU-Dap school-based prevention trial. J Epidemiol Community Health 63: 722-728.
82. Levy S, Sherritt L, Harris SK, Gates EC, Holder DW, et al. (2004) Test-retest reliability of adolescents' self-report of substance use. Alcohol Clin Exp Res 28: 1236-1241.
83. Flisher AJ, Evans J, Muller M, Lombard C (2004) Brief report: Test-retest reliability of self-reported adolescent risk behaviour. J Adolesc 27: 207-212.
84. Winters KC, Stinchfield RD, Henly GA, Schwartz RH (1990) Validity of adolescent self-report of alcohol and other drug involvement. Int J Addict 25: 1379-1395.