Please take our brief survey

Blueprints Programs = Positive Youth Development

Return to Search Results

Promising Program Seal

Communities That Care

Blueprints Program Rating: Promising

A prevention system designed to reduce levels of adolescent delinquency and substance use through the selection and use of effective preventive interventions tailored to a community's specific profile of risk and protection.

  • J. David Hawkins, Ph.D.
  • University of Washington School of Social Work
  • Social Development Research Group
  • 9725 3rd Ave. NE, Suite 401
  • Seattle, WA 98115
  • jdh@u.washington.edu
  • Alcohol
  • Delinquency and Criminal Behavior
  • Tobacco
  • Violence

    Program Type

    • Community, Other Approaches

    Program Setting

    • Community (e.g., religious, recreation)

    Continuum of Intervention

    • Universal Prevention (Entire Population)

    A prevention system designed to reduce levels of adolescent delinquency and substance use through the selection and use of effective preventive interventions tailored to a community's specific profile of risk and protection.

      Population Demographics

      Communities that Care was implemented in racially and ethnically diverse communities. The evaluation focused on grades 5-9.

      Age

      • Infant (0-2)
      • Early Childhood (3-4) - Preschool
      • Late Childhood (5-11) - K/Elementary
      • Early Adolescence (12-14) - Middle School
      • Late Adolescence (15-18) - High School
      • Early Adulthood (19-22)

      Gender

      • Male and Female

      Race/Ethnicity

      • All Race/Ethnicity

      Race/Ethnicity/Gender Details

      Communities That Care was implemented in communities with a diverse population. At nine years post baseline, only males showed significant sustained effects.

      Because assessment of risk and protective factors in a participating community is conducted, and interventions are chosen that focus on the R&P factors that are of highest priority in a community, CTC has a focus on all R&P factors.

      • Individual
      • Peer
      • School
      • Family
      • Neighborhood/Community
      Risk Factors
      • Individual: Antisocial/aggressive behavior, Early initiation of antisocial behavior*, Early initiation of drug use*, Favorable attitudes towards antisocial behavior*, Favorable attitudes towards drug use*, Gang involvement, Physical violence, Rebelliousness, Stress, Substance use*
      • Peer: Interaction with antisocial peers, Peer rewards for antisocial behavior, Peer substance use
      • Family: Family conflict/violence, Family history of problem behavior, Parental attitudes favorable to antisocial behavior, Parental attitudes favorable to drug use, Poor family management*
      • School: Low school commitment and attachment, Poor academic performance
      • Neighborhood/Community: Community disorganization, Laws and norms favorable to drug use/crime, Low neighborhood attachment, Perceived availability of drugs, Perceived availability of handguns, Transitions and mobility
      Protective Factors
      • Individual: Clear standards for behavior, Coping Skills, Perceived risk of drug use, Prosocial involvement, Refusal skills, Religious service attendance, Rewards for prosocial involvement, Skills for social interaction
      • Peer: Interaction with prosocial peers
      • Family: Attachment to parents, Opportunities for prosocial involvement with parents, Rewards for prosocial involvement with parents
      • School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school
      • Neighborhood/Community: Opportunities for prosocial involvement, Rewards for prosocial involvement

      *Risk/Protective Factor was significantly impacted by the program.

      See also: Communities That Care Logic Model (PDF)

      Communities That Care (CTC) is a prevention system, grounded in science that gives communities the tools to address their adolescent health and behavior problems through a focus on empirically identified risk and protective factors. CTC provides a structure for engaging community stakeholders, a process for establishing a shared community vision, tools for assessing levels of risk and protection in communities, and processes for prioritizing risk and protective factors and setting specific, measurable, community goals. CTC guides the coalition to create a strategic community prevention plan designed to address the community's profile of risk and protection with tested, effective programs and to implement the chosen programs with fidelity. CTC instructs the coalition to monitor program implementation and to periodically reevaluate community levels of risk and protection and outcomes, and to make adjustments in prevention programming if indicated by the data. Implementation of CTC is organized into five stages, each with its own series of "benchmarks" and "milestones" to help guide and monitor implementation progress. CTC is installed in communities through a series of six training events delivered over the course of 6 to 12 months by certified CTC trainers.

      Communities That Care (CTC) is a prevention system that gives communities the tools to address adolescent health and behavior problems through a focus on empirically identified risk and protective factors. CTC mobilizes community leaders and a community prevention coalition (called the "community prevention board") to plan and implement a set of tested interventions to reduce elevated risk factors and promote protective factors in the community. According to CTC's theory of change, it should take from 2 to 5 years to observe community-level changes in targeted risk factors in CTC communities, and from 5 to 10 years to observe community-level changes in substance use and delinquency outcomes.

      Implementation of CTC is organized into five stages, each with its own series of "benchmarks" and "milestones" to help guide and monitor implementation progress. Technical assistance is provided to local CTC coordinators and prevention coalition members to help ensure completion of these steps and procedures, identify any barriers to successful implementation, and discuss strategies for overcoming obstacles. Assistance is delivered via weekly phone calls and emails and twice-yearly site visits.

      Phase 1 is a Community Readiness Assessment phase. Here, attitudinal and organizational characteristics of community members, leaders, and organizations thought to influence the mobilization process are assessed. Important individuals and organizations necessary to initiate CTC are identified.

      Phase 2 introduces the community to CTC through a training event that orients key community leaders to prevention science and the community activation processes of CTC. The training defines roles and responsibilities of the key leaders and those of the community prevention board. Key leaders are expected to hold the community prevention board and staff accountable for planning and carrying out CTC and to identify and secure resources necessary to implement preventive interventions planned through the CTC process. Key leaders then identify and invite community members who will make up the community prevention board, or, alternatively, identify an existing coalition in the community to take on the CTC prevention board functions. Prevention board members attend a 2-day orientation training.

      In Phase 3, the CTC board completes assessments of levels of youth problem behaviors and risk and protective factors, as well as assessments of existing community resources. Board members participate in a 2-day training on how to evaluate the collected data in terms of community risk and protective factors, and the CTC board prioritizes two to five risk factors to target for preventive action. The profiles of risk and protection provide baseline data for later assessments of the community's progress in changing levels and trends in the factors targeted by the board's prevention plan. Following the prioritization of risk and protective factors, CTC board members attend a 1-day resource assessment training with the goal of identifying gaps in existing policies, programs, and services that address the community's prioritized factors.

      In Phase 4, the CTC board develops its community action plan. Community board members attend a 2-day Community Plan Training that reviews tested, evidence-based policies, programs, and actions that have demonstrated effectiveness. The board defines measurable objectives with respect to reducing prioritized risk factors, enhancing protective factors, and reducing substance use and delinquency, and develops a plan to fill gaps in existing services through the implementation of tested, effective policies and programs. The CTC typically encompasses preventive actions from the prenatal period through young adulthood. The board selects policies and programs from a menu of tested preventive interventions for elementary and middle school students. Communities' action plans describe the interventions selected and include work plans to implement those new interventions, monitor and provide feedback on implementation quality, and assess progress towards specified process and outcome goals.

      In Phase 5, the chosen preventive interventions are implemented, and implementation quality is monitored by the CTC community prevention board. At the outset of Phase 5, CTC boards receive the Community Plan Implementation Training to develop the skills and plans necessary to implement and monitor their community's action plan and sustain the CTC effort. Beginning in Year 2 and continuing into Year 5, program developers conduct trainings on the selected programs and provide technical assistance to ensure high-quality implementation and monitoring of progress toward implementation and outcome goals. Monitoring of implementation is accomplished through program-specific implementation checklists completed by program providers, checklists completed by community board members and agency supervisors who observe 10% to 15% of program sessions, and participant pre- and post-tests. During Phase 5, the board also engages local media to educate community members about risk and protective factors for adolescent problem behaviors, generates public support for the new preventive interventions indicated and motivates community members to take part in the new preventive interventions.

      Communities That Care (CTC) is guided theoretically by the social development model (SDM), which posits that bonding to prosocial groups and individuals and clear standards for healthy behavior are protective factors that inhibit the development of problem behaviors. The SDM hypothesizes that bonding is created when people are provided opportunities to be involved in a social group like a coalition, family, or classroom, when they have the skills to participate in the social group, and when they are recognized for their contributions to the group.

      This theoretical framework is applied in CTC in two ways. First, CTC encourages community stakeholders to adopt the SDM in their daily interactions with young people as a strategy for promoting healthy development. A goal in CTC communities is to ensure that all young people are provided developmentally appropriate opportunities, skills, and recognition, as well as healthy standards for behavior, by adults and organizations in the community. Second, the social development model guides the community mobilization and training component of CTC itself. CTC seeks to create opportunities for all interested community stakeholders to participate in developing a shared vision for positive youth development based in prevention science. Through CTC trainings, diverse community representatives develop skills to work together effectively, thus increasing the likelihood that opportunities for interaction lead to rewarding experiences. The CTC process also suggests appropriate recognition activities to enhance the reinforcement of community board members for their participation in the process.

      • Attachment - Bonding
      • Skill Oriented

      The Community Youth Development Study (CYDS) is the first community-randomized trial of CTC. The initial 5-year experimental study was conducted in 24 communities across seven states nationally. Sites were matched within state, and then randomly assigned to 12 intervention and 12 control communities in 2003. To test the effects of CTC in achieving observable reductions in targeted risk factors, delinquent behavior, and substance use within the 5 years of the study as hypothesized by CTC's theory of change, the intervention communities were asked to focus their prevention plans on interventions for youths aged 10 to 14 years and their families. Data from a panel of 4,407 fifth grade students was surveyed annually through 10th grade from 2004 to 2009. Levels and trends in adolescent drug use and delinquency were assessed. A quasi-experimental study of CTC was conducted in Pennsylvania sites using a state-funded survey of adolescents in grades 6, 8, 10, and 12 in public and private schools. The full data sets available contained data on 92 school districts and 43,842 students in 2001 and 159 school districts and 101,988 students in 2003.

      A second evaluation (Rhew et al., 2016) also used data from the CYDS but examined a series of cross-sectional surveys between 2000 and 2008. The study pooled the 2000 and 2002 assessments for baseline measures and the 2006 and 2008 assessments for follow-up measures and included 6th, 8th, and 10th graders. Sample sizes over survey years ranged from 4,647 to 5,077 for 6th graders, 4,491 to 4,984 for 8th graders, and 3,854 to 4,726 for 10th graders.

      Randomized CTC Study:

      Hawkins, Brown et al., 2008
      The levels of risk factors targeted by CTC communities were significantly lower among panel students in grade 7 in intervention communities than in control communities after 1.67 years of implementing preventive interventions selected through the CTC process. Students in control communities were significantly more likely to initiate delinquent behavior between fifth and seventh grades than students in CTC communities. No significant condition effects were found on substance use initiation between grades 5 and 7.

      Hawkins et al., 2009
      The incidence of delinquent behavior, alcohol, cigarette, and smokeless tobacco initiation were significantly lower in CTC than in control communities between grades 5 and 8. In grade 8, the prevalence of alcohol and smokeless tobacco use in the last 30 days, binge drinking in the past 2 weeks, and the number of different delinquent behaviors committed in the past year in grade 8 were significantly lower in CTC communities compared to control communities. Significant intervention effects on the onset of marijuana or inhalant use or the prevalence of marijuana, cigarette, inhalant, prescription drug, or other illicit drug use in the past 30 days were not observed by the spring of 8th grade.

      Hawkins et al., 2011 (one year after the end of technical assistance)
      Mean levels of targeted risks increased less rapidly between grades 5 and 10 in CTC than in control communities and were significantly lower in CTC than control communities in grade 10. Students in CTC communities had 38% lower odds of initiating the use of alcohol, 46% lower odds of beginning to smoke by grade 10, and 21% lower odds of initiating delinquent behavior, relative to students in control communities. There were no significant differences by intervention status in the incidence of smokeless tobacco, marijuana, inhalant, or prescription drug use by the spring of 10th grade. Regarding prevalence measures, students in CTC communities had 21% lower odds of smoking cigarettes in the past month, 17% lower odds of reporting any delinquent behavior, and 25% lower odds of reporting any violent behavior in the past year. There were no significant differences between conditions in rates of binge drinking the past 2 weeks, past-month alcohol, smokeless tobacco, marijuana, inhalant, prescription, and other illicit drug use, or the variety of different delinquent or violent acts in which students engaged.

      Hawkins et al., 2014 (eight years post baseline, grade 12)
      Students in CTC communities were more likely than students in control communities to have abstained from any drug use, drinking alcohol, smoking cigarettes, and engaging in delinquency. They were also less likely to ever have commited a violent act. There were no significant differences by intervention group in the prevalence of past-month or past-year substance use, or past-year delinquency or violence.

      Oesterle et al., 2015 (nine years post baseline, age 19)
      At nine years post baseline, at age 19, the only significant intervention effects were among males, who were more likely to abstain from cigarette use (ARR=1.22, p<.05) and delinquency (ARR=1.33, p<.05) than the control group. There were no other significant effects among the sample.

      Prevention System Outcomes
      CTC communities exhibited significantly greater increases in adopting a science-based approach to prevention, collaboration across community sectors, and collaboration regarding specific prevention activities between 2001 and 2004 relative to control communities (Brown et al., 2007). Furthermore, Rhew et al. (2011) found that many prevention system characteristics, such as higher levels of adoption of a science-based approach to prevention, were sustained through 2009, 1.5 years after study-funded resources for CTC ended. Also, only one of the 12 coalitions had disbanded by 2009. The 11 remaining coalitions continued to meet benchmarks of phases 2 through 5 of the CTC system, and two-thirds reported having a paid staff person (Gloppen et al., 2011).

      Implementation Analyses
      Research on implementation fidelity throughout the randomized study show that CTC was well implemented with fidelity ratings averaged across three groups of raters showing that between 89% and 100% of the CTC milestones in the first four phases of CTC implementation were "completely met" or "majority met" in the 12 intervention communities (Quinby et al., 2008). Throughout the study, there were high rates of adherence to the programs' core components and in accordance with dosage requirements regarding the number, length, and frequency of sessions (Fagan et al., 2008a,b).

      Pennsylvania Quasi-experimental Study:

      Feinberg et al., 2007
      In general, the pattern of findings found in the analysis of the Pennsylvania Youth Survey data represents evidence that communities employing the CTC model had lower levels of risk factors and problem behaviors than communities not employing CTC. Outcomes were best at the 6th grade in 2003 favoring CTC sites on 30-day alcohol and cigarette use and delinquent behavior. Delinquency was also lower in the intervention group at grade 10, and drug involvement and 2-week prevalence of binge drinking in grade 12. Of 82 comparisons in 2001 and 82 in 2003, 4 significantly favored CTC sites in 2001 and 16 in 2003.

      Feinberg et al., 2010
      When contrasting grade cohorts that were actually exposed to evidence-based programs ("expected impact" cohorts), compared to grade cohorts in the same schools that were not exposed to EBPs combined with students from non-CTC schools, youth in "expected impact" CTC grade cohorts demonstrated significant and beneficial effects for risk/protective factors, academic grades, and delinquency. There was also a population-level impact, with decreased growth in delinquency across adolescence.

      Additional Process Studies:

      Arthur et al., 2003
      Two community mobilization interventions were successful at mobilizing community boards to plan and implement prevention activities, and both approaches were able to recruit and involve the types of community members they targeted on their planning boards. The Washington State CYAP was successful at involving youth in planning youth-oriented activities. The Communities That Care process used in the Oregon TOGETHER! project was effective at involving key community leaders in organizing prevention boards in their communities. However, the Oregon TOGETHER! project was more successful than the Washington State CYAP project at promoting planning and program activities aimed at specific, empirically based risk factors identified through a community risk assessment process. Even without funding, Oregon TOGETHER! prevention boards were more likely than the funded Washington CYAP community teams to collect empirical indicators of community risk and protective factors, develop action plans describing strategies to reduce prioritized risk factors, and implement programs aimed at reducing these risk factors.

      Harachi et al, 1996
      At the end of the four-year demonstration of Oregon TOGETHER!, using the CTC model, 31 communities remained active in the project, which Oregon has institutionalized, and 28 of them were involved in the implementation of risk reduction programs. Within a year after training, 28 boards had completed comprehensive risk-focused prevention plans. Less than a year into the planning and implementation phase, 27 had begun implementing risk reduction strategies. These findings appear to indicate that once put in place in communities, the CTC system can be maintained for several years even without significant dedicated funding.

      Repeated Cross-sectional Study (Rhew et al., 2016):

      The repeated cross-sectional study using CYDS data found a possible iatrogenic effect of the program on antisocial behavior among 6th graders. None of the 12 outcomes were significant among 8th or 10th graders. In separate analyses comparing 6th graders in 2004 to 10th graders in 2008, the study found a significantly greater reduction on smokeless tobacco use among treatment participants.

      Randomized Control Study in 24 communities:

      Through Grade 7 (Hawkins, Brown et al., 2008):

      • Students in control communities were significantly more likely to initiate delinquent behavior between fifth and seventh grades than were students in CTC communities.
      • No significant intervention condition effects were found on substance use initiation between grades 5 and 7.

      Through Grade 8 (Hawkins et al., 2009):

      • The incidence of delinquent behavior, alcohol, cigarette, and smokeless tobacco initiation were significantly lower in CTC than in control communities between grades 5 and 8.
      • In grade 8, the prevalence of alcohol and smokeless tobacco use in the last 30 days, binge drinking in the past 2 weeks, and the number of different delinquent behaviors committed in the past year in grade 8 were significantly lower in CTC communities compared to control communities.

      Through Grade 10, one year after the end of technical assistance (Hawkins et al., 2011):

      • The incidence of alcohol use, cigarette use, and delinquency was lower by grade 10 among students in CTC communities than in control communities.
      • The prevalence of current cigarette use and past-year delinquent and violent behavior were significantly lower in CTC than in control communities in grade 10.

      Through Grade 12, eight years post baseline (Hawkins et al., 2014)

      • Abstinence from drug use, drinking alcohol, smoking cigarettes, and engaging in delinquency lower in the CTC communities than control communities.
      • Less likely to ever have committed a violent act in CTC communities, relative to control communities.

      Through age 19, nine years after baseline (Oesterle et al., 2015):

      • The incidence of cigarette use and delinquency was lower for males in CTC communities than in control communities.

      Community-Level Prevention Service System Outcomes (Brown et al., 2007; Rhew et al., 2011 draft):

      • CTC communities exhibited significantly greater increases in adopting a science-based approach to prevention, collaboration across community sectors, and collaboration regarding specific prevention activities between 2001 and 2004, relative to control communities.
      • CTC communities reported higher levels of adoption of a science-based approach to prevention in 2009, 1.5 years after study-funded resources for CTC ended.
      • All but one of the 12 CTC coalitions was still in existence in 2009 and sustaining the coalition's structure and prevention activities.

      Pennsylvania Quasi-experimental Study (Feinberg et al., 2007)

      • In general, the pattern of findings shows that communities employing the CTC model had lower levels of risk factors and problem behaviors (delinquency and alcohol/drug use) than communities not employing CTC.
      • When contrasting grade cohorts that were actually exposed to evidence-based programs ("expected impact" cohorts), compared to grade cohorts in the same schools that were not exposed to EBPs combined with students from non-CTC schools, youth in "expected impact" CTC grade cohorts demonstrated significant and beneficial effects for risk/protective factors, academic grades, and delinquency.

      Significant Program Effects on Risk and Protective Factors:

      • The levels of risk factors targeted by CTC communities were significantly lower among panel students in grade 7 in intervention communities than in control communities after 1.67 years of implementing preventive interventions selected through the CTC process (Hawkins, Brown, et al., 2008).
      • Mean levels of targeted risks increased less rapidly between grades 5 and 10, and were significantly lower in grade 10, in CTC than in control communities (Hawkins et al., 2011).

      Repeated cross-sectional study using CYDS data:

      Among 6th graders, there was a possible harmful effect as control participants had significantly better outcomes than treatment participants in:

      • Antisocial behavior

      In comparing change between 6th and 10th grade, the study found significant improvement in treatment communities than control communities for:

      • Lifetime use of smokeless tobacco

      Higher levels of community adoption of a science-based approach to prevention in 2004 predicted significantly lower levels of youth problem behaviors in 2007. Effects of the CTC intervention on youth problem behaviors were mediated fully by community adoption of a science-based approach to prevention, as reported by key community leaders (Brown et al., 2013).

      Communities That Care has been implemented among large distributions of both rural and urban populations, and should be generalizable to similar populations.

      The randomized trial of CTC was an efficacy trial in which training, technical assistance, and resources for a community coordinator as well as resources to implement tested, effective interventions were provided to intervention communities. These resources may not be available in all communities. In addition, the major RCT study does not include urban or suburban populations, and as such the findings may not be generalizable to larger communities. We can only assess that CTC works with the specific menu of programs used in these communities.

      Feinberg et al., 2007: The removal of 11 high-poverty CTC school districts from the analyses limited the ability to understand the effects of CTC among lower poverty communities. The possibility of a selection bias exists due to the quasi-experimental study design.

      Rhew et al., 2016:

      • Treatment and control groups matched on demographic factors, but baseline equivalence not reported
      • Design unable to test for differential attrition among students
      • Few if any positive effects of program
      • Possible iatrogenic effect on antisocial behaviors among 6th graders

      It should be noted that Communities That Care is a prevention system designed to reduce levels of adolescent delinquency and substance use through the selection and use of effective preventive interventions tailored to a community's specific profile of risk and protection. It is not a program or intervention in the traditional sense, but rather a delivery system for evidence-based programs.

      • Blueprints: Promising
      • SAMHSA: 3.2 - 3.6

      Heidi Peterson
      Certified Prevention Specialist
      Director, Communities That Care
      Tooele City, Utah
      Phone: (435) 843-2188
      Fax: (435) 843-2189
      E-Mail: heidip@tooelecity.org
      Web-Address: www.tooelecity.org

      Gery Shelafoe, CPC-R
      NorthCare Network
      200 West Spring Street
      Marquette, MI 59855
      906.225.7323
      gshelafoe@northcarenetwork.org

      Vaughnetta J. Barton, MSW
      Communities in Action
      Communities That Care
      School of Social Work
      University of Washington
      Mail: Box 354900, Seattle, WA 98195-4900
      4101 15th Avenue NE, Seattle, WA
      206.221.8641
      vjbarton@uw.edu

      Arthur, M. W., Ayers, C. D., Graham, K. A., & Hawkins, J. D. (2003). Mobilizing communities to reduce risk for drug abuse: A comparison of two strategies. Pp. 129-144 in The Handbook of Drug Abuse Prevention: Theory, Science, and Practice, edited by Z. Sloboda and W. J. Bukoski. Kluwer Academic/Plenum Publishers: New York, NY.

      Brown, E. C., Hawkins, J. D., Arthur, M. W., Briney, J. S., & Abbot, R. D. (2007). Effects of Communities That Care on prevention services systems: Findings from the community youth development study at 1.5 years. Prevention Science, 8, 180-191.

      Brown, E. C., Hawkins, J. D., Rhew, I. C., Shapiro, V. B., Abbott, R. D., Oesterle, S., ... Catalano, R. F. (2013). Prevention system mediation of Communities That Care effects on youth outcomes. Prevention Science, DOI 10.1007/s11121-013-0413-7.

      Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008a). Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development Study. American Journal of Community Psychology, 41, 235-249.

      Fagan, A. A., Hanson, K., Hawkins, J. D., & Arthur, M. W. (2008b). Implementing effective community-based prevention programs in the Community Youth Development Study. Youth Violence and Juvenile Justice, 6, 256-278.

      Fagan, A. A., Hanson, K., Briney, J. S., & Hawkins, J. D. (2012). Sustaining the utilization and high quality implementation of tested and effective prevention programs using the Communities That Care prevention system. American Journal of Community Psychology, 49,365-377.

      Feinberg, M. W., Greenberg, M. T., Osgood, D. W., Sartorius, J., & Bontempo, D. (2007). Effects of the Communities That Care model in Pennsylvania on youth risk and problem behaviors. Prevention Science, 8, 261-270.

      Feinberg, M. W., Jones, D., Greenberg, M. T., Osgood, D. W., & Bontempo, D. (2010). Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science, 11, 163-171.

      Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health, 51, 259-264.

      Harachi, T. W., Ayers, C. D., Hawkins, J. D., Catalano, R. F., & Cushing, J. (1996). Empowering communities to prevent adolescent substance abuse: Process evaluation results from a risk- and protection-focused community mobilization effort. The Journal of Primary Prevention, 16, 233-254.

      Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbot, R. D., & Catalano, R. F. (2008). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15-22.

      Hawkins, J. D., Catalano, R. F., Arthur, M. W., Egan, E., Brown, E. C., Abbot, R. D., & Murray, D. M. (2008). Testing Communities That Care: The rationale, design and behavioral baseline equivalence of the Community Youth Development Study. Prevention Science, 9, 178-190.

      Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbot, R. D., Fagan, A. A., & Catalano, R. F. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Archives of Pediatric Adolescent Medicine, 163(9), 789-798.

      Hawkins, J. D., Oesterle, S., Brown, E. C., Monahan, K. C., Abbott, R. D., Arthur, M. W., & Catalano, R. F. (2011). Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial.Archives of Pediatric Adolescent Medicine, published online October 3, 2011, doi: 10.1001/archpediatrics.2011.183.

      Kuklinski, M. R., Hawkins, J. D., Plotnick, R. D., Abbott, R. D., & Reid, C. K. (2013). How has the econonmic downturn affected communities and implementation of science-based prevention in the randomized trial of Communities That Care? American Journal of Community Psychology, 51, 370-384.

      Oesterle, S., Hawkins, J. D., Kuklinski, M. R., Fagan, A. A., Fleming, C., Rhew, I. C., ... Catalano, R. F. (2015) Effects of Communities that Care on males’ and females’ drug use and delinquency 9 years after baseline in a community-randomized trial. American Journal of Community Psychology, 56, 217-228.

      Quinby, R. K., Hanson, K., Brooke-Weiss, B., Arthur, M. W., Hawkins, J. D., & Fagan, A. A. (2008). Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology, 36, 313-332.

      Rhew, I. C., Brown, E. C., Hawkins, J. D., & Briney, J. S. (2011 Draft). Sustained effects of Communities That Care on prevention service system transformation. Seattle, WA: Social Development Research Group, University of Washington School of Social Work.

      Rhew, I. C., Hawkins, J. D., Murray, D. M., Fagan, A. A., Oesterle, S., Abbott, R. D., & Catalano, R. F. (2016). Evaluation of community-level effects of Communities That Care on adolescent drug use and delinquency using a repeated cross-sectional design. Prevention Science, 17, 177-187.

      Blair Brooke-Weiss
      Social Development Research Group
      University of Washington School of Social Work
      9725 3rd Ave. NE, Suite 401
      Seattle, WA 98115-2024
      (206) 543-5709
      email: bbrooke@myuw.net
      www.communitiesthatcare.net

      Study 1

      Brown, E. C., Hawkins, J. D., Arthur, M. W., Briney, J. S., & Abbot, R. D. (2007). Effects of Communities That Care on prevention services systems: Findings from the community youth development study at 1.5 years. Prevention Science, 8, 180-191.

      Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health, 51, 259-264.

      Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbot, R. D., & Catalano, R. F. (2008). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15-22.

      Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbot, R. D., Fagan, A. A., & Catalano, R. F. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Archives of Pediatric Adolescent Medicine, 163(9), 789-798.

      Hawkins, J. D., Oesterle, S., Brown, E. C., Monahan, K. C., Abbott, R. D., Arthur, M. W., & Catalano, R. F. (2011). Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial.Archives of Pediatric Adolescent Medicine, published online October 3, 2011, doi: 10.1001/archpediatrics.2011.183.

      Kuklinski, M. R., Hawkins, J. D., Plotnick, R. D., Abbott, R. D., & Reid, C. K. (2013). How has the econonmic downturn affected communities and implementation of science-based prevention in the randomized trial of Communities That Care? American Journal of Community Psychology, 51, 370-384.

      Oesterle, S., Hawkins, J. D., Kuklinski, M. R., Fagan, A. A., Fleming, C., Rhew, I. C., ... Catalano, R. F. (2015) Effects of Communities that Care on males’ and females’ drug use and delinquency 9 years after baseline in a community-randomized trial. American Journal of Community Psychology, 56, 217-228.

      Rhew, I. C., Brown, E. C., Hawkins, J. D., & Briney, J. S. (2011 Draft). Sustained effects of Communities That Care on prevention service system transformation. Seattle, WA: Social Development Research Group, University of Washington School of Social Work.

      Brown, E. C., Hawkins, J. D., Arthur, M. W., Briney, J. S., & Abbot, R. D. (2007). Effects of Communities That Care on prevention services systems: Findings from the community youth development study at 1.5 years. Prevention Science, 8, 180-191.

      Gloppen, K. M., Arthur, M. W., Hawkins, J. D., & Shapiro, V. B. (2012). Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health, 51, 259-264.

      Hawkins, J. D., Brown, E. C., Oesterle, S., Arthur, M. W., Abbot, R. D., & Catalano, R. F. (2008). Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health, 43, 15-22.

      Hawkins, J. D., Oesterle, S., Brown, E. C., Arthur, M. W., Abbot, R. D., Fagan, A. A., & Catalano, R. F. (2009). Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Archives of Pediatric Adolescent Medicine, 163 (9), 789-798.

      Hawkins, J. D., Oesterle, S., Brown, E. C., Monahan, K. C., Abbott, R. D., Arthur, M. W., & Catalano, R. F. (2011). Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial.Archives of Pediatric Adolescent Medicine, published online October 3, 2011, doi: 10.1001/archpediatrics.2011.183.

      Kuklinski, M. R., Hawkins, J. D., Plotnick, R. D., Abbott, R. D., & Reid, C. K. (2013). How has the econonmic downturn affected communities and implementation of science-based prevention in the randomized trial of Communities That Care? American Journal of Community Psychology, 51, 370-384.

      Oesterle, S., Hawkins, J. D., Kuklinski, M. R., Fagan, A. A., Fleming, C., Rhew, I. C., ... Catalano, R. F. (2015) Effects of Communities that Care on males’ and females’ drug use and delinquency 9 years after baseline in a community-randomized trial. American Journal of Community Psychology, 56, 217-228.

      Rhew, I. C., Brown, E. C., Hawkins, J. D., & Briney, J. S. (2011 Draft). Sustained effects of Communities That Care on prevention service system transformation. Seattle, WA: Social Development Research Group, University of Washington School of Social Work.

      Evaluation Methodology

      Design: The Community Youth Development Study (CYDS) is the first community-randomized trial of CTC. The CYDS is built upon the Diffusion Project, a prior 5-year descriptive study of prevention activities and youth outcomes in 41 communities in the 7 collaborating states. At the outset of the Diffusion Project, the drug abuse prevention agencies of the 7 states identified 20 communities they thought were trying to implement risk- and protection-focused prevention services. These 20 communities were then matched, within each state, on population size, racial and ethnic diversity, economic indicators, and crime rates to comparison communities that were not thought to be using a risk- and protection-focused approach, and the 20 community pairs were recruited to participate in the study. In one instance, 2 comparison communities were identified, resulting in a total of 41 communities. During the 5 years of the Diffusion Project, neither community in 13 of the 20 pairs advanced in the use of science-based prevention to the point of selecting and using tested, effective preventive interventions to address prioritized community risks. These 13 pairs of communities were deemed eligible for inclusion in the CYDS study. A total of 12 of these 13 pairs of matched communities were successfully recruited to participate in CYDS and comprise the study sample.

      Recruitment required securing letters from the superintendent of schools, the mayor or city manager, and the lead law enforcement officer, agreeing to all data collection activities required of the project. One member in each pair of eligible communities was randomized to the intervention or control condition by a coin toss prior to recruitment into CYDS. The earlier matching of the intervention and control communities with regard to size, poverty, diversity, and crime indices increased the likelihood of baseline comparability between intervention and control communities in the CYDS. In the spring of every other year since 1998 and continuing through 2008, all assenting 6th, 8th, 10th, and 12th-grade students in participating communities completed the CTC Youth Survey (CTCYS). The response rates for the CTCYS samples were 79% in 1998, 72% in 2000, and 82% in 2002. A total of 2.3% of surveys were eliminated due to dishonest or invalid responses. The number of students providing data was 14,293 in 1998, 12,992 in 2000, and 14,910 in 2002.

      Although CTC typically encompasses preventive actions from the prenatal period to young adulthood, the intervention communities in the Community Youth Development Study (CYDS) were asked to focus their plans on youths in grades 5-9 (10 to 14 years of age) and their families. The enrolled student population in a single grade in these communities ranged from 40 to 485, with only 2 communities exceeding 400 students per grade. Including communities already studied for 5 years in the Diffusion Project provided multiple baseline data points for this study. CTC implementation began in the summer of 2003. Intervention communities were trained to use the baseline data collected by the Diffusion Project in 1998, 2000, and 2002 to prioritize specific risk and protective factors for attention. Each CTC community prevention board prioritized between two and seven risk factors to target with preventive interventions. Some risk factors were targeted across multiple communities. Other risk factors were targeted less frequently. By April 2004, intervention communities had selected preventive interventions to address their prioritized needs and had created strategic community plans to implement these interventions.

      The 12 intervention communities chose 13 evidence-based programs to implement during the 2004-2005 school year, 16 programs during the 2005-2006 school year, and 14 programs during the 2006-2007 school year. These included All-Stars, Life Skills Training, Lion's Quest Skills for Adolescence, Project Alert, Olweus Bullying Prevention Program, Program Development Evaluation Training, Participate and Learn Skills, Big Brothers Big Sisters, Stay Smart, Academic Tutoring, Strengthening Families 10-14, Guiding Good Choices, Parents Who Care, Family Matters, and Parenting Wisely.

      Cross sectional surveys were conducted in the spring of every other year, from 1998 through 2008, with all assenting 6th, 8th, 10th, and 12th grade public school students, using the CTC Youth Survey. The data from these surveys was used to prioritize risks to target with preventive interventions.

      A longitudinal panel of students in each community enrolled in the 5th grade in the first year of CYDS was also obtained. The initial sample for the longitudinal panel was the population of 5th grade public school students in the 24 participating communities in the spring of 2004. Students in the panel who remained in the study communities through the 2004-2005 school year were re-surveyed in the spring of 2005, as were students whose parents consented to the survey for the first time in grade 6. The third annual wave of data was collected in the spring of 2006 when students in the panel who were progressing normally were in grade 7, about 1.67 years after the prevention programs chosen by CTC communities were first implemented. The fourth annual wave of data was collected in the spring of 2007 when panel students progressing normally were in grade 8, about 2.67 years after the prevention programs chosen by CTC communities were first implemented. The sixth annual wave of data was collected in the spring of 2009 when students were in grade 10, approximately 4.67 years after the prevention programs chosen by CTC communities were first implemented. At this point, CTC communities had not received any technical assistance for one year. Ninety-four percent of students in the longitudinal panel completed the survey in Wave 6. Wave 8 in 2012, when subjects were about age 19 and had finished high school, included 89% of the randomized sample (N = 3986).

      During grades 5 and 6, parents of 4,420 students (76.4% of the overall eligible population) consented to participate in the study. Thirteen of these students were absent during scheduled dates of data collection and were not available for surveying. Three additional students who reported being honest only "some of the time" or having used a fictitious drug during the survey as a validity screen were excluded from the analysis. The resulting sample of 4,404 students comprised the analysis sample (the sample reported in Hawkins et al., 2011, at 10th grade was 4,407); 26.5% did not have Wave 1 (grade 5) data because they were recruited in Wave 2 (grade 6), 3.9% and 3.8% of students missed Wave 2 and Wave 3 data collection, respectively, because they were not available for a follow-up interview. A small proportion of students were not available for follow-up interviews. Overall, 93.5% of panel students participated in at least 5 of 6 waves of data collection. There was no systematic bias from differential accretion or differential attrition in control and intervention conditions.

      Beginning in grade 7, a planned missing-data three-form design was initiated to accommodate the growing number of items in the survey. A subset of forms was distributed evenly across two of the three versions of the survey, with each form administered randomly to one-third of the active panel sample. All but two of the targeted risk factor items and all delinquent behaviors, substance use measures, and demographic characteristics were asked of the entire sample.

      Missing data were dealt with via multiple imputation. Using NORM version 2.03, 10 separate data sets, including data from the first 3 waves, were imputed separately by intervention condition. Imputation models included student and community characteristics, risk and protective factors, substance use and delinquent behavior outcomes, and dummy-coded indicators of community membership. Imputed data sets were combined to include intervention and control groups for analysis. Hawkins et al. (2009) used a similar method of dealing with missing data in which 40 separate data sets including data from all 4 waves were imputed separately by intervention condition. Imputation models included student and community characteristics, drug use and delinquent behavior outcomes, and community membership. Imputed data sets were subsequently combined to include both intervention and control groups for analysis.

      Baseline Differences Reported from Diffusion Project (1998-2002) (Hawkins, Catalano et al., 2008).
      There were no significant differences between conditions in 2002 in any of the demographic variables examined at grades 6, 8, and 10. The intervention and control students did not differ significantly with regard to average age; gender; proportions of whites, African Americans, Native Americans, Latinos, or those of other ethnic or racial groups; proportion eligible for free lunch; reported religious attendance; or proportion reporting a family history of substance abuse problems. Of the trends examined from 1998 to 2002 for the 10 demographic variables, only one, the slope for 8th grade family history of substance use, differed between conditions, with control and intervention communities showing similar prevalence levels in 1998 and control communities increasing significantly more than intervention communities over time. There were no significant differences at any grade level between the intervention and control conditions in 2002 in the prevalence of use of any of the 7 substances examined. There was, however, a significant between-conditions difference for 10th grade binge drinking (in 1998), with the control communities demonstrating a higher prevalence than the intervention communities. These differences diminished by 2002. In addition, there was a significantly higher prevalence for 10th grade past 30-day marijuana use (in 1998) with the control communities demonstrating a higher prevalence than the intervention communities. Again, these differences diminished to similar prevalence levels in 2002. No other significant differences in slopes were found across the three grades and seven drugs examined. In 2002, only one of the four antisocial behaviors compared across communities for grades 6, 8, and 10 in 2002 revealed a significant difference. Eighth grade students in the control communities reported a significantly higher prevalence of having attacked someone with the intention of hurting them compared to those in the treatment communities. From 1998 to 2002, only one behavior, 10th-grade students' reports of being drunk or high at school, showed a significantly different slope across conditions. The prevalence of this behavior among 10th grade students was higher in control than intervention communities in 1998, but by 2002 these differences had again diminished to similar levels. To summarize, of 66 tests for differences in baseline rates, only four were significant.

      Community-level prevention services system analysis (Brown et al., 2007):
      In an analysis examining the changes in three community-level outcomes 1.5 years after implementing CTC, community leaders (referred to as positional leaders) in each of the CTC communities were identified and interviewed by research staff. Each of these positional leaders was asked to identify two individuals in the community they thought were the most knowledgeable about current prevention efforts in the community. From this list, research staff interviewed the five additional people in each community recommended the most frequently by the positional leaders (referred to as referred leaders). This procedure resulted in samples of 352 community leaders for the 2001 survey and 335 community leaders for the 2004 survey. Forty-three percent of the 2001 sample (n = 153) were reinterviewed in 2004, resulting in a cumulative sample size of 534 community leaders. On average, the 2001 sample was significantly older than the 2004 sample; respondents did not differ significantly on any other respondent characteristics between 2001 and 2004 samples or between CTC and control groups within each survey year.

      Sample: The sample of students who were in the 5th grade during the 2003-2004 school year was split evenly between male and female students, was 70% Caucasian, 9% Native American, 4% African American, and 20% Hispanic. At grade 5, students were an average of 11.1 years of age, and 55% of the analysis sample was in CTC communities. On average, 89% of the students completing the CTC Youth Survey in participating communities in 2002 were European American (range = 64% to 98%), 3% were African American (range = 0% to 21%), 10% were of Hispanic origin (range = 1% to 65%) and 37% were eligible for free or reduced-price lunch (range = 21% to 66%).

      The sample of positional and referred leaders in the CTC group in 2001 was 63.5% male, an average of 47.6 years of age with an average of 16.5 years spent living in the community, and 45.4% held a master's degree or had a higher level of education. In 2004, the sample was 69.4% male, an average of 48.4 years of age with an average of 17.4 years spent living in the community, and 44.7% held a master's degree or had a higher level of education.

      Measures: Effects of the CTC intervention on risk factors, protective factors, substance use, and other problem behaviors were assessed using repeated cross-sectional and longitudinal student surveys in both intervention and control communities.

      Cross-Sectional and Longitudinal Student Surveys: The CTC Youth Survey (CTCYS) provided reliable and valid measures of an empirically derived set of community, family, school, peer, and individual risk and protective factors and drug use and antisocial behavior outcomes. The 45-minute survey was group administered during one classroom period. The design allowed for comparisons across six cohorts for each grade over three baseline waves collected in the Diffusion Project and three waves of CTCYS data collected in CYDS. The effects of the elementary-grade interventions implemented in CTC communities on risk and protective factors, drug use, and delinquent behavior were assessed by comparing cross-sectional 6th grade samples over time in CTC and control communities. The cumulative effects of the elementary and middle school interventions implemented in CTC communities are assessed by comparing 8th grade surveys across CTC and control communities over time. The repeated cross-sectional 10th grade surveys were designed to detect changes in risk, protection and drug use and delinquency that resulted from the interventions provided during the middle-school grades. A panel sample was also resurveyed annually for the study.

      Targeted risk factors: Risk factor scales consisted of composites of multiple items. Scoring of risk factors entailed standardizing scale items across all three waves of data and taking the mean value of standardized items within a scale for each separate wave of data. Scales missing one or more items were coded as missing data with scale score imputed using multiple imputation analyses. Each CTC community's specific set of targeted risk factors was compared to the same set of risk factors in its matched control community. Risk factors were averaged within each wave of data collection.

      Onset of delinquent behavior: Onset of delinquent behavior was defined as the first occurrence of four delinquent acts (stealing, property damage, shoplifting, attacking someone) between grades 5 and 8. More serious delinquent behaviors (carrying a gun to school, beating up someone, stealing a vehicle, selling drugs, and being arrested) were added to the eighth grade survey as developmentally appropriate. A measure of the variety of delinquent acts committed in the past year ranging from 0 to 9 was constructed from the eighth-grade data.

      Onset of substance use: Items measuring onset of substance use consisted of the first student-reported lifetime use of any of four types of drugs: alcohol, marijuana, cigarettes, or other illicit drugs, between grades 5 and 7. Another study (Hawkins et al., 2009) expanded the drug use measures to include alcohol, cigarettes, smokeless tobacco, marijuana, and inhalants between grades 5 and 8. The prevalence of binge drinking during the past 2 weeks and use of alcohol, cigarettes, smokeless tobacco, marijuana, and inhalants in the past month were measured in grades 5 and 8. Grade 8 measures also included the prevalence of use of prescription drugs and other illicit drugs (psychedelics, ecstasy, stimulants, and cocaine) in the past month.

      Student and community characteristics: Variables measuring student characteristics included age at time of the grade 6 survey; gender; race/ethnicity; whether the student was Hispanic; parental education level; attendance at religious services during grade 5; and rebelliousness, which consisted of the mean of three items: "I like to see how much I can get away with" "I ignore the rules that get in my way" and "I do the opposite of what people tell me, just to get them mad." Variables measuring community demographic characteristics included the total population of students in the community, percentage increase in the student population of the community between 2001 and 2004, and the percentage of students who were eligible for free or reduced-price school lunch.

      Community key informant interviews: Telephone interviews were conducted with key community leaders in all communities twice during the Diffusion Project in 1998 and 2001 (prior to the initiation of the CTC intervention), and again in the CYDS in 2004 and 2007 to assess the degree to which all study communities had adopted a science-based approach to the prevention of adolescent substance abuse and delinquency. The Community Key Informant Interview (CKI) assessed the degree to which 10 positional community leaders and the 5 prevention leaders most frequently nominated by these positional leaders as knowledgeable about prevention reported the use of epidemiological data to guide prevention planning and resource allocation decisions; the selection and use of tested, effective programs; and ongoing monitoring of implementation and changes in risk, protection, and youth outcomes. Evaluation of CTC's impact on community prevention systems included comparisons between the intervention and control communities on measures of adoption of a science-based prevention approach and prevention collaboration from the CKI interview.

      Community resource documentation: The community resource documentation process (CRD) was used to measure the type, number, and scope of prevention activities consistent with tested and effective interventions for youths in grades 5 through 9 and their families in the 24 study communities. The CRD data collection process was a combination of telephone interviews and mail surveys for prevention service providers and program directors, school principals, and civic leaders conducted in 2001, 2004, and 2007. The CRD process provided data on the numbers of programs and policies of each of seven types that were consistent with tested and effective preventive interventions, as well as measures of the scope of each type of program in the community. The seven types of programs and policies included were school prevention curricula, school-wide organizational changes, parenting programs, tutoring programs, mentoring programs, school policies, and community policies.

      Community board interviews: Ten CTC board members in each community were surveyed annually by phone to monitor the CTC board functioning in the intervention communities. The CTC board survey assessed the dimensions associated with developing and maintaining an effective community coalition. This instrument incorporated constructs of community board functioning shown to predict higher implementation in previous studies evaluating the effectiveness of CTC boards and other community prevention coalitions. In addition, monthly reports from each community coordinator documented community board, task force, and staff activities, and included meeting minutes and attendance records. Annual community action plans developed by the CTC board in each intervention community documented assessment results, plans for implementing tested, effective interventions during the coming year, and evaluation plans.

      Communities That Care milestones and benchmarks: The CTC milestones and benchmarks rating tool (Quinby et al., 2008): After each CTC training workshop, community coordinators rated whether or not they had achieved each milestone and benchmark during that phase and in the previous phase. In December 2004, after 18 months of CTC implementation, coordinators also evaluated their progress on all the milestones and benchmarks from all five phases. Each benchmark was rated by the coordinator using a dichotomous measure ("not achieved" or "achieved"), and each milestone was rated on a 4-point scale (from "none of the milestones met" to "milestone completely met"). In addition, each coordinator rated the extent to which all work during each CTC phase was completed using a 4-point scale (from "board will not meet goal" to "board accomplished the phase completely"). Intervention staff and certified CTC trainers also rated each community's implementation progress in December, 2004. The essential benchmarks were rated independently by intervention staff and certified CTC trainers using a 4-point scale. The overall level of CTC system implementation in the 12 intervention communities was assessed by calculating the average of the milestone ratings across the three raters for each milestone. Staff also rated the extent to which each milestone was challenging to achieve on a 4-point scale. Challenge ratings were compared across the 12 communities to ascertain those milestones which were rated as "very challenging" or "mostly challenging" for at least 6 of the 12 intervention communities. Each benchmark rating within these identified milestones was then assessed in the same manner.

      Intervention implementation measures: In the intervention communities, monitoring of preventive intervention implementation is an ongoing process that seeks to ensure that the interventions chosen by the community prevention boards achieve the fidelity, scope, and intensity necessary to achieve the community's risk reduction and protective factor enhancement objectives. Implementation measures assisted the intervention communities to monitor their progress in implementing the tested, effective interventions in their community prevention plans. The feedback provided by these measures was used to support continuous quality improvement of prevention programs during phase 5. Program sessions were directly observed and individually rated for program adherence by independent observers or participants were surveyed to measure the degree to which their knowledge, attitudes, and behaviors changed in the desired direction during implementation (Fagan et al., 2008a). In addition, measures of quality of delivery, participant responsiveness, program participation, and implementation challenges were used (Fagan et al., 2008b) to further assess implementation effectiveness from 2004 to 2006.

      Analysis:
      Baseline Analysis Using Diffusion Study (Hawkins, Catalano et al., 2008):
      Forty imputed data sets using SA PROCMI were generated using all items on the survey to address missing data. For the dichotomized covariates and outcomes, a generalized linear mixed model using the conical logit link and binomial error function was tested with random intercepts and linear slopes at the community level. The GLIMMIX macro of the SAS procedure, MIXED, was used to run these models. For the interval-level covariates that were examined, a general linear mixed model with random intercepts and linear slopes at the community level was run in SAS PROC MIXED. The intercept for all models was positioned at 2002 to reflect the most proximate time point prior to intervention. The SAS procedure MIANALYZE was used to combine the results across the 40 imputations.

      (Hawkins, Brown et al., 2008):
      Pre- post-test ANCOVA was implemented using the general linear mixed model to test for differences in average levels of targeted risk factors between CTC and control communities. The Gaussian-distributed outcome measure consisted of the community-pair-specific targeted risk factors obtained from the grade 7 administration of the Youth Development Survey, with regression adjustment for 5th grade baseline levels of targeted risk factors, student characteristics, and community characteristics. All student characteristics were grand-mean centered. The analysis modeled all student characteristics as nonrandomly varying events and all community characteristics as fixed events. To account for hierarchical data structure, random effects were included to model the (1) the correlation of students within communities, (2) the correlation of communities within matched pairs of communities, (3) the variability of intervention effects across matched pairs of communities and (4) residual error. The intervention effect was estimated as the adjusted within-matched pair difference between CTC and control community means in targeted risk factors, and was tested against the average variation within matched pairs among the CTC versus control community means. The pre- post-test ANCOVA was conducted using HTML version 6.0, with results averaged across imputed data sets using Rubin's method.

      Multilevel discrete-time survival analysis was used to assess the effects of the CTC intervention on preventing the initiation of delinquent behavior and substance use between grades 5 and 7. To assess effects on students who had not yet initiated these behaviors, students who had already initiated delinquent behavior (22%) or substance use (27.5%) prior to the intervention were not included in the analysis.

      The ML-STSA was implemented using the generalized linear mixed model with logit link for the dichotomous outcomes. Students who did not initiate delinquent behavior or substance use, respectively, during 6th or 7th grades were treated as right-censored observations. Student- and community-level variables were included in the model as covariates to control for possible community differences; intervention condition was included in the model as a community-level variable; and random effects were included to account for variation among students within communities, communities within matched pairs of communities, intervention effects across matched pairs of communities, and residual error. The effect of the intervention was estimated as the adjusted within-matched pair difference in community-level hazard of onset between CTC and control communities, assuming proportional hazards over time, and was tested against the average variation in hazard of onset among the matched pairs of CTC and control communities. Analyses were conducted using ML-wiN version 2.02, with results averaged across imputed data sets using Rubin's rules.

      (Hawkins et al., 2009)
      Intervention effects on the incidence and prevalence of delinquency and drug use were assessed using the Generalized Linear Mixed Model (GLMM) with logit or Poisson link functions for the dichotomously coded or count-based outcomes, respectively. Random-intercept models were estimated to account for variation across time within students, among students within communities, and communities within matched pairs of communities. The effect of the CTC intervention in preventing the incidence of delinquency and drug use between grades 5 and 8 was examined using multilevel discrete-time survival analysis (ML-DTSA). The effect of the CTC intervention on reducing the prevalence of past-year delinquency and current drug use in grade 8 was assessed using a mixed-model ANCOVA. All analyses were adjusted for student- and community-level variables. As above, the results of all analyses were averaged across imputed data sets using Rubin's rules. An omnibus Group Test Statistic was applied to both the ML-DTSA and mixed-model ANCOVAs before analyses of effects on specific drugs.

      The risk of initiating delinquent behavior and drug use was assessed for those students in the sample who had not yet initiated delinquent behavior (78.8%), alcohol use (78.5%), cigarette use (91.7%), smokeless tobacco use (97.6%), marijuana use (99.4%), or the use of inhalants (91.4%), respectively, prior to the grade 5 survey. In each respective analysis, students initiating delinquent behavior or use of a specific drug in one grade were not eligible for initiation of that behavior in subsequent grades. Students who did not initiate delinquent behavior or drug use during 6th, 7th, or 8th grade were treated as right-centered observations.

      The mixed-model ANCOVA assessed the prevalence of delinquency in the past year, drug use in the past month, and binge drinking in the past 2 weeks in grade 8. In addition to student and community characteristics, respective grade 5 delinquency or drug use measures were included as pre-intervention covariates.

      (Brown et al., 2007):
      Respondent and community characteristics were included in analyses as control variables. Respondent characteristics included age, gender, positional (versus referred) status, education, how long the respondent resided in the defined community (in years), whether or not the respondent responded in both 2001 and 2004, and the community sector from which the respondent was sampled. A three-level hierarchical linear model (HLM) was used to model change in each of the three outcomes as a function of survey year nested within respondents, in turn, nested within communities. Outcomes for respondents who were not interviewed in both survey years were treated as missing data for the year in which they did not participate.

      (Hawkins et al., 2011 - Sustainability Analysis)
      Differential change in levels of average targeted risk by intervention condition from grade 5 to grade 10 was assessed using 3-level latent growth models to account for nesting of time within students and students within communities. Multilevel discrete time survival analysis was used to examine the effect of CTC on the incidence of drug use and delinquency between grades 5 and 10. All analyses included student and community covariates. The effect of CTC on grade 10 prevalence rates of substance use, delinquency, and violence was assessed using a generalized linear mixed model, with student and community characteristics and the respective preintervention measure included as baseline covariates.

      (Oesterle et al., 2015)
      Generalized linear mixed models with random intercepts were used to model variability across the sample and adjust for clustering within the randomized communities. A Poisson regression was used to calculate adjusted risk ratios and prevalence ratios. Missing data was imputed using multiple imputations assuming the data was missing at random. Models for abstinence included only those with no substance use or delinquency at baseline, while models for cumulative incidence controlled for baseline outcomes.

      Outcomes

      Delinquency and Drug Outcomes in Longitudinal Panel:

      (Hawkins, Brown et al., 2008 - Through 7th Grade)
      An ANCOVA was conducted using levels of targeted risk factors at grade 5 as the dependent variable and including intervention condition and all background variables as predictors in the model. Analysis revealed that mean levels of targeted risks observed at grade 5 were not significantly different by intervention condition, indicating that CTC and control groups had equivalent baseline levels of targeted risk factors prior to the intervention. Controlling for grade 5 levels of risk and student and community characteristics, grade 7 risk levels were significantly higher for students in control communities compared with students from CTC communities. In addition, grade 5 levels of risk, students' age, and parental education were associated with grade 7 levels of risk. No other background variables were significantly associated with levels of targeted risks in grade 7.

      There was a significant intervention effect on the initiation of delinquent behavior between 5th and 7th grades, but no significant effect on substance use initiation. Among the student characteristics, age, gender, race/ethnicity, parental education, and rebelliousness were significantly associated with onset of delinquent behavior. Students' race/ethnicity, parental education, religious attendance, and rebelliousness were significantly associated with onset of substance use. All associations were in the expected directions. The community demographic characteristics included in analyses as covariates were not associated significantly with either outcome.

      (Hawkins et al., 2009 - Through 8th Grade)
      A significant program effect on the initiation of the use of alcohol, cigarettes, and smokeless tobacco by spring of 8th grade favored students in the intervention communities. CTC communities were also less likely to initiate delinquent behaviors than control communities. Significant intervention effects on the onset of marijuana or inhalant use in the panel were not observed by the spring of 8th grade.

      The omnibus global test for overall effects on current drug use prevalence was statistically significant. The ANCOVA analyses showed significantly higher prevalence in the 8th grade in control communities compared to CTC communities of alcohol use in the past 30 days; binge drinking in the past 2 weeks; and smokeless tobacco use in the past 30 days. Control communities engaged in significantly more delinquent behaviors than CTC communities in the year prior to the 8th grade survey. Eighth-grade students in the panel in control and CTC communities did not differ significantly in the prevalence of marijuana, cigarette, inhalant, prescription drug, or other illicit drug use in the past 30 days.

      (Hawkins et al., 2011 - Through Grade 10: one year after the end of technical assistance)
      Although equivalent at baseline, the mean levels of targeted risk between grades 5 and 10 were significantly smaller in CTC than in control communities by grade 10. CTC reduced the incidence of the use of alcohol and cigarettes by grade 10 among students who had not yet initiated use by fifth grade (adjusted odds ratio 0.62 for alcohol and .54 for cigarette use). There was also a significant intervention effect on the incidence of delinquent behavior between grades 6 and 10 (adjusted odds ratio 0.79). There were no significant differences by intervention status in the incidence of smokeless tobacco, marijuana, inhalant, or prescription drug use by the spring of 10th grade.

      The prevalence of past-month cigarette use was lower for CTC communities than control communities (AOR=0.79). Tenth grade students in CTC communities had 17% lower odds of reporting any delinquent behavior and 25% lower odds of reporting any violent behavior in the past year compared with students in control communities. However, the variety of different delinquent or violent acts in which students engaged was not significantly different. There were also no differences between CTC and control communities in the incidence of binge drinking in the past 2 weeks, past-month alcohol, smokeless tobacco, marijuana, inhalant, prescription, or other illicit drug use.

      Hawkins et al., 2014 (Wave 7, grade 12)
      Students in CTC communities were more likely than students in control communities to have abstained from any drug use, drinking alcohol, smoking cigarettes, and engaging in delinquency. They were also less likely to ever have commited a violent act. There were no significant differences by intervention group in the prevalence of past-month or past-year substance use, or past-year delinquency or violence.

      Oesterle et al, 2015 (Wave 8, Age 19)
      At the 9-year follow-up, males in the intervention group experienced sustained effects and had higher rates of abstinence from cigarette use and delinquency (risk ratios of 1.22-1.33 indicate small effects). There were no other significant differences among males, females, or the full sample.

      Community-Level Prevention Service Outcomes:

      (Brown et al., 2007)
      CTC communities exhibited significantly greater increases in adopting a science-based approach to prevention, collaboration across community sectors, and collaboration regarding specific prevention activities between 2001 and 2004 relative to control communities.

      (Rhew et al., 2011 draft)
      CTC had evidence of sustained effects on prevention system characteristics 1.5 years after study-funded resources for CTC ended. Leaders from CTC communities reported higher levels of adoption of a science-based approach to prevention and a higher percentage of funding desired for prevention activities in 2009 than did leaders in control communities. CTC communities showed a higher increase over time in community norms against adolescent drug use compared to control communities. There was no evidence of a main effect of CTC on community collaboration for prevention, general community support for prevention, or use of the social development strategy.

      (Gloppen et al., 2012)
      The extent to which coalitions continued to meet specific benchmarks was measured 20 months following the end of study support for CTC. Only 1 of 12 CTC coalitions had dissolved by winter of 2009. Although funding had decreased in 7 of the 11 coalitions, two-thirds of the coalitions reported having a paid staff person. CTC coalitions continued to report significantly higher scores on the benchmarks of phases 2 through 5 of the CTC system than did the prevention coalitions in the control communities, indicating that CTC coalitions maintained a more scientific approach to prevention than coalitions that did not receive CTC training.

      (Kuklinski et al., 2013)
      Many CTC coalitions were resilient two years after the economic downturn that began December, 2007. CTC coalitions continued to implement science-based prevention to a significantly greater degree than control coalitions (7 of 11 coalitions maintained 2007 implementation levels through 2009).

      Implementation Fidelity:

      (Quinby et al., 2008)
      CTC implementation fidelity ratings averaged across three groups of raters show that between 89% and 100% of the CTC milestones in the first four phases of CTC implementation were "completely met" or "majority met" in the 12 intervention communities, indicating that the first four phases of the CTC system were well implemented in the communities in this trial. There was high overall agreement (95%) in milestone ratings among the different groups of raters. Five milestones were rated as either "very challenging" or "mostly challenging" for half or more of the communities: preparing archival data to supplement the CTC Youth Survey Data, identifying resources required for new programs and policy implementation, addressing readiness issues, securing a champion, and engaging all stakeholders to support the community action plan.

      (Fagan et al., 2008a)
      By using the CTC model to select and monitor the quality of prevention activities, the 12 CYDS communities replicated 13 prevention programs with high rates of adherence to the programs' core components and in accordance with dosage requirements regarding the number, length, and frequency of sessions. Adherence scores ranged from 73% to 99% across all program replications, indicating that program staff taught the majority of program objectives and ensured completion of most of the program components. Dosage scores were also high, as 94% of the dosage criteria were met across all communities. In addition, 81% of the program cycles delivered all required lessons, in the specified amount of time, and with the recommended frequency of delivery.

      (Fagan et al., 2012)
      During the active phase of research (2007, 3.5 years after CTC was begun in communities) and at the end of grant-supported activities (2010, 6.5 years after baseline and 1.5 years folllowing the end of training, TA, and funding to intervention communities), according to agency directors and program providers, CTS communities implemented significantly more tested and effective prevention programs and had higher rates of program sustainability compared to control communities at both time periods. CTC sites reached more children and families with prevention services at each time period, but this was only significant during the intervention stage. Only one significant fidelity effect emerged, which indicated that CTC sites provided more program oversight during the sustainability phase.

      (Fagan et al., 2008b)
      According to implementer and observer reports, across all 16 programs implemented in CTC communities, large proportions of required material were taught and core components delivered, nearly all lessons were offered in accord with the length and frequency specified by program developers, implementers were prepared and enthusiastic and used a variety of teaching techniques to convey material, and high levels of engagement from program participants were observed.

      Feinberg, M. W., Greenberg, M. T., Osgood, D. W., Sartorius, J., & Bontempo, D. (2007). Effects of the Communities That Care model in Pennsylvania on youth risk and problem behaviors. Prevention Science, 8, 261-270.

      Feinberg, M. W., Jones, D., Greenberg, M. T., Osgood, D. W., & Bontempo, D. (2010). Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science, 11, 163-171.

      This was the first broad-scale quasi-experimental evaluation of youth outcomes in communities using the Communities That Care Program. A total of 15 risk factors and 6 outcomes (substance use and delinquent behaviors) were analyzed for a large sample of youth in Pennsylvania public and private schools.

      Evaluation Methodology

      Design: Data was obtained utilizing the results from a state-funded surveillance survey of a large sample of adolescents in public and private schools, the Pennsylvania Youth Survey (PAYS). Although not designed as an evaluation of CTC, PAYS utilized the CTC Youth Survey. The survey results included communities that do and do not have CTC coalitions. The PAYS data was collected in 2001 and 2003, by the PA Commission on Crime and Delinquency. In both years, the sample consisted of students in schools that participated as part of a stratified random sampling procedure or schools that volunteered to participate in the survey. In 2001 and 2003 a random sample of schools was chosen to participate as follows: Schools with over 19 (2001) or 50 (2003) students were divided into 6 regions of the state, and for each of the 4 targeted grades (6, 8, 10, and 12) in each region, a separate random sample was drawn. Each school's grade was assigned a likelihood of participation equivalent to the proportion of the regional student population comprised by the school's grade. In addition to the targeted grades, "piggyback" grades at a school could be included in the survey at a financial discount to the school over the typical survey cost. This procedure yielded 43,842 respondents in 2001 and 38,845 respondents in 2003. In 2003, additional schools volunteered to participate in the survey in order to monitor risks and problems in their communities. The full data sets available contained data on 92 school districts and 43,842 students in 2001 and 159 school districts and 101,988 students in 2003. A total of 50 school districts in 2001, and 102 districts in 2003 were associated with a CTC site. For 2001, the mean time since the initial grant began was 16.8 months. For 2003, the mean time was 42.9 months.

      From the full data sets, data was removed for (1) 3,752 students in 2003 because the school district identifier was missing in the data set; (2) 6 school districts whose association with CTC could not be determined; and (3) private schools because CTC sites typically focus programs in public, not private contexts.

      Sample: The school districts in the combined 2001-2003 sample had an average of 6% of households below the poverty line (range 1 to 22); and an average of 7.9% female-headed households (range 3-35.2). Although many of the school districts in the rural and small town areas were predominantly white, some areas had predominant minority populations. The average percentage for nonwhites was 6.1 (range 0.7 to 77.8%), and the average percent Hispanic was 2 (range 0.2 to 22.1%).

      Measures: The CTC Youth Survey (CTCYS) was the student self-report measure used to assess risk and protective factors for adolescent behavior problems as well as substance use and delinquency. Please see the above write-up for a detailed description of the CTCYS. The 6 outcome measures used for this study included an 8-item scale assessing delinquent behaviors in the past year; a seven-item "drug involvement" scale assessing use of several substances in the past 30 days (alcohol, smoking and smokeless tobacco, marijuana, LSD, cocaine, inhalants); and 4 single items assessing past 30-day alcohol use, binge drinking, being drunk or high in school, and tobacco use. Binge drinking and being drunk in school were not assessed for 6th graders, given the low rates of these behaviors at that age.

      Community demographic information was gathered from 2000 census data aggregated at the school district level by the National Center for Educational Statistics. Variables used included percentage of households below the poverty line, percentage of households with nonwhite racial/ethnic backgrounds, population size, and percentage of female-headed households. The indicator of population density was based on census figures and obtained from publicly available data maintained by the Center for Rural Pennsylvania.

      Data was collected from each CTC site regarding which programs they implemented, age groups or grades that participated, and dates of implementation. Each program was then checked to see if it was on the SAMHSA list of effective or model programs and if so, was coded as evidence-based. Grade cohorts were also coded at each school as potentially in the range or out of the range that could be affected by any of the CTC-sponsored programs.

      Analysis: Feinberg et al., 2007: Separate analyses were conducted for the two waves of 2001 and 2003 PAYS data, and separately by grade (6th, 8th, 10th, and 12th) since risk and outcomes vary in a nonlinear fashion across grades. Analyses were conducted with statistical techniques that accounted for the clustering of students within school districts (multilevel regression models: SAS Proc Mixed). The indicator of community poverty was included in all analyses as a covariate; and a dichotomous variable representing condition (CTC vs. non-CTC) defined the program effect. Percentage of families living in poverty was used as a control (covariate) in all analyses.

      Outcomes

      Feinberg et al., 2007
      Sample Adjustments and Equivalence: Analyses indicated that CTC districts did not significantly differ from non-CTC districts regarding percentage of nonwhite families, population size, population density, or percentage of female-headed family households. However, results indicate that school districts in CTC sites had a higher average proportion of poor family households (7.66%) compared to non-CTC districts (6.55%). The common region of support was examined and districts outside this region were eliminated. As a result, 11 CTC districts were eliminated that were higher on the poverty variable than any non-CTC districts (8 of these districts contributed data in 2001, and 7 districts contributed data in 2003). After eliminating these districts, the two groups no longer differed on level of poverty or other variables.

      Results:
      Overall, the pattern of results indicated that CTC school districts had lower levels of some risk factors and rates of substance use and delinquent behaviors than did non-CTC school districts. The results for all CTC sites compared to non-CTC sites indicated 4 significant comparisons favoring CTC in 2001 and 16 in 2003. Findings were particularly strong for the 6th and 12th grades in 2003. In addition, there were many fewer significant comparisons favoring non-CTC sites across the 2 years than would be expected by chance.

      The first follow-up analysis included only those CTC grade cohorts that could have been directly impacted by an evidence-based model program. Because numerous CTC sites began implementing programs between 2001 and 2003, there were too few such cases in 2001 to analyze. For the 2003 data, analysis of cohorts most likely to be affected resulted in 22 significant comparisons favoring the CTC sites.

      Examination of the distribution of significant effects indicated differential effects by grade and by risk factor/outcome. For the 2003 data, limited to CTC grade cohorts expected to be impacted by evidence-based programs, the greatest number of significant effects was found in the 6th grade. CTC influence on risk factors for the 6th grade included poor family supervision and discipline, friends' behaviors and attitudes, and the individuals own attitudes; analyses also indicated positive effects on each of the outcomes for 6th grade, with the exception of the composite drug use index. For the 6th through the 10th grades, analyses yielded consistent evidence of CTC influence on the outcome of delinquent behavior. And for the 12th grade, evidence of CTC effects was demonstrated for alcohol use and the multi-item measure of drug involvement.

      Given that some of the variables were skewed, the data was reanalyzed utilizing models in SAS's PROC GLIMMIX procedure with alternate distributional assumptions. Poisson distributions were utilized and Ordered multinomial models were utilized where appropriate. Results were very nearly identical to the original models utilizing PROC MIXED with assumed normal distributions. In the 2003 expected impact analyses, 21 models yielded significant results with the reanalyses, compared to 22 in the original analyses.

      Effect sizes were computed as odds ratios with multinomial models. These analyses were conducted for grades and risk factors/outcomes that had demonstrated significant differences between CTC and non-CTC sites in the 2003 expected impact subsample. Responses were recoded to facilitate analyses; for the variables analyzed, recoding yielded between 7 and 16 response levels. Because there were more than two levels of response for each variable, the resulting odds ratio was not interpreted as the likelihood of use vs. nonuse due to CTC as would be the case if typical logistic models had been used. Instead, the odds ratio resulting from this ordinal logistic model reflects the likelihood of a student in a non-CTC district reporting a higher level of risk or substance use than a student in a CTC district. Odds ratios for the models analyzed ranged from 1.12 to 1.36, with an average of 1.23.

      Feinberg et al., 2010
      As previous cross-sectional findings were open to the criticism that self-selection of communities into CTC may have biased results, this analysis utilized longitudinal data and tested for impact on change. The authors state that this design to a large extent removes the possibility that selection bias is responsible for the findings. It was not possible to track individuals over time due to the anonymous survey, thus the analyses concern changes in groups of individuals over time. For example, 6th graders responding to the 2001 survey were considered the same grade cohort as the 8th graders from the same school responding to the 2003 survey. Thus, subjects were nested within measurement periods which were nested within school district. In this study, each grade cohort at each school that was ever exposed to an evidence-based program was coded a "1" as having an "expected impact" by a universal EBP. All other grade cohorts (i.e., combining non-CTC and non-expected impact CTC grade cohorts) were coded as 0. Three-level hierarchical models were used to examine change in risk/protective factors, grades, delinquency and substance use over time.

      There were no differences between CTC and non-CTC communities' grade cohorts in change of risk/protective factors, grades, and substance use. There was a significant difference between groups in delinquency, with youth in CTC communities, relative to control communities, demonstrating less growth in delinquency. In the models that tested Expected-Impact CTC grade-cohorts and comparison grade-cohorts (i.e., non-CTC grade cohorts and all other CTC grade cohorts), there were significant and beneficial intervention effects for risk/protective factors, academic grades, and delinquency, but not substance use.

      Brief Bullets:

      • In general, the pattern of findings shows that communities employing the CTC model had lower levels of risk factors and problem behaviors (delinquency and alcohol/drug use) than communities not employing CTC.
      • When contrasting grade cohorts that were actually exposed to evidence-based programs ("expected impact" cohorts), compared to grade cohorts in the same schools that were not exposed to EBPs combined with students from non-CTC schools, youth in "expected impact" CTC grade cohorts demonstrated significant and beneficial effects for risk/protective factors, academic grades, and delinquency.

      Limitations: The removal of 11 high-poverty CTC school districts from the analyses in order to gain an equivalent sample of CTC and non-CTC schools is problematic and also limits the ability to understand the effects of CTC among lower poverty communities. The possibility of a selection bias exists due to the quasi-experimental study design, as districts could have been selected in such a way as to bias the outcomes. Some schools were randomly selected, but other schools joined the sample by volunteering. The majority of the cases in 2003 came from volunteering schools, and most of the positive findings were observed mainly in 2003. It would be important to understand how volunteerism was related to the CTC condition and how the inclusion of volunteer schools may have affected the results.

      Arthur, M. W., Ayers, C. D., Graham, K. A., & Hawkins, J. D. (2003). Mobilizing communities to reduce risk for drug abuse: A comparison of two strategies. Pp. 129-144 in The Handbook of Drug Abuse Prevention: Theory, Science, and Practice, edited by Z. Sloboda and W. J. Bukoski. Kluwer Academic/Plenum Publishers: New York, NY.

      This paper presents findings from process evaluations of two distinct community mobilization interventions without randomization to conditions and without a control group. Two independent statewide community mobilization efforts initiated in 1990 are examined, the Oregon TOGETHER! Project and the Washington State Community Youth Activity Program (CYAP). Community prevention planning boards in each project received training on risk and protective factors for alcohol and other drug abuse, how to assess and prioritize risk factors in their communities, and how to develop strategic action plans tailored to the prioritized risks. Evaluations of the two projects assessed the success for these boards at developing risk-reduction plans and programs.

      The Oregon TOGETHER! project explicitly involved key community leaders as well as grassroots community members during the community mobilization process. In contrast, the Washington State CYAP project focused its mobilization effort on grassroots community members and youths. The Oregon TOGETHER! project used a series of three training sessions, applying the CTC model to guide the community boards through the process. The Washington State CYAP project used a single training seminar and the Together We Can planning kit.

      A comparative analysis of process evaluation data from the two projects assessed the extent of (1) community mobilization and key leader and youth development; (2) adoption of the risk- and protective-factor prevention planning strategy, including completion of community risk and resource assessments and development of strategic action plans; and (3) implementation of risk-and protection-focused prevention strategies. The Oregon TOGETHER! project conducted telephone surveys of community board members during the spring of 1992, approximately 1 year following the third and final training workshop provided to participating communities, to assess each board's progress in developing action plans and implementing activities to reduce risks for substance abuse. In addition, action plans submitted by the community boards were analyzed to assess the completeness of the plans and the potential effectiveness of planned strategies for reducing identified risk factors and enhancing protective factors.

      Washington State CYAP project collected information about the development and activities of each term through follow-up telephone interviews with team leaders and members in December 1991, 10 months following the CYAP leader training workshop. These interviews assessed team characteristics, members' understanding of the risk- and protective-factor model, and prevention activities carried out by the teams. In addition, the risk-reduction action plans were analyzed to assess completeness and the potential effectiveness of planned strategies.

      Both approaches were successful at mobilizing community boards to plan and implement prevention activities, and both approaches were able to recruit and involve the types of community members they targeted on their planning boards, the Washington State CYAP was successful at involving youth in planning youth-oriented activities. The Communities That Care process used in the Oregon TOGETHER! project was effective at involving key community leaders in organizing prevention boards in their communities. However, the Oregon TOGETHER! project was more successful than the Washington State CYAP project at promoting planning and program activities aimed at specific, empirically based risk factors identified through a community risk assessment process. Even without funding, Oregon TOGETHER! prevention boards were more likely than the funded Washington CYAP community teams to collect empirical indicators of community risk and protective factors, develop action plans describing strategies to reduce prioritized risk factors, and implement programs aimed at reducing these risk factors.

      Brief Bullets:

      • The Oregon TOGETHER! project was more successful than the Washington State CYAP project at promoting planning and program activities aimed at specific, empirically based risk factors identified through a community risk assessment process.
      • Even without funding, Oregon TOGETHER! prevention boards were more likely than the funded Washington CYAP community teams to collect empirical indicators of community risk and protective factors, develop action plans describing strategies to reduce prioritized risk factors, and implement programs aimed at reducing these risk factors.

      Harachi, T. W., Ayers, C. D., Hawkins, J. D., Catalano, R. F., & Cushing, J. (1996). Empowering communities to prevent adolescent substance abuse: Process evaluation results from a risk- and protection-focused community mobilization effort. The Journal of Primary Prevention, 16, 233-254.

      This evaluation examines the TOGETHER! Communities for Drug-Free Youth use of the Communities That Care strategy for community mobilization for the comprehensive, risk- and protection-focused prevention of adolescent substance abuse. The project mobilized, trained, and assisted Oregon communities to assess local risks and resources and to develop and implement comprehensive prevention plans tailored to their priority risks. This was not a randomized, controlled experimental trial.

      The project mobilized 35 Oregon communities to conduct quantitative assessments of community risk factors and protective resources, to develop comprehensive prevention plans incorporating promising approaches to priority risk factors, and to implement their plans. At the end of the four-year demonstration, 31 communities remained active in the project which Oregon had institutionalized, and 28 of them were involved in the implementation of risk reduction programs. Within a year after training, 28 boards had completed comprehensive risk-focused prevention plans. Less than a year into the planning and implementation phase, 27 had begun implementing risk reduction strategies. These findings appear to indicate that once put in place in communities, the CTC system can be maintained for several years even without significant dedicated funding.

      Brief Bullets:

      • Once put in place in communities, the CTC system can be maintained for several years even without significant dedicated funding.

      Rhew, I. C., Hawkins, J. D., Murray, D. M., Fagan, A. A., Oesterle, S., Abbott, R. D., & Catalano, R. F. (2016). Evaluation of community-level effects of Communities That Care on adolescent drug use and delinquency using a repeated cross-sectional design. Prevention Science, 17, 177-187.

      Evaluation Methodology

      Design:

      Recruitment: The data for this study was from the CYDS, a community-randomized controlled trial of 24 small towns in Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington. The CYDS is discussed further in Study 1. The towns were selected from a larger sample of 41 communities participating in an observational study of the diffusion of science-based prevention strategies. Of these, 24 were eligible and consented to participate.

      Assignment: The participating communities were matched within states according to population size, racial/ethnic composition, crime rate, and socioeconomic indicators. From each pair, one town was randomly assigned to the treatment condition the other to the control condition. Communities in the control condition did not receive any training in or funding for CTC activities from the CYDS project during the course of the study.

      The repeated cross-sectional design compared all students in community schools at each survey year rather than followed individuals longitudinally. The surveys excluded students present at the start of the program who left the schools and included new students who entered the schools after the program start. All comparisons from baseline to follow-up thus included somewhat different samples of students.

      Attrition: Assessments occurred at baseline (2000 and 2002) and follow-up (2006 and 2008). Across the years of the study, the response rate among 6th graders ranged from 79 to 87%, for 8th graders ranged from 72 to 84%, and for 10th graders ranged from 60 to 78%.

      Sample: The sample was approximately half male (47.7 to 49.9%) and largely white (64.1% to 77.2%). The average age was 11.6 among 6th graders, 13.6 among 8th graders, and 15.6 among 10th graders.

      Measures: The study included 9 measures of substance use and 12 measures total. Participants were asked to report use of alcohol, cigarettes, marijuana, and smokeless tobacco in the past 30 days and in their lifetimes. In addition, participants were asked whether or not they participated in binge drinking in the past 2 weeks. The study also included 2 measures of delinquency, based on whether or not the individual had sold illegal drugs, stole or tried to steal a motor vehicle, attacked someone, brought a handgun to school, or were arrested. In assessments after the baseline, 3 additional behaviors were added: stole something worth $5, purposely damaged or destroyed someone else’s property, and shoplifted. The study recoded these responses into a count of the number of delinquent behaviors and a dichotomous report of any delinquent behavior. Finally, the study had one measure of change in risk and protective factors prioritized by the community.

      Analysis: To examine the effects of CTC on youth outcomes, the study used a two-stage ANCOVA approach. In the first stage, the study used individual-level data to estimate the adjusted prevalence of each problem behavior for a community in a given year, by grade. This was then used to create an adjusted pooled baseline level in 2000 and 2002 for each outcome and community and an adjusted pooled follow-up level in 2006 and 2008 for reach outcome and community. In the second stage, the study used a mixed effects model in which the adjusted post-intervention pooled community prevalence was regressed on study condition, the pooled baseline level for the community, percent free or reduced price lunches, and number of students in the community.

      The study also conducted analyses of changes in prevalence over time within a specific pseudo cohort followed from 6th grade to 10th grade. The study assumed that most students in 6th grade in 2004 were in 10th grade in 2008, creating the pseudo cohort. The analyses used for this pseudo cohort were similar to those above for the grade samples.

      Intent-to-Treat: The study used all available data with multiple imputation for missing data.

      Outcomes

      Implementation Fidelity: The study did not include any reports of implementation fidelity.

      Baseline Equivalence: As discussed above, the communities were assigned in matched pairs based on population size, racial/ethnic composition, crime rate, and socioeconomic indicators. The study did not provide information on equivalence of baseline outcome scores of either communities or students.

      Differential Attrition: Although there were no formal tests for attrition, the study examined differences between conditions on stable demographic characteristics at posttest. Program completers in the treatment group were more likely to be white as compared to the completers in the control group. The study did not discuss differential attrition in the outcome measures.

      Posttest: The study found one possible iatrogenic effect on antisocial behavior among 6th graders. Those in the treatment group self-reported significantly more antisocial behavior at posttest than the control group. None of the other outcomes were significant among 6th graders and none of the 12 outcomes were significant for either 8th or 10th graders.

      In its analysis of a pseudo cohort, the study found that of 12 outcomes, only lifetime use of smokeless tobacco saw significant improvement from 6th to 10th grade when comparing the treatment to control groups.

      Long-Term: No long-term follow-up was reported.