Please take our brief survey

Blueprints Programs = Positive Youth Development

Return to Search Results

Model Program Seal

Promoting Alternative Thinking Strategies (PATHS)

Blueprints Program Rating: Model

A classroom-based social emotional learning program for elementary students to reduce aggression and behavior problems in children. The PATHS curriculum teaches skills in five conceptual domains: self-control, emotional understanding, positive self-esteem, relationships, and interpersonal problem solving.

  • Director
  • Co-developers
  • Prevention Research Center
  • Penn State University
  • 110 HDFS - Henderson Building South
  • University Park, PA 16802-6504
  • (814) 863-0112
  • mxg47@psu.edu
  • Antisocial-aggressive Behavior
  • Delinquency and Criminal Behavior
  • Externalizing

    Program Type

    • Cognitive-Behavioral Training
    • School - Individual Strategies
    • Skills Training
    • Social Emotional Learning

    Program Setting

    • School

    Continuum of Intervention

    • Universal Prevention (Entire Population)

    A classroom-based social emotional learning program for elementary students to reduce aggression and behavior problems in children. The PATHS curriculum teaches skills in five conceptual domains: self-control, emotional understanding, positive self-esteem, relationships, and interpersonal problem solving.

      Population Demographics

      PATHS is implemented with elementary school age youth (grades K-6). A modified version to be age-appropriate for preschool students (called Head Start REDI) is treated separately by Blueprints. PATHS has been shown to be effective for both males and females, different ethnic and socio-demographic populations, and a wide variety of populations, including students in regular education and special needs settings.

      Age

      • Late Childhood (5-11) - K/Elementary

      Gender

      • Male and Female

      Gender Specific Findings

      • Male

      Race/Ethnicity

      • All Race/Ethnicity

      Race/Ethnicity/Gender Details

      Although samples have included the entire range of SES strata, as well as children from a wide diversity of ethnic, cultural, and family-structural backgrounds, there has been only one analysis that considered differences in program effects by race (CPPRG 1999). In examining the two sites with sufficiently diverse samples for comparison (Nashville and Seattle), it found similar program effects among African Americans and European Americans.

      For gender, one analysis found that PATHS boys had improved scores as contrasted with comparison boys on the overall teacher ratings of behavior and the dimensions of frustration tolerance and peer sociability. Another study found no gender differences on prosocial behaviors and emotional regulation as rated by teachers, although peers rated control boys as more aggressive and hyperactive than intervention boys (CPPRG, 2010).

      Risk: poor self-control, lack of commitment to school, favorable attitudes toward problem behavior and early initiation, impulsiveness, and peer rejection.

      Protective: prosocial orientation, positive peer relations, bonding to school.

      • School
      • Peer
      • Individual
      Risk Factors
      • Individual: Antisocial/aggressive behavior, Early initiation of antisocial behavior, Favorable attitudes towards antisocial behavior, Hyperactivity*
      • School: Low school commitment and attachment, Repeated a grade
      Protective Factors
      • Individual: Clear standards for behavior, Problem solving skills*, Prosocial behavior, Skills for social interaction*
      • Peer: Interaction with prosocial peers
      • School: Opportunities for prosocial involvement in education, Rewards for prosocial involvement in school

      *Risk/Protective Factor was significantly impacted by the program.

      See also: Promoting Alternative Thinking Strategies (PATHS) Logic Model (PDF)

      The PATHS curriculum is a comprehensive program for promoting emotional and social competencies and reducing aggression and behavior problems in elementary school-aged children (grades K-6) while simultaneously enhancing the educational process in the classroom. The evaluation of the preschool version, called Head Start REDI, is treated separately by Blueprints.

      The Grade Level PATHS Curriculum consists of separate volumes of lessons for each grade level (K - 6), all of which include developmentally appropriate pictures, photographs, posters, and additional materials (www.channing-bete.com/prevention-programs/paths/). Five conceptual domains, integrated in a hierarchical manner, are included in PATHS lessons at each grade level: self-control, emotional understanding, positive self-esteem, relationships, and interpersonal problem-solving skills. Throughout the lessons, a critical focus of PATHS involves facilitating the dynamic relationship between cognitive-affective understanding and real-life situations. PATHS is designed to be taught two to three times per week (or more often if desired, but not less than twice weekly), with daily activities to promote generalization and support ongoing behavior. PATHS lessons follow lesson objectives and provide scripts to facilitate instruction, but teachers have flexibility in adapting these for their particular classroom needs. Although each unit of PATHS focuses on one or more skill domains (e.g., emotional recognition, friendship, self-control, problem solving), aspects of all five major areas are integrated into each unit. Moreover, each unit builds hierarchically upon and synthesizes the learning which preceded it.

      The PATHS curriculum is designed to be used by educators and counselors in a multi-year, universal prevention model. To encourage parent involvement and support, parent letters, home activity assignments, and information are also provided.

      The PATHS curriculum is a comprehensive program for promoting emotional and social competencies and reducing aggression and behavior problems in elementary school-aged children (grades K-6) while simultaneously enhancing the educational process in the classroom.

      PATHS is now available by grade level in the following grades: Kindergarten, Grade 1, Grade 2, Grade 3, Grade 4, and Grade 5/6. The original multi-year version is also available from the publisher. The grade level versions maintain all key elements of the original version and now organize them more discretely by grade levels. The preschool version of the program, called Head Start REDI, is treated separately by Blueprints.

      PATHS targets five major conceptual domains: (1) self control; (2) emotional understanding; (3) positive self-esteem; (4) relationships; and (5) interpersonal problem solving skills. In addition, a 30-lesson non-mandatory supplementary unit reviews and extends PATHS concepts that are covered in other units.

      The PATHS curriculum is designed for use by regular classroom teachers. Lessons are sequenced according to increasing developmental difficulty and designed for implementation in approximately 20-30 minutes 2 to 3 times per week. The curriculum provides detailed lesson plans, exact scripts, suggested guidelines, and general and specific objectives for each lesson. However, the curriculum has considerable flexibility so that it can also be integrated with an individual teacher's style. Lessons include such activities as dialoguing, role-playing, story-telling by teachers and peers, social and self-reinforcement, attribution training, and verbal mediation. Learning is promoted in a multi-method manner through the combined use of visual, verbal, and kinesthetic modalities.

      PATHS incorporates seven factors considered critical for effective, school-based SEL curricula. These included the use of:

      • an integration of a variety of successful approaches and promising theories
      • a developmental model, including neuropsychological brain development
      • a multi-grade level paradigm
      • a strong focus on the role of emotions and emotional development
      • generalization of skills to everyday situations
      • ongoing training and support for implementation
      • multiple measures of both process and outcome for assessing program effectiveness

      PATHS is based on five conceptual models. The first, the ABCD (Affective-Behavioral-Cognitive-Dynamic) Model of Development focuses on the promotion of optimal developmental growth for each individual. The ABCD model places primary importance on the developmental integration of affect (i.e., emotion, feeling, mood) and emotion language, behavior, and cognitive understanding to promote social and emotional competence. The second model incorporates an eco-behavioral systems orientation and emphasizes the manner in which the teacher uses the curriculum model and generalizes the skills to build a healthy classroom atmosphere (i.e., one that supports the children's use and internalization of the material they have been taught). The third model involves the domains of neurobiology and brain structuralization/organization. PATHS incorporates strategies to optimize the nature and quality of teacher-child and peer-peer interactions that are likely to impact brain development as well as learning. The fourth paradigm involves psychodynamic education (derived from Developmental Psychodynamic Theory) which aims to coordinate social, emotional, and cognitive growth. Finally, the fifth model includes psychological issues related to emotional awareness, or as it is more popularly labeled, emotional intelligence. As such, a central focus of PATHS is encouraging children to discuss feelings, experiences, opinions, and needs that are personally meaningful, and making them feel listened to, supported, and respected by both teachers and peers. As a result, the internalization of feeling valued, cared for, appreciated, and part of a social group is facilitated, which, in turn, motivates children to value, care for, and appreciate themselves, their environment, their social groups, other people, and their world.

      • Biological - Neurobiological
      • Cognitive Behavioral
      • Self Efficacy
      • Skill Oriented
      • Social Learning

      Results from randomized and quasi-experimental studies published between 1983 and 2014 are described here. Evaluations have examined the effectiveness of the PATHS curriculum on the emotional development of elementary school-aged children using random assignment to either a treatment condition or a control group. These evaluations have included three different populations including children in regular education, deaf/hearing impaired, and a variety of special education-classified children. The intervention model focused on increasing children's ability to discuss emotions, utilize a larger emotions vocabulary, and understand meta-cognitive aspects of emotions (e.g., awareness of cues for recognizing emotions in oneself and others, understanding display rules, simultaneity of emotional experience, and strategies for changing emotional states), effectively manage peer conflict and peer relations, and utilize self-control and emotional regulation skills to resolve both intrapersonal and interpersonal conflict. Two independent replications of PATHS using cluster randomized designs are also reported (Crean & Johnson, 2013; Schonfeld et al., 2014). Crean & Johnson (2013) examined program effects on children's aggression outcomes, while Schonfeld et al. (2014) examined child academic outcomes. In addition, a longitudinal study is reported that examines adolescent delinquency, substance use and antisocial behavior outcomes of students from schools randomized to one of four conditions (PATHS, Triple-P, PATHS+Triple-P, control) seven and nine years after program commencement (Averdijk et al., 2016).

      Study 1 (Seattle Study): Greenberg et al., 1995

      • PATHS was effective in improving children's range of affective vocabulary, their ability to provide appropriate personal examples of the experience of basic feelings, their beliefs that they can hide, manage, and change their feelings, and their understanding of cues for recognizing feelings in others.
      • Among children at higher risk (special education), PATHS significantly improved both their understanding of how others manage and hide their feelings, and how feelings can be changed.
      • Among regular education children only, PATHS resulted in improved comprehension of complex feeling states.
      • No significant effects were found with the meta-cognitive recognition of one's own feelings, understanding of the simultaneity of feelings, or understanding of how one can manage one's own feelings.

      Study 2 (Mediating Role of Neurocognition): Riggs et al., 2006

      • Tests on the mediating effects of neurocognition demonstrated that inhibitory control at immediate posttest significantly mediated the relation between experimental condition and both teacher-reported externalizing and internalizing behavior at 1-year follow-up.

      Study 3 (Study of Special Needs Kids): Kam, Greenberg, and Kusche, 2004

      • Lower scores on internalizing symptomology for special-needs students at the two-year follow-up.
      • Special needs children who began with high rates of self-reported conduct problems and depressive symptoms had significant reductions at post-test and maintenance of these reductions for two years post-intervention.
      • At three-year follow-up, the trajectories for internalizing and externalizing behavior and depression continued to decline for intervention students, whereas control students' trajectories increased (internalizing/externalizing) or declined at a lower rate (depression). The size of the negative feelings vocabulary increased in the treatment groups, while the size of positive feelings vocabulary did not. There was a marginally significant reduction in the percentage of aggressive solutions intervention children generated, and there was a significant increase in the percentage of solutions that were nonconfrontational and indicated self-control.

      Study 4 (Study of Deaf Children): Greenberg and Kusche, 1998

      • Significant improvement in social problem-solving skills, emotional recognition skills, reading achievement and non-verbal planning skills, and teacher and parent-rated social competence.
      • Teacher ratings of behavior indicated significant improvements in social competence and in frustration tolerance.
      • Mediational analyses indicated that (a) improvement in emotional understanding was related to lower parent report of lowered externalizing problems at home; (b) improvement in role-taking skills was related to higher teacher ratings of emotional adjustment, and reductions in behavior problems at school and at home; and (c) improvement in problem-solving was related to higher teacher ratings of emotional adjustment and social competence and decreases in behavior problems at home and school.
      • One- and two- year post-test results indicated maintenance of effects. Results with the wait-list control group indicated replication of effects in the second sample.

      Study 5: Kam, Greenberg, and Walls, 2003 (PATHS combined with Big Brothers/Big Sisters)

      • When both the quality of implementation and principal support were high, students demonstrated significantly greater reductions in aggression and behavioral dysregulation, and significant increases in emotional competence when compared to students in the school with the lowest principal support.

      Study 6: Curtis & Norgate, 2007

      • Significant improvement for 5 to 7 year olds on all five subscales of the SDQ: Emotional symptoms, Conduct problems, Hyperactivity, Peer problems, and Consideration, for the treatment schools but not for the comparison schools.

      Study 7: (Switzerland Study): Malti et al., 2011; Malti et al., 2012; Averdijk et al., 2016

      • A developer-independent study demonstrated the following significant program effects, compared to a control group: reduction in children’s aggressive behavior (based on parent and teacher reports); reduction in ADHD symptoms (based on teacher reports); and no beneficial program effect on children’s social competence.
      • Two-year follow-up results indicate significant differences between the PATHS intervention group and the control group on teacher-reported aggression and ADHD.
      • Follow-up treatment effects were moderated by level of moral emotion at baseline (aggression and ADHD), and initial level of competent problem-solving strategies (ADHD).
      • Follow-up seven and nine years post program commencement showed a reduction in adolescent delinquency (i.e., fewer self-reported policy contacts).

      Study 8: Seifert et al., 2004

      • A posttest-only analysis of two cohorts in a single inner-city school in Providence, Rhode Island, demonstrated intervention benefits on two of ten outcomes: a global social competence rating done by the interviewers and a social-emotional competence composite.

      Study 9: CPPRG, 2010

      • After a 3-year intervention, the universal version of PATHS combined with Fast Track significantly reduced teacher-rated scores on authority acceptance, cognitive concentration, and social competence.
      • Among boys, the intervention reduced peer ratings of being aggressive and hyperactive.

      Study 10: Crean & Johnson, 2013

      • Significant reductions in teacher-reported conduct problems.
      • Significant linear effects on student-reported normative beliefs about aggression, aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies.

      Study 12: Schonfeld et al., 2014

      For treatment students, as compared to control students, higher test score proficiency in:

      • Reading in 4th grade
      • Writing in 5th and 6th grades
      • Math in 4th grade

      Study 14: Fishbein et al., 2016

      Compared to control students, children in treatment schools improved:

      Teacher-Rated (non-independent) Outcomes

      • Aggression
      • Internalizing
      • Social competence
      • Emotion regulation
      • Prosocial behavior
      • Impulsivity
      • Inattention
      • Teacher-child closeness and conflict
      • Peer relationship problems
      • Academic skills

      Child Performance Outcomes

      • Inhibition task accuracy
      • Motor-skills impulsivity performance

      Across multiple studies, PATHS relative to a control group showed:

      • Lower rate of conduct problems and externalizing behaviors (e.g., aggression),
      • Lower internalizing scores and depression,
      • Better understanding of cues for recognizing feelings in others,
      • Better ability to resolve peer conflicts, identify feelings, identify problems, and greater empathy for others,
      • Less anger and attribution bias,
      • Reduction in ADHD symptoms, and
      • Better scores on measures of authority acceptance, cognitive concentration, and social competence.
      • Higher test score proficiency in reading, writing, and math
      • Reduction in adolescent delinquency

      Significant Program Effects on Risk and Protective Factors:

      • Improvements in social problem solving, emotional understanding, and self-control,
      • Higher scores on peer sociability and social school functioning.

      Models used regression analyses in which the significance of the direct effect between intervention and later conduct problems were compared with and without entering the mediator into the model. Prevention impact on teacher reports of externalizing problems was partially mediated by emotion regulation (ß reduced from p <.01 to p =.26). A similar impact was also found for students' self-report of conduct problems in which emotion regulation partly mediated (ß reduced from p <.01 to p =.29). No mediational effects were found for either understanding of emotional cues or social problem-solving.

      In a different study (Riggs et al., 2006), inhibitory control at immediate posttest significantly mediated the relation between experimental condition and both teacher-reported externalizing and internalizing behavior at 1-year follow-up.

      For PATHS, the effect sizes of significant results were small to moderate (Malti et al. 2011). Compared to a control group, PATHS significantly reduced aggressive behavior across time (teacher reports: d=0.42; p<.05; parent reports: d=0.26, p<.05). Children in the PATHS intervention group also evidenced a significantly greater reduction in ADHD symptoms compared to the control group (teacher reports: d=0.46, p<.05).

      In the Crean and Johnson (2013) independent replication, effect sizes were moderate for aggression (-.20), acting out behavior (-.24) and conduct problems (-.15). Schonfeld et al. (2014) reported small effect sizes for significant variables (OR 1.51-1.91). Averdikj et al. (2016) reported a small effect size (ES = -0.157) for long-term adolescent delinquency outcomes.

      It is unknown if this program can be generalized to all special-needs classrooms, or is better suited to a particular form of special needs child (e.g., behavioral problems vs. learning disabilities). Based on the overall success of multiple evaluations of the PATHS curriculum, however, it would appear that the program is generalizable to a wide array of hearing and deaf children in both regular and special needs classrooms. No analysis of effects by race was provided in any of the evaluations, but samples are generally ethnically/racially diverse. In two recent evaluations PATHS was found to be effective among children in elementary schools in Zurich, Switzerland (Malti et al., 2011; Averdijk et al., 2016).

      Seattle Study (Greenberg et al., 1995; Riggs et al., 2006):

      Sample attrition in the first year of implementation was high and reduced the sample size significantly, thus reducing the power to accurately detect differences. High levels of student mobility further limited comparisons between students receiving one or two years of the intervention. No analysis of differential attrition or mobility was conducted for the full sample (although this was done for the regular classroom subsample), which would further inform the interpretations of the analyses.

      Studies 3-6:

      Small sample sizes within treatment groups make it difficult to generalize the outcomes to larger, more diverse populations. No analyses of differential attrition were performed.

      Study 7

      Malti et al., 2011:

      • Low parental attendance of the Triple-P lessons.
      • It is unclear whether interviewers were blind to group membership.
      • Even though attrition was low and imputation of missing data was done, no test for differential attrition was conducted.
      • No significant program effects were observed for Triple-P and only a few significant findings emerged for PATHS.
      • Baseline equivalence for socio-demographic determinants was not established.

      A verdijk et al., 2016:

      • No reliability or validity was reported for 3 out of 7 outcome measures
      • No baseline outcome controls (due to large gap in ages from baseline to follow up)
      • Incomplete tests for baseline differences
      • No tests for differential attrition
      • Across 13 outcomes, only one non-negligible effect was found

      Study 8 (Seifert et al. 2004):

      • No pretest assessment, assessment of baseline equivalence, or information on attrition.
      • The comparisons across cohorts may be confounded by time.
      • Only outcomes based on interviewer ratings reached significance, not outcomes based on child self-reports.
      • Interviewers rating children likely were not blinded to the condition.
      • Reports of teacher dissatisfaction with the program suggest implementation problems.

      Study 9 (CPPRG 2010):

      • A concurrent intervention for high-risk students meant that the sample excluded the worst behaving students and that the other ongoing intervention might have influenced the program outcomes.
      • Baseline tests for equivalence compared schools but not children.
      • Teachers who delivered the intervention also did ratings of classroom children, and results proved stronger for teacher ratings than child ratings.
      • Attrition was high because the study was limited to children who had stayed in the same school for all three years, and differential attrition was apparent on several baseline measures.
      • Contrary to intent-to-treat procedures, only students who participated in the program for all three years were followed and used in the analysis.

      Study 12 (Schonfeld et al., 2014)

      • Not an intent-to-treat study – excluded those not participating in all four program years
      • No tests for baseline equivalence of outcomes
      • No controls for baseline scores
      • Tests for differential attrition incomplete
      • Sample from one large, urban school district

      Study 14 (Fishbein et al. 2016)

      • No information on student-level attrition
      • No information on reliability/validity provided for 8 of the 14 outcome measures
      • Teachers who delivered the program also completed the assessments (with effects in favor of the treatment on 13 of the 13 teacher-rated measures)
      • There was an effect on 15 out of 23 measures, but only 2 of these effects were assessed using independent measures (and it still wasn’t clear whether those collecting these data were blind to condition)
      • Small sample size (n = 4 schools, and schools were the unit of assignment)
      • Incorrect level of analysis with no adjustment for unit of randomization (schools)

      A preschool version of PATHS called Head Start REDI is treated as a separate program in Blueprints.

      The PATHS curriculum has been integrated into the FAST track program, which seeks to integrate the provision of universal, selective, and indicated services into a comprehensive model that involves the child, school, family and community. Please see the FAST Track write-up for detailed information on this comprehensive program. Riggs, Sakuma, and Pentz (2007) modified the PATHS program to address obesity (see writeup on PATHWAYS).

      Riggs, N. R., Sakuma, K. K., & Pentz, M.A. (2007). Preventing risk for obesity by promoting self-regulation and decision-making skills. Evaluation Review, 31, 287-310.

      • Blueprints: Model
      • Crime Solutions: Effective
      • OJJDP Model Programs: Effective
      • SAMHSA: 2.6-3.2

      Denine Goolsby
      Executive Director Humanware
      Cleveland Public Schools
      1111 Superior Avenue
      Cleveland, OH 44114
      PH: 216-838-0107

      Flavia Hernandez, Principal
      McCormick Elementary School
      Chicago Public Schools
      2712 S. Sawyer Avenue
      Chicago, IL 60623
      PH: 773-535-7252

      Carmen Navarro, Principal
      Mariano Azuela Elementary School
      Chicago Public Schools
      3707 W. Marquette Road
      Chicago, IL 60629
      PH: 773-535-7395

      Caroline Boxmeyer, Associate Professor University of Alabama
      Hale County/Sawyerville Head Start Center
      850th 5th Avenue East
      Box 870326
      Tuscaloosa, Alabama
      PH: 205-348-1325

      Averdijk, M., Zirk-Sadowski, J., Ribeaud, D., & Eisner, M. (2016). Long-term effects of two childhood psychosocial interventions on adolescent delinquency, substance use, and antisocial behavior: A cluster randomized controlled trial. Journal of Experimental Criminology, 12, 21-47.

      Conduct Problems Prevention Research Group. (2010). The effects of a multiyear universal social-emotional learning program: The role of student and school characteristics. Journal of Consulting and Continuing Psychology, 78(2), 156-168.

      Conduct Problems Prevention Research Group (Karen Bierman, John Coie, Kenneth Dodge, Mark Greenburg, John Lochman, Robert McMahon, and Ellen Pinderhughes). (1999). Initial Impact of the Fast Track prevention trial for conduct problems: I. The high-risk sample. Journal of Consulting and Clinical Psychology, 67, 631-647.

      Crean, H. F., & Johnson, D. B. (2013). Promoting Alternative THinking Strategies (PATHS) and elementary school aged children's aggression: Results from a cluster randomized trial. American Journal of Community Psychology, 52, 56-72.

      Curtis, C., & Norgate, R. (2007). An evaluation of the Promoting Alternative Thinking Strategies curriculum at key stage 1. Educational Psychology in Practice, 23, 33-44.

      Eisner, M. P., Malti, T., & Ribeaud, D. (2011). Large-scale criminological field experiments. In D. Gadd, S. Karstedt, & S.F. Messner (Eds), Sage handbook of criminological research methods(pp. 410-424). London: Sage.

      Greenberg, M. T., & Kusche, C. A. (1998). Preventive intervention for school-aged deaf children: The PATHS Curriculum. Journal of Deaf Studies and Deaf Education, 3, 49-63.

      Greenberg, M. T., Kusche, C. A., Cook, E. T., & Quamma, J. P. (1995). Promoting emotional competence in school-aged children: The effects of the PATHS curriculum. Development and Psychopathology, 7, 117-136.

      Kam, C., Greenberg, M. T., & Kusché, C. A. (2004). Sustained effects of the PATHS curriculum on the social and psychological adjustment of children in special education. Journal of Emotional and Behavioral Disorders, 12, 66-78.

      Kam, C., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using PATHS Curriculum. Prevention Science, 4, 55-63.

      Little, M., Berry, V., Morpeth, L., Blower, S., Axford, N., Taylor, R., ... Tobin, K. (2012). The impact of three evidence-based programmes delivered in public systems in Birmingham, UK. International Journal of Conflict and Violence, 6(2), 260-272.

      Malti, T., Ribeaud, D., & Eisner, M. P. (2011). The effectiveness of two universal preventive interventions in reducing children’s externalizing behavior: A cluster randomized controlled trial. Journal of Clinical Child & Adolescent Psychology, 40(5), 677-692.

      Malti, T., Ribeaud, D., & Eisner, M. (2012). Effectiveness of a universal school-based social competence program: The role of child characteristics and economic factors. International Journal of Conflict and Violence, 6, 249-259.

      Riggs, N. R., Greenberg, M. T., Kusché, C. A., & Pentz, M. A. (2006). The mediational role of neurocognition in the behavioral outcomes of a social-emotional prevention program in elementary school students: Effects of the PATHS curriculum. Prevention Science, 7, 91-102.

      Schonfeld, D. J., Adams, R. E., Fredstrom, B. K., Weissberg, R. P., Gilman, R., Voyce, C., T... Speese-Linehan, D. (2014). Cluster-randomized trial demonstrating impact on academic achievement of elementary social-emotional learning. School Psychology Quarterly, advance online publication.

      Seifert, R., Gouley, K., Miller, A.L., & Zabriski, A. (2004). Implementation of the PATHS curriculum in an urban elementary school. Early Education & Development, 15(4), 471-486.

      Channing-Bete Company
      One Community Place
      South Deerfield, MA 01373-0200
      Phone: (800) 477-4776
      Fax: (800) 499-6464
      custsvcs@channing-bete.com
      http://www.channing-bete.com/prevention-programs/paths/paths.html

      Also see:
      pathseducation.com
      Contact: Andy Spears
      andy@spearsstrategy.com

      Study 7

      Averdijk, M., Zirk-Sadowski, J., Ribeaud, D., & Eisner, M. (2016). Long-term effects of two childhood psychosocial interventions on adolescent delinquency, substance use, and antisocial behavior: A cluster randomized controlled trial. Journal of Experimental Criminology, 12, 21-47.

      Malti, T., Ribeaud, D., & Eisner, M. P. (2011). The effectiveness of two universal preventive interventions in reducing children’s externalizing behavior: A cluster randomized controlled trial. Journal of Clinical Child & Adolescent Psychology, 40(5), 677-692.

      Malti, T., Ribeaud, D., & Eisner, M. (2012). Effectiveness of a universal school-based social competence program: The role of child characteristics and economic factors. International Journal of Conflict and Violence, 6, 249-259.

      Seattle Study
      Greenberg, Kusché, Cook, & Quamma, 1995

      Evaluation Methodology

      Design: Participants were selected in different ways for the regular and special needs subsamples. Regular education children were drawn from the second and third grades of four schools in the Seattle school district. These schools were representative of the district profile, with the exception of having a lower percentage of Asian-American students. The prevention model was initially described to principals and teachers at each school. After faculty discussion, building-based decisions were made regarding participation. Schools were aware that once they decided to participate, they had a 50% chance of being randomized as a control school. All four schools that were approached decided to participate and two were randomly assigned as intervention schools.

      Classrooms for special need children were drawn from the Seattle, Highline, and Shoreline school districts. A presentation was made to interested special needs teachers. Each teacher was free to participate or decline, knowing that participation ensured only a 50% chance of receiving the intervention. Fourteen teachers elected to participate and were then randomized to either a treatment or control condition. Informed consent was received from approximately 70% of eligible students. Although the study assessed 426 students at the spring pre-test, 96 subjects were lost to follow-up due to school moves between spring of pre- and spring of post-test.

      Of the 286 participating children, 130 received the intervention (83 regular education, 47 special education) and 156 were in control classrooms (109 regular education, 47 special education). Children were initially tested in either the spring or fall prior to the intervention year; most children were tested in the spring in order not to delay the onset of the intervention during the first few weeks of class. The children were then interviewed during the following spring, approximately one month post-intervention.

      Sample: The final study sample included 286 children (167 males, 119 females) who attended school in the metropolitan Seattle area and were available for both pre- and post-testing. The children were attending first and second grade at pre-test, and second and third grade at the time of post-test. Ages ranged from 6 years, 5 months to 10 years, 6 months at pre-test, with a mean age of 8 years, 0 months. The mean age at post-test was 8 years, 10 months, with a range from 7 years, 0 months to 11 years, 2 months. The sample consisted of 165 Caucasians, 91 African Americans, 11 Asian Americans, 7 Filipino Americans, 7 Native Americans, and 1 Hispanic. Four children were of unknown ethnic origin. Sixty-seven percent of the children (n = 192) were in a regular education program, and 33% (n = 94) were in self-contained special education classrooms. Within the special education sample, children were classified in the following categories according to school records: learning disabilities (n = 44), mild mental retardation (n = 23), severe behavior disorders (n = 22), or multihandicaps (n = 5).

      Measures: Each child was individually assessed using the Kusche Affective Interview Revised (KAI-R). This interview was developed as an expansion of previous interviews to assess children's emotional understanding at both an experiential and at a metacognitive level to probe a wide range of affective states and situations. Five domains of emotional understanding were assessed: ability to discuss one's own emotional experiences, cues used to recognize emotions, issues regarding the simultaneity of emotions, display rules for emotions, and whether and how emotions can change.

      1. Students' feelings vocabularies were measured by summary counts of total positive and negative feelings stated at the start of the interview and by accurate definitions of five complex feelings (proud, guilty, jealous, nervous/anxious, and lonely). Definitions were coded on a trilevel scale analogous to that of the Wechsler Intelligence Scale for Children-Revised (WISC-R). Two questions were used to assess children's general knowledge of feelings. First, they were asked "Are feelings OK to have?" with follow up questions based on either an affirmative or negative response. Children's abilities to discuss personal emotional experiences were assessed by asking them to provide examples of times when they had felt ten specific emotions (happy, sad, mad, scared, love, proud, guilty, jealous, nervous/anxious, and lonely).

      2. Children's ability to identify three specific emotional states in themselves and other people were assessed. They were asked "How do you know when you are feeling ____?" (happy, mad, or jealous), with a follow-up question asking how they know when others are feeling happy, mad or jealous. Responses were scored based on children's use of facial cues, situational cues, and internal feeling states.

      3. For issues of simultaneity of emotions, three pairs of feelings were probed. Children were asked "Can someone feel _____ and _____ at the very same time?" (sad/mad, happy/sad, and love/anger). Children were asked to provide examples in support of their responses, which were scored based on the level at which the child was able to report simultaneous feelings directed toward the same target.

      4. Assessment of whether and how emotions are hidden included children's understanding about their own ability to hide feelings as well as whether or not other's can hide feelings. In addition, children were asked if feelings should sometimes be hidden. Children were first asked: "Can you hide your feelings?" Percentages of this response were used as a measure of children's understanding of hiding feelings. Children who responded affirmatively were additionally asked how they could hide them, and this was coded using a developmental stage level system based on responses. Parallel questions were asked regarding others hiding feelings from the child, and similar response categories were used. Finally, children were asked: "Do you think there are times when people should hide their feelings?" which was coded using a simple yes or no format.

      5. Children were also asked a series of questions to probe their understanding of whether and how emotions can change using yes/no questions. To assess the efficiency of problem-solving, the WISC-R subtests of Coding and Block Design were used.

      In addition, the 112-item CBCL-TRF checklist was used to measure behavioral and emotional problems commonly seen by teachers. Responses yield eight narrow-band and two broad-band scores: Internalizing and Externalizing. Separate norms are utilized for boys and girls aged 6-11. To assess individualized changes in behavior, the Teacher Goal-Oriented Rating Form (TGOR) was utilized.

      Analysis: A series of three-way repeated measures ANOVAs were conducted to assess the general effects of the intervention. The two between-subjects factors were Intervention Status (intervention vs. control) and Educational Placement (regular vs. special needs); the within subjects factor was Time. As the special needs population was a heterogeneous grouping that included different types of students, further exploratory analyses were conducted to examine potential differential effects of type of special education classification. Using the same analytic model as above, a series of three-way repeated measures ANOVAs were conducted with two between-subjects factors; intervention status (intervention or control) and educational placement (learning disability, behavior disorder, mild mental retardation, and multiple disabilities), the within subjects factor was time.

      Outcomes

      Post-test:

      Feelings vocabulary
      : Children who received the PATHS curriculum significantly increased the number of feeling words they could generate between pre- and post-tests as compared to children in the control group. Both intervention and control children demonstrated a significant developmental change in terms of the number of positive and negative feeling words listed from pre- to post-test. Treatment group children in regular education classrooms showed a significant increase in knowledge of the five complex emotions relative to regular education children in the comparison group. No intervention effect was found for special needs students. A significant main effect of time indicated a developmental advancement across the year for all students.

      General questions about feelings: There were no significant effects of the intervention on children's level of reasoning regarding why (or why not) all feelings were OK.

      Discussion of own emotional experiences: Children in the intervention group significantly improved their ability to provide appropriate personal examples of the five basic feelings, but not of the five complex feelings (happy, sad, mad, scared, love, proud, guilty, jealous, nervous/anxious, and lonely) from pre- to post-test as compared to children in the control group. A significant effect for time indicated a general developmental increase in children's ability to give more appropriate examples of complex feelings at post-test.

      Cues to recognize emotions: No effects of the intervention were found on children's ability to describe cues used to recognize their own emotions. A main effect for time, however, indicated a general developmental advance in the level of children's reasoning about recognizing their own emotions between pre- and post-test. Children in the intervention group improved their level of reasoning with regard to knowing how others feel more than did children in the control group. In addition, a significant time effect indicated a general developmental advance in the level of children's reasoning about recognizing the feelings of others.

      Understanding simultaneous feelings: There were no significant intervention effects. However, a significant time effect indicated developmental increases over the one year period.

      Display rules for emotions: Children in the intervention group said "yes" significantly more often at post-test when asked if feelings could be hidden by themselves and by others when compared to children in the control group. No developmental change was noted for these responses. No intervention effects were found for children's level of reasoning about hiding their own feelings. Intervention children in special needs classrooms increased their level of understanding for other people's strategies for hiding feelings as compared to special needs children in the control group. A significant time effect indicated general developmental improvement in the level of reasoning about both themselves and others.

      Changing feelings: Children in the intervention group were significantly more likely to respond positively to questions about changes in feeling states than were children in the control group. However, significant Time x Intervention Status x Educational Placement effects indicated that much of the Time x Intervention Status effect was the result of a large change in the special needs intervention group. Intervention children also showed a higher level of reasoning in their examples of how feelings can change when a picture cue was not provided compared to children in the control group. Special needs children in the intervention group improved significantly more than intervention children in regular education classrooms, as well as more than children in the control group.

      Differential effects within special needs: There were fewer differential effects of intervention than would be expected by chance, indicating that improvements in special needs children were shown equally across the three identified groups.

      Differential effects related to level of teacher-rated psychopathology: In order to examine the question of differential effects related to level of child behavior problems, two sets of analyses were conducted; one on the level of externalizing problems and the other on the level of internalizing problems. Among intervention students, those with low and moderate scores showed significant improvement for externalizing problems. Intervention children with high TRF scores showed the greatest relative improvement in the number of appropriate examples given for basic emotions, but comparison children with high TRG scores showed the greatest relative decline during the intervention period. Similarly, for both questions concerning efficacy regarding changing feelings, intervention children with moderate and high scores showed the greatest relative improvement, while control children with high TRF scores showed significant declines.

      Intervention children with the lowest scores for internalizing problems demonstrated the greatest improvement on providing appropriate examples of advanced emotions, and comparison children with moderate and high TRF scores declined during the intervention period. These findings were repeated for both questions concerning efficacy regarding changing feelings. Similarly, intervention children with moderate or high TRF scores showed significant change in developmental level of understanding regarding how feelings change, and children in the control group with high internalizing scores showed declines during the same period. Regular education boys in the intervention group scored significantly higher on social competence scales as assessed by teacher, parent, and child ratings.

      Mediatiational Role of Neurocognition in the Behavioral Outcomes
      Riggs, Greenberg, Kusche, and Pentz, 2006

      This study reported using the same sample as Greenberg et al. (1995), but without the special needs sample. However, since the listed sample sizes differ across the two studies, this one is listed separately.

      Design . Four schools were randomly assigned to treatment or comparison conditions. The total recruited sample was 329 students enrolled in the second and third grade at the time of pretesting, and the final sample was 318 students. The sample sizes equaled 153 for the intervention group and 165 for the control group. A total of 68 classroom sessions were devoted to PATHS teaching.

      Data were collected at pretest, nine months later (posttest), and one year follow-up.

      Sample Characteristics . About 50% were girls, 55% were Caucasian, 33% were African American, and 22% were from other racial backgrounds.

      Measures . Student surveys: 1) IQ was estimated using a two subtest short form of the WISC-R, which includes Vocabulary and Block Design; 2) Inhibitory Control was measured using the Stroop Test, which activates the anterior cingulated in adults, a neural region that interacts with both the limbic and prefrontal function; and 3) the Verbal Fluency subtest of the McCarthy Scales of Children’s Abilities, which requires children to name as many items as they can in four common categories.

      Teacher Surveys: Child behavior problems were assessed using the Teacher Report Form of the Child Behavior Checklist.

      Analysis . First, hierarchical regression models were estimated to determine the effects of the PATHS Curriculum on teacher-reported behavior outcomes. Covariates for these models included pretest behavior scores, age, and IQ. Next, hierarchical regression models estimated the effects of the PATHS Curriculum on children’s inhibitory control and verbal fluency 9 months later. Covariates included pretest neurocognitive scores, age, and IQ. Third, hierarchical linear models estimated the effects of both the intervention and the neurocognitive mediators on the behavioral outcomes, again including pretest covariates. The mediators in this third model should significantly influence the outcomes and attenuate the effect of the intervention found in the first model.

      Missing data were replaced with sample means.

      Outcomes

      Implementation Fidelity . Treatment teachers attended a three-day training and received weekly consultation and observation from project staff. Fidelity assessments were conducted.

      Baseline Equivalence and Attrition . Paired t-tests revealed group differences at pretest for Verbal Fluency and IQ, with the control group scoring higher. No analysis of differential attrition was reported, perhaps because missing data were replaced with sample means.

      Posttest and Long-Term . The results based on both the posttest and one-year follow-up focused on each step of the mediation analysis.

      (1) Regression analyses indicated that there was a significant prevention effect on both inhibitory control and verbal fluency.

      (2) Posttest inhibitory control was negatively related to teacher-reported externalizing and internalizing behavior at 1-year follow-up. Posttest verbal fluency was negatively related to teacher ratings of internalizing behavior at 1-year follow-up. That is, children who had greater inhibitory control at posttest demonstrated fewer externalizing and internalizing behavior problems and children who were more verbally fluent demonstrated fewer internalizing behavior problems at 1-year follow-up. After taking these neurocognitive variables into account, the intervention condition continued to have a significant effect on externalizing behavior and internalizing behavior.

      (3) Sobel tests for mediation demonstrated that inhibitory control at immediate posttest significantly mediated the relation between experimental condition and both teacher-reported externalizing and internalizing behavior at 1-year follow-up. These findings demonstrate that the direct effect of the PATHS program on inhibitory control significantly reduced the relation between PATHS and both outcomes. However, a Sobel test demonstrated that the mediating role of verbal fluency in the relation between the experimental condition and teacher-reported internalizing behavior only approached significance.

      Long-term Study of Special Needs Children
      Kam, Greenberg, & Kusché (2004)

      This study used the same sample of children in special education classrooms as Greenberg et al. (1995) in Study 1. However, this study reported a sample of 133 children, while Greenberg et al. (1995) reported a sample of 94 children. The difference appears to come from the analytic techniques used in each study. Greenberg et al. (1995) reported on the sample available for both pretest and posttest. This study used models that include cases with partial missing data and therefore has a larger sample size.

      Design: This study examined the effects of the PATHS Curriculum on diverse outcomes at post-test, 1-year, 2-year, and 3-year follow-up. The sample consisted of 133 special needs children (in grades 1-3 at time of pretest) who had been previously assigned by their schools to special education classes. Eighteen teachers of special education classes elected to participate and were randomly assigned to either the intervention or control group. About 70% of the parents of children in the classes consented to testing. Children were initially tested in either the spring or fall prior to the intervention year; they were assessed again each spring for the next three years.

      The rate of missing data varied by outcome, but large attrition generally occurred in the follow-up years. All participating children had baseline and posttest data, but from 6% to 48% were missing data on some measures for the follow-ups.

      Sample: The sample included 52% White, 40% African American, and 8% children from other ethnic minority populations. All children had been previously assigned by their schools to special education classes. Of the 133 children, most had learning disabilities (53), but the sample also included children with mild mental retardation (23), physical disabilities (21), emotional and behavioral disorders (31), and multiple handicaps (5).

      Measures: Measures are described in Study 1:

      • Feelings Vocabulary
      • Social Problem-Solving Skills
      • Child Self-Report of Depression
      • Teacher Ratings of Problem Behavior (Externalizing and Internalizing)
      • Teacher Ratings of Social Competence

      Analyses: Students' outcome trajectories were modeled across the early elementary grades using individual growth-curve analyses and multilevel models. Trajectories for students in the intervention group were compared to those for students in the control group. Sustained intervention effects were indicated as positive changes in trajectories above and beyond those observed in the control group.

      In estimating outcome trajectories in the multilevel models, the analysis used all available data for subjects, even if not complete for all assessments. The analysis thus meets the intent-to-treat criterion.

      Outcomes

      Baseline Equivalence and Differential Attrition. At baseline, the intervention and control groups were equivalent on all outcomes except internalizing behaviors (where the intervention group had higher scores). The study made no mention of tests for differences across the groups on sociodemographic characteristics or types of disability.

      The study did not report on differential attrition. It noted that the multilevel models used the maximum amount of information that, under the assumption of data missing at random, gives unbiased and efficient estimates. However, the study did not compare the rate of attrition across groups or the baseline values of those having missing data with those having complete data,

      Posttest and Long-Term. Separate analyses were not done for posttest and follow-ups. Rather, the results examined linear changes in outcomes over the full period from baseline to 3-year follow-up.

      Trajectories of teacher-rated behaviors: Teachers' ratings of students externalizing and internalizing behaviors can best be described as changing linearly from Time 1 (baseline) to Time 4 (3-year follow-up). A significant difference was found between the intervention and control groups in the estimated mean rate of growth in both types of behaviors, with teacher ratings of behaviors decreasing over time in the intervention group whereas those in the control group increased over time in both cases. No significant group differences were found for the trajectories of teacher-reported social competencies (frustration tolerance, assertive skills, task orientation, and peer sociability).

      Trajectories of self-reported depression: A linear growth curve model fit the Child Depression Inventory (CDI) data relatively well. Depression scores reported by children in the intervention group declined at a significantly greater rate than did the scores reported by children in the control group.

      Affective vocabulary: Linear trend models fit well with both positive and negative feelings vocabulary data. A significant difference between the two groups was found in the size of the negative feelings vocabulary at Time 4. There was no significant difference in the rates of change in the size of negative and positive feelings vocabulary.

      Social problem-solving skills: No significant intervention group difference was found in the growth curve analyses of efficacy in social problem solving among children in the special needs classrooms. Children in the intervention displayed a marginally significant reduction in the percentage of aggressive solutions they generated and a significant increase in the percentage of solutions that were nonconfrontational and indicated self-control.

      Outcomes - Brief Bullets

      • Over the one-year of intervention and the three-year follow-up, the trajectories for internalizing and externalizing behavior and depression continued to decline for intervention students, whereas control students' trajectories increased (internalizing/externalizing) or declined at a lower rate (depression).
      • No significant intervention group difference was found in the growth curve analyses of efficacy in social problem solving, but there was a significant increase in the percentage of solutions that were nonconfrontational and indicated self-control.

      Generalizability: Although small, the sample was ethnically/racially diverse and appears to be equally effective among minorities and Caucasians. It is unknown whether the program will have the same positive outcomes with children with varying degrees and severity of behavioral problems and learning disabilities, or with children in rural or suburban settings.

      Limitations: Small sample sizes within treatment groups make it difficult to generalize the outcomes to larger, more diverse populations. In addition, despite substantial attrition, no analysis of differential attrition was conducted at any of the follow-ups.

      Metropolitan Seattle Study of Deaf Children
      Greenberg and Kusche, 1998

      Design: The participants in this project consisted of 79 severely and profoundly hearing-impaired children who were enrolled in self-contained classrooms for deaf children (grades 1-6) in 6 local elementary schools in the Seattle area. The study design was quasi-experimental. Schools were randomly assigned to intervention vs. waitlist control group status. After the first year, the intervention was replicated on the wait-list control children. Teachers were trained in the intervention model and implemented PATHS lessons during most of one school year. The participants represented approximately 85% of all of the deaf children who were served in the area and who also met the following criteria: (1) basic education occurred using both sign language and speech (Total Communication), (2) unaided hearing loss was >60 decibels in the better ear averaged across the speech range, (3) deafness was diagnosed prior to 36 months of age, (4) non-verbal intelligence was greater than 75, and (5) no known significant additional handicaps were present. The intervention and comparison groups did not differ significantly on relevant pretest variables.

      Sample: The children ranged in age from 67 to 146 months of age. The sample was primarily White (84%), and the average child had a profound unaided hearing loss.

      Measures: Measures included an interview of social-problem solving, tests of non-verbal cognitive abilities, achievement testing, and teacher and parent ratings of behavioral difficulties and competencies.

      Analysis: Mediational analyses were conducted to test the theoretical model that changes in understanding of emotions, ability to take others' perspectives, and social problem-solving skills were related to changes in behavioral outcome. For a detailed overview of the analyses used in all of the PATHS evaluations, please see the description above for the pilot study.

      Outcomes

      Post-test: Results indicated that the intervention led to significant improvement in students' social problem-solving skills, emotional recognition skills, and teacher and parent-rated social competence. Teacher ratings of behavior indicated that there were significant improvements in social competence and in frustration tolerance. Results also indicated significant improvement in reading achievement and non-verbal planning skills in the intervention sample. There was no effect in this normative sample on teacher or parent-rated psychopathology.

      Mediational analyses: Results indicated that (a) improvement in emotional understanding was related to lower parent report of lowered externalizing problems at home; (b) improvement in role-taking skills was related to higher teacher ratings of emotional adjustment, and reductions in behavior problems at school and at home; and (c) improvement in problem-solving was related to higher teacher ratings of emotional adjustment and social competence and decreases in behavior problems at home and school.

      Long-term: One- and two- year post-test results indicated maintenance of effects. Results with the wait-list control group indicated replication of effects in the second sample.

      Outcomes - Brief Bullets

      • Significant improvement in social problem-solving skills, emotional recognition skills, reading achievement and non-verbal planning skills, and teacher and parent-rated social competence.
      • Teacher ratings of behavior indicated significant improvements in social competence and in frustration tolerance.
      • Mediational analyses indicated that (a) improvement in emotional understanding was related to lower parent report of lowered externalizing problems at home; (b) improvement in role-taking skills was related to higher teacher ratings of emotional adjustment, and reductions in behavior problems at school and at home; and (c) improvement in problem-solving was related to higher teacher ratings of emotional adjustment and social competence and decreases in behavior problems at home and school.
      • One- and two- year post-test results indicated maintenance of effects. Results with the wait-list control group indicated replication of effects in the second sample.

      Generalizability: The small sample size makes generalizing any outcomes problematic at best. Based on the success of previous evaluations of the PATHS curriculum, however, it would appear that the program is generalizable to a wide array of children in both regular and special needs classrooms.

      Limitations: Small sample sizes within treatment groups make it difficult to generalize the outcomes to larger, more diverse populations (the sample was 84% Caucasian). No long-term follow-up data is available and thus no assessments can be made regarding the sustainability of positive outcomes.

      Effectiveness trial in a high-risk community
      Kam, Greenberg, and Walls, 2003

      Design: This evaluation used a quasi-experimental matched-group design. Random assignment to intervention groups was not carried out because the local funding source required that schools in neighborhoods with the most high-risk profile receive the intervention. The sample in the overall intervention consisted of 350 first graders in six elementary schools in Harrisburg, Pennsylvania. Three of the schools received the intervention and three other schools served as comparison schools. A total of 13 classrooms with 164 students received the intervention. All of the participating schools served neighborhoods with very high rates of poverty and crime. Due to high family mobility and school reorganizations, student mobility averaged approximately 35-40% during the 1999 school year. A psychological consultation agency was contracted to coordinate the program implementation in targeted schools in the Harrisburg school district. Teachers received training from the program developers in two one-day workshops held approximately six weeks apart. Additional support for implementing the PATHS Curriculum was provided by an on-site PATHS Coordinator. The support included a weekly visit by the coordinator to PATHS classrooms and continuous consultation with the teachers, and logistics/materials support. The PATHS coordinator met with the school building principals on a monthly basis; principals were also strongly encouraged to attend workshop training. The PATHS Coordinator received, as needed, ongoing consultation from the program developers. The first year intervention is more brief than a "standard" implementation of the PATHS Curriculum. Because of the timing of funding, the curriculum was taught for only four months, from January to April, as compared to a full school year.

      **While the PATHS Curriculum was the major component of the intervention (see above studies for a detailed description of the Curriculum), the Dauphin County project also had a second component of intervention provided by the Big Brothers and Big Sisters in the area. The latter provided mentoring to 14 students in the intervention schools who teachers identified to have special needs.

      Sample: The sample was 47.14% male and 79.42% African American. Approximately 85% of the children in the schools sampled came from low-income families (as indexed by participation in the free lunch program). More than 65% of the students in the participating schools performed below the 30th national percentile in reading and mathematics.

      Measures: Students behaviors at school were assessed by teachers at both the pre- and post-testing period using the 31-item Teacher Social Competence Rating Scale (TSCRS). Behaviors measured included: aggression, dysregulated behaviors, attentional control, and social-emotional competence. The quality of PATHS program implementation in classrooms was measured by observations made by the local PATHS Coordinator. Two aspects of classroom environment and implementation quality were rated: (1) How well are PATHS concepts and skills taught by the teacher? and (2) How well is the teacher generalizing PATHS skills across the school day? Principal support for PATHS implementation was measured by PATHS Coordinator and PATHS Supervisor ratings. Two measures of ratings were used: (1) quality of principal support for PATHS, and (2) quality of support for the PATHS technical assistance team (PATHS Coordinator and Supervisor).

      Analysis: Analysis of covariance was used to analyze each outcome separately. A baseline measure of the outcome was entered in to the regression as well as the two dummy variables representing principal support. The classroom implementation measure was included as a continuous variable, as well as interaction terms between principal support and implementation. Planned comparisons were made on the predicted changes in classrooms that had a high and low degree of implementation, but had different levels of principal support. Due to a high rate of intercorrelation, observational ratings of PATHS program implementation were averaged over a four-month period.

      Outcomes

      Post-test: There was no significant main effect for implementation quality in predicting any of the four outcomes. Significant main effects were found, however, for principal support. In addition, significant interaction effects were found between the effects of principal support and implementation in the changes in all four domains (aggression, behavior dysregulation, social-emotional competence, and on-task behaviors). These results indicate that the effects of implementation work differently in schools with different degrees of principal support. When both the quality of implementation and principal support were high, students demonstrated significantly greater reductions in aggression and behavioral dysregulation, and significant increases in emotional competence when compared to students in the school with the lowest principal support. Similarly significant, but weaker differences on the same student outcomes were also shown when the school with the lowest principal support was compared to the average of the two schools with higher principal support.

      Long-term: No long-term data was collected or analyzed in this evaluation.

      Outcomes - Brief Bullets

      • When both the quality of implementation and principal support were high, students demonstrated significantly greater reductions in aggression and behavioral dysregulation, and significant increases in emotional competence when compared to students in the school with the lowest principal support.

      Generalizability: This evaluation was performed among high-risk urban youth with a high percentage of African Americans in schools with varying levels of principal support for implementing the PATHS Curriculum. It is unknown to what degree these results can be generalized to populations outside of this specific demographic, but when viewed in terms of the larger body of PATHS evaluations, it appears as if the program itself is successful among a variety of diverse populations.

      Limitations: No mention is made as to how schools were targeted for participation in the study, and random assignment was not used due to stipulations made by the local funding source. The authors also do not specify how the PATHS Curriculum was altered in order to fit into the shortened four-month implementation period. No mention was made of the specific involvement of the Big Brothers and Big Sisters program with regard to special needs youth. It would have been informative to know approximately how many intervention and control students were also Big Brothers and Big Sisters children. Student mobility was very high (35-40%), yet no mention is made of attritional analyses.

      Evaluation of 5 to 7 year olds in the UK
      Curtis and Norgate, 2007


      Design: This quasi-experimental project (labeled by the investigators as a pilot project) implemented PATHS in five schools, with three control schools. Random assignment was not conducted, though groups were matched on age range and catchment area. At least two members of staff from each school attended two days of initial training provided by educational psychologists, and these staff then conducted the training in their own schools.

      Sample: PATHS was administered to the children in Key Stage 1 (the term for Year 1 and Year 2 in England and Wales, ages 5 to 7) in five treatment schools and three comparison schools. It is not noted how many students received the curriculum. Surveys were completed by 287 students (114 PATHS and 173 control), and a semi-structured interview was completed with 17 teachers in the PATHS schools.

      Measures: The Strengths and Difficulties Questionnaire (SDQ, Goodman, 1997) was administered to the 287 students, and a semi-structured interview was completed with 17 teachers. The SDQ is composed of five constructs: (1) Emotional symptoms; (2) Conduct problems; (3) Hyperactivity; (4) Peer problems; and (5) Consideration.

      Analyses: Student surveys: Independent t -tests were conducted to compare the pretest scores on the five constructs of the SDQ between the intervention and control group. Pretest scores were significantly different for the treatment and comparison groups. Mixed analyses of variance (ANOVAs) and paired t -tests were conducted to compare the mean scores before and after intervention for both groups on each of the five constructs. Analyses examining mediating variables were not conducted.

      Teacher interviews: Interviews were recorded and transcribed, and then analyzed using a content analysis approach (Weber, 1990). Major categories were identified and changed until they were able to accommodate all of the data. The material from these categories was then organized into subcategories.

      Outcomes

      Posttest (School Surveys): The ANOVA indicated that the change over time in mean scores was statistically significant, as was the interaction between the two conditions, in all of the dimensions within the SDQ (Emotional symptoms, Conduct problems, Hyperactivity, Peer problems, and consideration, all p values <.0001). The change in scores from pretest to posttest was significant for the intervention group but not for the comparison group.

      Teacher Interviews: Results indicated that in all schools using PATHS, a whole-school emphasis had been adopted, PATHS materials were displayed in hallways and classrooms, staff showed flexibility in the use of sessions, and all staff were knowledgeable about the program. Teachers particularly felt that the introduction of the "pupil of the day" had a positive impact on the students. Teachers also noted their own ideas to keep the curriculum fresh and interesting (e.g., role-play, social and self-reinforcement, story-telling, and modeling). All schools using PATHS worked to get parents involved so that the ideas and skills in PATHS were recognized and reinforced at home. Perceptions of how PATHS helped (despite that it was one of many programs being delivered across schools): Building a vocabulary of feelings, Developing the ability to describe own feelings, Recognizing emotions in others, Empathy, Developing self-control/managing emotions, Developing cooperation, and Dealing with problems. Finally, teachers felt that PATHS was a good fit for their schools. Criticisms included that teachers wanted more ideas on how to keep the curriculum "fresh and exciting," and practical ideas as the curriculum involved much sitting listening, and discussion, which was not appropriate for all children.

      Outcomes - Brief Bullets

      • Significant improvement for 5 to 7 year olds on all five subscales of the SDQ: Emotional symptoms, Conduct problems, Hyperactivity, Peer problems, and Consideration, for the treatment schools but not for the comparison schools.
      • Qualitative teacher interviews indicated perceptions of a good fit for the program at the school and positive changes in the areas targeted by PATHS

      Generalizability: This is the second study conducted in the UK, but the first with a comparison group (cf. Kelly, Longbottom, Potts, and Williamson, 2004). However, the relatively small sample size (or at least unknown percentage of participants surveyed) severely limits generalizability. The students in the study were only in Year 1 and Year 2 (aged 5-7), the youngest end of the age range of students for which the curriculum was designed.

      Limitations: The sample size was relatively small (n =287), but of greater concern is the lack of information about the total number of classrooms and students served. Thus, the percentage of students receiving the curriculum surveyed is unknown. Significant differences at pretest between the treatment and comparison schools are of concern. In particular, although there were no significant pre-posttest improvements for the comparison schools, the levels of behavioral and emotional problems were already at a lower level compared with the treatment schools.

      PATHS and Triple P
      Malti et al. (2011), Malti et al. (2012), Averdijk et al. (2016)

      This study used a combined prevention approach that focused on both the school and family context. The goal was to compare PATHS (Promoting Alternative Thinking Strategies) and Triple-P (Positive Parenting Program) in reducing children’s externalization behavior and increasing children’s social competence. This study constitutes an independent evaluation outside the controlled environment of developer-led trials. The strength of this study is the use of a high quality design (RCT) and a long-term, 2-year post-intervention follow-up assessment (Malti et al., 2011) and a 7- and 9-year follow-up assessment (Averdijk, 2016). Finally, the study was conducted in Switzerland, which adds to the geographical generalizability. Malti et al. (2012) examined moderator interactions.

      Evaluation Methodology

      Design:
      Recruitment:
      The data for this study were obtained from the Zurich Project on the Social Development of Children (Z-Proso), an ongoing prospective longitudinal study of a cohort of children who entered elementary school in the city of Zurich, Switzerland, in 2004. The final sample consisted of 1,675 first graders from 56 elementary schools.

      Assignment:
      Sampling was based on a cluster randomized approach involving all 90 public primary schools in Zurich. Schools were blocked by school size and socio-economic background of the catchment area, then a stratified sample of 56 schools (comprising 1,675 children) was drawn. All selected schools participated, and 14 “quadruplets” of similar size and socio-economic background were formed. Schools in each quadruplet were randomly allocated to four treatment conditions: PATHS (n = 442), Triple-P (n = 422), PATHS+Triple-P (n = 397), control (n = 414). The programs were delivered sequentially: Level 4 of Triple P was implemented between waves 1 and 2 (i.e., year 1 of primary school) whereas PATHS was implemented between waves 2 and 3 (i.e., year 2). Malti et al. (2011) used data at annual intervals between 2004/2005 and 2006/2007 (T1-T3); Wave 4 (T4) was conducted 2 years later in 2008/2009. It is unclear whether the data collection was conducted at the beginning or the end of the particular academic year. Averdijk (2016) used data from the baseline assessment collected in 2004/2005 (wave 1, when students were 7 or 8) and follow up data collected in 2011 (wave 5, when students were 13 years old) and 2013 (wave 6, when students were 15 years of age).

      The version of PATHS used in this study was the “Fast Track Project” version. This school-based 1-year program included 46 primary lessons addressing problem-solving skills, social relationships, self-regulation, rule understanding, emotion understanding, and positive self-esteem. The PATHS classes consumed about 67 min per week during the 1-year implementation phase with an average of 2.4 sessions per week. PATHS was implemented in year 2 (2005/2006) when students were in second grade. Trained teachers implemented the lessons, and five trained coaches were also available to visit each class four to six times during the implementation period to give teachers feedback. Implementation was monitored using teacher and child questionnaires developed by the program designers. Teachers were also observed by the coaches. Content, methods, and materials were culturally adapted to the Swiss school system.

      Triple-P has many levels and the authors employed the standard Triple-P (level-4) for this study. Triple-P was implemented as a group-based parent-training program of four weekly 2- to 3-hours sessions with video elements, group discussion, role-playing, and homework for parents. Triple-P was implemented in year 1 (2004/2005) when children were in first grade. Control schools operated as usual.

      Attrition:
      As reported in Malti et al. (2011), at T1 the response rates were 81% for the child interviews (n = 1,361), 74% for the parent interviews (n = 1,240), and 81% for the teacher assessments (n = 1,350). At T2 the respective response rates were 97%, 95%, 96%; at T3 96%, 95%, 94%, and at T4 83%, 86%, and 92%. The computer-assisted, face-to-face interviews with parents were conducted at the parent’s home. In the first three waves, computer-assisted child assessments were conducted at the school. In the fourth wave, classroom-based paper-and-pencil surveys were utilized. The child’s teacher completed questionnaires on the child’s social development and returned it by mail. The interviews were conducted by 44 trained interviewers. It is unclear whether interviewers were blind to group membership. The overall attrition rate reported in Averdijk (2016) for the control group was 27% in wave 5 and 30% in wave 6. For the PATHS group, overall attrition was 29% (wave 5) and 30% (wave 6). For the combined Triple P and PATHS group, overall attrition was 36% (wave 5) and 38% (wave 6).

      Sample:
      At study commencement for wave 1, the sample consisted of first graders (mean age 7 years) with 48% girls and 52% boys. About 91% of the students were in regular classes, whereas the other 9% were in special-needs classes. About 78% lived with their biological parents, 20% with their biological mother only, and 2% with their biological father only or with foster parents. As for the socioeconomic background of the primary caregiver, 25% had little or no secondary education, 30% had vocational training, 29% had attended vocational school or had a baccalaureate degree or advanced vocational diploma, and 16% had a university degree. Socio-economic status was assessed through the International Socio-Economic Index of occupational status (ISEI) with an average score of 44.56 for households in the sample. In only half of the households (55%) at least one parent was of Swiss nationality, demonstrating the cultural diversity of the sample.

      Measures:
      Primary outcome measures reported in Malti et al. (2011) include:

      • Externalizing behavior: The teachers, parents, and children evaluated the externalizing behavior of the children at all four measurement times (T1–T4) using Tremblay’s Social Behavior Questionnaire (SBQ). The children were shown drawings of specific behaviors of a child and asked whether (s)he sometimes does what is shown in the pictures (answer options yes/no). Three subscales of the SBQ were employed measuring aggressive behavior (alpha = .72-.93), impulsivity/attention deficit hyperactive disorder (ADHD; alpha = .62-.91), and non-aggressive conduct disorders (NACD; alpha = .69-.78 for teacher’s reports).
      • Social competence: Social competence of children was measured using the Prosocial Behavior subscale of the Social Behavior Questionnaire (7-10 items, alpha = .59-.93). Children were presented with hypothetical scenarios and their responses were rated as aggressive strategies and socially competent strategies (interrater agreement .80-.87).

      For the two-year follow-up study conducted by Malti et al. (2012), four moderator variables were also examined: three child characteristic variables (social problem-solving, moral emotions, child initial behavior) and economic factors. Children's social-cognitive skills were measured before the intervention by measuring their problem solution strategy responses to four hypothetical vignettes. Proportional mean scores for aggressive and socially competent problem-solving strategies were calculated. Moral emotions were measured using a revised version of the original happy-victimizer task. Measures of economic risk included socioeconomic status, occurrence of financial difficulties (via parent interview at baseline), and single parent household.

      Averdijk et al. 2016 included 13 measures, five of which reported reliability statistics for the sample. These measures, eight from the youth and five from the teachers, included:

      • Self-reported delinquency (15 items constructed into a “total variety scale” which the author reports has been termed the preferred criminal offending scale as they display high reliability and validity).
      • Teacher-reported deviance (7 items constructed into a total variety scale).
      • Self-reported police contacts related to an offense.
      • Self-reported substance use (the sum of 4 items) and teacher-reported substance use (the sum of 3 items on smoking, drinking, and illegal drugs).
      • Teacher- and self-reported aggressive behavior, as measured by the Social Behavior Questionnaire (SBQ). The reliabilities were α=0.84 (wave 5) and α=0.83 (wave 6) for the youths and α=0.93 (wave 5) and α=0.92 (wave 6) for the teachers.
      • Self-reported peer aggression derived from Olweus (1993). The reliabilities were α=0.78 (wave 5) and α=0.75 (wave 6).
      • Teacher- and self-reported prosocial behavior, as measured by the Social Behavior Questionnaire (SBQ). The reliabilities were α=0.82 (wave 5) and α=0.80 (wave 6) for the youths and α=0.93 (wave 5) and α=0.90 (wave 6) for the teachers.
      • Self-reported conflict resolution, an eight-item scale adapted from Wetzels et al. (2001). Four items were used to create a measure for aggressive conflict resolution strategies (α=0.72 at wave 5, α=0.67 at wave 6) and 4 items comprised the competent conflict resolution strategies scale (α=0.71 at wave 5, α=0.71 at wave 6).
      • Teacher-reported non-aggressive conduct disorder, as measured by the Social Behavior Questionnaire (SBQ). The reliabilities were α=0.83 (wave 5) and α=0.85 (wave 6).

      Analysis:
      The authors employed longitudinal multilevel models to account for the hierarchical data structure (time was nested within children and children were nested within schools). Treatment assignment, measured at the school-level, was coded as two dummy variables to compare the PATHS and Triple-P conditions separately with the control condition. This design allows specifying different timings of the interventions as well as the inclusion of an interaction term between PATHS and Triple-P conditions. The models implicitly account for baseline scores. Moderator effects conducted by Malti et al. (2012) were tested by three-way interactions between intervention, the respective moderating variable, and time point. Similarly, Averdijk et al. (2016) conducted multilevel modeling with youths at level 1 and schools at level 2. Effects were estimated with the inclusion of several baseline sociodemographic covariates. Because the teen outcomes had little meaning at the age 7 pretest, the models used teacher and child measures of externalizing behavior as proxies for baseline outcomes.

      Intention-to-treat: The study followed the intent-to-treat principle. For the Malti et al. 2011 analyses, multiple imputation was used to account for missing data for children and parents. However, because Little’s MCAR test was not significant for the teacher’s data, multiple imputation was not necessary. Averdijk et al. (2016) handled missing data with robust full-information maximum-likelihood (FIML). Two sets of analyses were performed. The first used the dataset with all available data points of the target sample (n = 1,580 of 1,675). The second used stricter inclusion criteria, including only participants who participated both at age 7 years and at either age 13 years or age 15 years (n = 1,275 of 1,675). Results below are reported for the first set of analyses.

      Outcomes

      Implementation fidelity:
      PATHS
      : The teachers who implemented PATHS received a 2-day training course. The five PATHS coaches visited each class four to six times during the implementation period, after which they discussed the lesson with the teacher. A refresher seminar was held midterm, and regular PATHS newsletters helped to create a sense of cohesion among the participating teachers. The checklists completed by the coaches indicated high implementation quality and quantity. On average, 27 of the 30 obligatory lessons, 30 of the recommended vignettes, and 25 small-group activities were completed in the classes. The quality of classroom leadership, child motivation, and teaching of PATHS concepts received marks of 88%, 82%, 74%, respectively.

      Triple-P: Experienced Triple-P providers with a background in psychology, education, or guidance counseling who were licensed by Triple-P Switzerland administered the program. To assure uniform implementation quality, the providers were invited to two meetings to discuss key situations and receive coaching. Approximately 93% of the Triple-P course material was presented. However, parent participation in the Triple-P intervention was low. Of the 819 families randomized to Triple-P, only 257 (31%) had at least one parent who enrolled in the program. Of the 257 parents who did enroll, 220 (27%) attended at least one session and 153 (19%) completed all four course units.

      Participant overall satisfaction with the program was 4.33 (SD = .89) and provider competency was rated at 4.65 (SD = .73) on a 5-point scale, though it is not clear which program these data refer to (Triple P, PATHS or the combined treatment).

      Baseline Equivalence:
      Analyses reported by Malti et al. (2011) revealed no statistically significant baseline differences on any of the teacher, parent, or child outcome measures across treatment conditions. However, results for a baseline comparison of socio-demographic factors were not presented. Averdijk et al. (2016) noted only that baseline measures of externalizing did not differ significantly across conditions

      Differential attrition:
      No test for differential attrition was conducted by Malti et al. (2011). Across all four treatment conditions, there was little study attrition. Students completed an average of between 3.64 and 3.76 waves. There were no significant differences between groups due to attrition. In addition, the authors imputed missing data, which helps to adjust for the biasing effect of differential attrition. Attrition rates reported in Averdijk et al. (2016) at wave 5 and wave 6 were similar across conditions, but no other tests were presented.

      Posttest and Long-Term:

      As reported by Malti et al. (2011), the employed growth curve models do not allow for the separation between results for posttest and follow-up. As such, results were reported for time x group interactions.

      Children’s externalizing behavior: Overall, only a few program effects were observed for children’s externalizing behavior. For PATHS, 2 of 3 tests were significant based on teachers report, 1 of 2 was significant based on parent reports and none of 2 were significant based on child reports. Compared to a control group, PATHS significantly reduced aggressive behavior across time. This result was observed for both teacher (d=0.42; p<.05) and parent reports (d=0.26, p<.05) but not for child reported behavior. Those children in the PATHS intervention group also evidenced a significantly greater reduction in ADHD symptoms compared to the control group, but only based on teacher reports (d=0.46, p<.05). Finally, no significant change was observed for nonaggressive externalizing behavior for PATHS compared to the control group.

      For Triple-P and for the combined intervention (PATHS+Triple-P), no significant program effects were observed on any of the 7 outcome measures (according to reports by teachers, parents, or children).

      Children’s social competence: The study investigated program effects on prosocial behavior and problem solving strategies of children. However, no significant time x group interactions were found for the separate or combined intervention groups on any of the outcomes.

      Moderator analysis:
      The study found evidence for the moderating role of baseline behavior on program effectiveness. Three of the four significant effects (three-way interactions) suggested that children with high levels of baseline problem behavior benefitted more from either or both interventions than children with low levels of baseline problem behavior.

      Results reported by Malti et al. (2011) were for the PATHS only versus the control conditions, as the program effect of PATHS on outcomes variables was of primary interest to the researchers at follow-up. On the measure of teacher-reported aggression, there were significant differences between the treatment condition students compared to the control group students, where treatment students had a greater decrease in aggressive problem behaviors than students who did not receive treatment (ES = .12). These treatment effects were moderated by level of moral emotions at baseline, where children who exhibited higher levels of moral emotions and received the intervention showed larger reductions in aggression at follow-up than children who initially had low levels of moral emotions. Other moderator variables that predicted higher aggression at follow-up included baseline aggression, financial problems, single parent household, and non-Swiss nationality. Finally, SES and female gender predicted lower aggression at follow-up. On the measure of ADHD, there was a significant teacher-reported decrease in ADHD-related problems among children who received PATHS, compared to the children in the control condition (ES = .46). However, these treatment effects were moderated by the level of moral emotions at baseline and by initial level of competent problem-solving strategies, where intervention students who exhibited higher levels of moral emotions and competent problem-solving strategies showed larger reductions in teacher-reported ADHD. Additionally, aggressive problem-solving strategies, baseline ADHD, family financial problems and single-parent household predicted higher ADHD at follow-up. There were no significant program effects on prosocial behavior. There were also no significant interaction terms representing potential moderation by child characteristics or economic factors found.

      In the study conducted by Averdijk et al. (2016), only 1 of 13 tests at age 13 (seven years after program commencement) emerged as significant. Results showed a reduction in adolescent delinquency (i.e., fewer self-reported police contacts) for the PATHS group. There were no significant effects after nine years. No effects were detected for students who received the combined treatment of PATHS+Triple P.

      Providence Quasi-Experimental Study

      Seifert et al., 2004

      This pilot study done by a team independent of the developers of PATHS lacks a pretest assessment.

      Evaluation Methodology

      Design . This QED examined one urban elementary school, a magnet school in inner-city Providence, Rhode Island. The design compared two cohorts of children – one younger cohort receiving the 1-year intervention in 2001 during first grade, the other older cohort not receiving the intervention. The intervention group was thus tested in 2002 in second grade, while the control group was tested in 2001 in second grade. Rather than randomization, the study assumed that the intervention second graders in 2002 were equivalent to control second graders in 2001 except for experiencing the intervention.

      All available children from the three first-grade and three second-grade classrooms participated. The first cohort undergoing the 1-year intervention had 62 students. The second cohort or control group began with 75 students. However, a group of 13 students entered the school in 2002 and did not experience the intervention during the previous year. Although part of the intervention cohort and tested in second grade (2002), their results were combined with the control group to increase the control group n to 88. The list below contrasts the three cohorts:

      • Cohort 1 (Treatment n=62): 2001 (first grade) PATHS; 2002 (second grade) assessment
      • Cohort 2 (Control n=75): 2001 (second grade) assessment; 2002 (third grade) not participating
      • Cohort 3 (Control n=13): 2001 (not attending); 2002 (second grade) assessment

      No pretest assessments were conducted, and no information on attrition is available.

      Sample Characteristics . The students were 68% Hispanic and 14% black. About 25% were interviewed in Spanish. Across the full school, 94% qualified for subsidized lunch programs; 31% received bilingual programs; and less than 40% met state standards for reading, mathematics, and writing.

      Measures . Ten measures came from the students and interviewers.

      Sociometric Status. Each child was asked to nominate classmates on 17 positive and negative descriptors. Using the first seven nominations of each student, the measures summed the number of times a student was nominated. Principal components analysis of 15 of the items produced scales for positive peer nominations such as “want to sit next to” (alpha = .82), and negative peer nominations such as “starts fights” (alpha = .85).

      Emotion Understanding. The first measure taps spontaneous emotion naming skills or the number of emotions identified when asked to name all the different feelings they could think of. The second measure taps accuracy of emotion recognition or the score on matching pictures to emotions.

      Social Status Self-Reports. The first measure taps perceived meanness of treatment by other children (e.g., kids say mean things to me), and the second taps perceived rejection (e.g., kids blame me when things go wrong). The alphas equaled .75 for perceived meanness and .76 for perceived rejection. A third measure taps negative feelings toward school (“I feel alone at school”) and positive feelings toward school (“school is fun for me”). The eight items in the scale have an alpha of .63.

      Child Depression Symptoms. The Childhood Depression Inventory measures the frequency of different depression symptoms and has been used reliably with children as young as first grade (alpha = .78 for this sample).

      Global Competence Scale. After the 30-minute child interview, interviewers used the Psychological Impairment Rating Scale to assign a global rating of social competence (alpha = .93). Given the different timing of assessment for the intervention and control cohorts, the interviewers likely knew the assignment of the children they rated.

      Social-Emotional Competence Composite. The composite combined standardized scores on the nine previously listed measures (alpha = .64).

      Analysis. The analysis performed t-tests (without controls) on the posttest scores of the two groups.

      Outcomes

      Implementation Fidelity . All teachers participated in two training sessions. At the beginning and end of the school year, interviews with teachers revealed only modest enthusiasm for the program. Complaints about dissatisfaction with the PATHS materials, lack of support, and time and effort needed to implement the program suggest poor fidelity.

      Baseline Equivalence and Differential Attrition . Lacking pretest assessment, the study could not examine baseline equivalence. No information was provided on whether all children who started in the intervention and control groups completed the posttest.

      Posttest . Of the ten outcome measures, two showed significant differences between the intervention and control cohorts: the global social competence rating done by the interviewers and the social-emotional competence composite. The results did not differ between students interviewed in Spanish and students interviewed in English.

      Long-Term . None

      Limitations

      • No pretest assessment, assessment of baseline equivalence, or information on attrition.
      • The comparisons across cohorts may be confounded by time.
      • Only outcomes based on interviewer ratings reached significance, not outcomes based on child self-reports.
      • Interviewers rating children likely were not blinded to the condition.
      • Reports of teacher dissatisfaction with the program suggest implementation problems.

      PATHS and Fast Track
      CPPRG, 2010

      This study evaluated PATHS as part of the Fast Track program (and is included as well in the Fast Track write-up). It hypothesized that the universal intervention program would show greater impact on children in schools with lower levels of disadvantage. It also hypothesized that the universal program would do more to reduce aggressive behaviors among students with initial elevation in aggressive behavior.

      Evaluation Methodology

      Design . The cluster randomized design studied schools and children over three years.

      Schools. Participating schools came from Nashville, Seattle, and rural central Pennsylvania. Within each site, investigators invited about 12 elementary schools in high-risk areas (i.e., with high delinquency and juvenile arrests) to participate. Schools had to reach consensus among principals and teachers to participate. For those schools agreeing, three cohorts of students participated in the program, each beginning in first grade and participating for three years. The exact number of schools was not listed, but in each grade there were approximately 190 intervention classrooms and 180 comparison classrooms across the three cohorts. Note that schools in Durham, North Carolina, began the project but dropped out after the city and county schools merged and reassigned many children.

      Students. Participating students in the classrooms needed to remain in the same school building from the beginning of grade 1 to the end of grade 3 and needed to have supplied information on the Social Health Profile and sociometric outcomes. These criteria produced a sample of 2,937 children across the three cities. However, this sample appears truncated. The study noted (p. 159) that “Children who were selected in kindergarten for additional intervention from the Fast Track project (and their high-risk control counterparts) were not included.” The sample thus excluded (p. 166) “the worst behaving children.”

      Randomization. After being grouped on size, achievement levels, poverty, and ethnic/racial diversity, schools were randomly assigned to intervention and control groups. For the intervention group, this version of the program contained 57 lessons in grade 1, 46 in grade 2, and 48 in grade 3. The lessons were adapted to the needs of regular students in high-risk schools and lasted from September to May of each school year. The related version of the program for high-risk children, although not examined in this study, occurred simultaneously with the universal intervention.

      Attrition. Limiting students to those who stayed in the same school over three years produced high attrition. In Nashville, only 30.9% of the original 1,560 children remained in the same school over three years. In Seattle, only 41.6% of the original sample of 1,825 children remained in the same school. In rural Pennsylvania, 75.0% of the 1,696 children remained in the same school. Further, the study may have violated the intent-to-treat principle by examining only students receiving the full intervention and failing to follow any student who did not remain in the same school over three years.

      Assessment. Assessments occurred in the fall of the first year, and the spring of the first, second, and third year. The last assessment served as a posttest for the three-year program.

      Sample Characteristics . The school characteristics differed across sites. The mean percentage of children receiving free or reduced lunch was 57% but ranged from 39% in rural Pennsylvania to 78% in Nashville. The mean percentage of ethnic minority students (primarily African American) was 36% but ranged from 1% in rural Pennsylvania to 55% in Nashville. The mean reading percentile was 45th but ranged from 32nd in Nashville to 57th in rural Pennsylvania. The study did not report on the characteristics of the sampled students.

      Measures . Outcomes came from teacher ratings and peer sociometric nominations.

      Teacher Ratings. Teachers were interviewed regarding the behavior of each individual child in their class at the four assessment points (pretest, after year 1, after year 2, and posttest). The interviews used two instruments, the Teacher Observation of Classroom Adaptation – Revised (TOCA-R) and the Social Health Profile (SHR), and produced three measures:

      • authority acceptance (alpha = .93) on oppositional and conduct problems from the TOCA-R,
      • cognitive concentration (alpha = .97) on attention and work completion from the TOCA-R, and
      • social competence (alpha = .87) from the SHR.

      For all three measures, high scores indicated more problems.

      Peer Nominations. Interviews with children asked them to nominate classroom peers who fit descriptions of aggressive, hyperactive-disruptive, and prosocial behaviors. Scores for each child came from the average ratings given by classmates.

      Analysis. The investigators recoded some variables into ordered categories to reduce skewness (e.g., social competence) and truncated others to reduce the influence of large values (e.g., peer nominations). The analyses then estimated hierarchical models using three levels: time, child, and school. Time was centered so that the intervention main effect showed group differences at the end of the intervention. Time-by-intervention effects showed how the trend varied across groups.

      The models controlled for baseline values of the teacher-rated measures but not for the peer nomination measures, which were not gathered at the start of the study.

      Outcomes

      Implementation Fidelity . More than 90% of the teachers attended a 2-day training workshop. Educational consultants spent an average of 1 to 1.5 hours in each classroom observing, demonstrating, and providing feedback. They also met individually with teachers.

      On average, teachers completed 48.2 lessons in the first grade (85%), 39.6 in the second grade (86%), and 38.4 in the third grade (80%). Fidelity ratings from educational consultants ranged from 3.0 to 3.2 on a scale from 1 (low skilled performance) to 4 (highly skilled performance).

      Baseline Equivalence and Differential Attrition . A series of analyses of variance indicated no significant differences between intervention and control schools on the percent of children who received free and reduced lunch, the percentage of ethnic minority children, or academic achievement scores. Tests for baseline equivalence did not compare the baseline characteristics of students in the intervention and control groups.

      To assess differential attrition, analyses compared baseline scores for the sample students who remained in their school for all three years with other students who left the school between grades 1 and 3. The groups showed no significant differences on gender or pretest authority acceptance at any of the sites, but differed on other characteristics for at least one site. Dropouts were more likely to be African American, have lower pretest scores on social competence, and have lower pretest scores on cognitive concentration.

      Since attrition was lower in rural Pennsylvania (25%) than in the other two sites (58% and 69%), the study replicated the models separately for the rural Pennsylvania sample. The results were similar, perhaps even stronger for the rural Pennsylvania sample. However, it’s hard to know if the stronger results indicate the lack of bias from attrition or general differences in the Pennsylvania sample.

      Posttest . For all three teacher-rated outcome measures, the intervention group had better scores at the posttest and improved more over the three years than the control group. The intervention main effects were positive and significant for authority acceptance (effect size = .24), cognitive concentration (effect size = .12), and social competence (effect size = .34). The time-by-intervention effects were statistically significant as well, indicating that intervention children experienced less of an increase over time in problems.

      In addition, tests for moderation revealed weaker intervention benefits in low-income schools for authority acceptance and social competence. They also indicated that, for the outcome of authority acceptance, the intervention had stronger benefits for children with higher baseline problems.

      For the peer nominations, the intervention failed overall to affect outcomes of aggressive, prosocial, and hyperactive ratings, but it reduced aggressive and hyperactive outcomes for boys.

      Limitations

      • A concurrent intervention for high-risk students meant that the sample excluded the worst behaving students and that the other ongoing intervention might have influenced the program outcomes.
      • Baseline tests for equivalence compared schools but not children.
      • Teachers who delivered the intervention also did ratings of classroom children, and results proved stronger for teacher ratings than child ratings.
      • Attrition was high because the study was limited to children who had stayed in the same school for all three years, and differential attrition was apparent on several baseline measures.
      • Contrary to intent-to-treat procedures, only students who participated in the program for all three years were followed and used in the analysis.

      Crean & Johnson, 2013

      This evaluation was an independent randomized replication of PATHS that looked specifically at aggression outcomes from third to fifth grade. It was a cluster randomized trial with 14 elementary schools.

      Evaluation Methodology

      Design: This evaluation was part of a larger study called the Social and Character Development (SACD) Research Program, which tracked the development of third grade students over three years in a total of 96 schools in three states. Participants for the current study were two cohorts of third grade students from 14 schools across three districts. Within districts, the schools were randomly assigned to either the treatment or a control condition. Prior to randomization, a pairwise matching procedure, stratified by cohort and district, was conducted to ensure the treatment and control schools were demographically similar. The control condition continued with their traditional educational practice.

      Intervention teachers received two days of training, as well as consultation and observation from project staff every other week. Implementation of PATHS occurred between mid-September and early June each year. Education consultants (ECs) provided assistance to teachers to enhance the quality of implementation through modeling, coaching, and provision of ongoing feedback on program delivery, as well as classroom and behavior management. They spent an average of 1-1.5 hours per week in each classroom.

      Sample Characteristics: A total of 779 students participated in the evaluation. Parental consent to participate in the research study was 58%. The three districts were comprised of schools from a northeastern urban school district (n=8), a northeastern suburban school district (n=4), and a Midwestern suburban school district (n=2). Cohort 1 (10 schools; n=588) students were located in all three participating districts, while Cohort 2 (4 schools; n=191) students were recruited in the second year of the project from the northeastern urban school district. The majority (57%) of participating students came from the urban school district. The study had more female (57%) than male (43%) participants, and a slight majority (51%) of students were White/Caucasian, followed by African-American (38%), and Other (10%). Additionally, 17% of students identified themselves as Hispanic. About one third (33%) of students came from single parent households. The distribution of family income indicated a large proportion of families were poor (39% of families reported earning less than $20,000/year and 39% of families reported earning between $20,000 and $39,000 per year). By contrast, 21% of families reported earning over $70,000 per year. Most households had a family member who had graduated high school (19%), had some college (38%) or had obtained a college degree (33%).

      Measures: Measures were collected from teachers and students. Teacher measures were collected primarily at pre- (fall) and post-(spring) intervention during third and fourth grade (exceptions: teacher measures on students were not collected from cohort 2 teachers in the fall of grade 4). In fifth grade, teacher measures were collected in the spring (except there were no teacher reports on students in cohort 2; and the teacher-child rating scale was collected by cohort 1 teachers in the fall). Child self-report measures were also collected primarily at pre- (fall) and post- (spring) intervention during third and fourth grade (measures were not collected in the fall of fourth grade for cohort 2 students). In fifth grade, child self-reported measures were only collected in the spring from cohort 1 students. One additional measure was collected by students mid-way through implementation (winter) each year.

      Teacher-reported measures included the Teacher Report on Students (TRS) and Teacher-Child Rating Scales (TCRS). The TRS assessed student responsibility, social competence and motivation, and quantity and quality of parent involvement in education. The TCRS assessed student acting out behavior problems, shy-anxious behavior problems, learning problems, frustration tolerance, assertive social skills, task orientation, and peer social skills. Teacher ratings of child aggression were measured using the Behavior Assessment Scale for Children-2 (BASC-2) Aggression Subscale, Teacher Version. Conduct Problems were measured using the BASC-2 Conduct Problems Subscale, Teacher Version. Acting out behaviors were measured using the Acting Out subscale of the TCRS. Internal consistency on these measures ranged from .72 to .94.

      Child self-report measures included the Child Report (CR), which measured normative beliefs about aggression, self-efficacy for peer interaction in conflict and non-conflict situation, empathy, altruism, aggression, minor delinquency, school engagement, school connectedness, feelings of school safety, and victimization at school, and the What Would I Do? assessment, a measure of social problem solving skills and hostile attribution biases (this measure was collected during the winter of each year). Dependent variable measures collected included self-reported aggression (The Aggression Scale) as part of the CR, self-reported minor delinquent acts (the Frequency of Delinquent Behavior Survey) as part of the CR, victimization at school (the Victimization Scale) as part of the CR, normative beliefs about aggression (Normative Beliefs About Aggression Scale) as part of the CR, aggressive social problem solving (Social Problem Solving Measure) as part of the What Would I Do? survey, hostile attribution bias and aggressive interpersonal negotiation strategies (Home Interview) as part of the What Would I Do? survey. Internal reliability on these measures was high (range from .68 - .93).

      Analysis: Analysis was intent-to-treat. Main analyses were conducted using three-level growth models (alpha set at the .10 significance level). Unconditional growth models were initially examined to determine the developmental trajectories of aggression outcomes over the study period. Because curvilinear change was noted for a number of the outcomes, three-level non-linear (linear + quadratic) growth models were utilized with time at level 1, individual at level 2, and school-level variables at level 3. For outcomes with only three data points (self-reported aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies), only linear effects could be examined.

      Outcomes
      Implementation Fidelity: Teachers reported their dosage to the educational consultants on a weekly basis. Over the course of the study, teachers reported teaching an average of 34.8 lessons per year. Classroom fidelity was assessed by the ECs on a monthly basis through quality of implementation ratings based on their direct observation of teacher instruction. Teachers were rated on 1) quality of teaching the PATHS concepts; 2) modeling and generalization of PATHS concepts throughout the school day; 3) quality of student compliance during PATHS lessons; and 4) openness to consultation. Reliability on these four measures was high (.87-.90). Six of the seven schools averaged greater than three (out of four) on quality of teaching PATHS concepts, modeling and generalization of PATHS concepts, and openness to consultation. Five of the seven schools also attained a three or better average on quality of student compliance during PATHS lessons.

      Baseline Equivalence and Differential Attrition: Two-level binomial models including the aggression outcome variables (teacher and self-report), conduct disorder, acting out behavior problems, and minor delinquency were used to predict attrition as well as new enterer status. There was a higher level of attrition among students in the urban schools. There was no differential attrition or new entering between students in the intervention and control groups.

      Posttest: There was a significant program effect on teacher-reported conduct problems over time. While teacher-reported outcomes on aggression and acting out behavior were not significant, they trended in the direction of deceleration of both behaviors and effects were more pronounced at the later grades. On child-reported measures, there were significant linear effects of the program on students' normative beliefs about aggression, aggressive social problem solving, hostile attribution bias, and aggressive interpersonal negotiation strategies. Sensitivity analyses conducted also confirmed the original findings.

      Limitations: One limitation of this study is that the children evaluated were not exposed to the PATHS curriculum in K-2, potentially limiting the program effects had they received implementation of the full program. Additionally, the control schools did implement some form of social and character development programming, as mandated by the state, and this evaluation does not take into account differential implementation of SACD content between conditions. The intent-to-treat model used in this study may have complicated results, as students entering the intervention at different points were exposed to differential levels of the intervention. Conservative analyses were conducted in order to account for implementation quality/fidelity, student exposure, and SACD programming in control schools. Another limitation was that raters of the program were not blind to study condition, which may bias results. There was also no long-term follow-up to determine maintenance of program effects.

      Little, M., Berry, V., Morpeth, L., Blower, S., Axford, N., Taylor, R., ... Tobin, K. (2012). The impact of three evidence-based programmes delivered in public systems in Birmingham, UK. International Journal of Conflict and Violence, 6 (2), 260–272.

      This study was presented in an article that described the implementation and evaluation of three programs delivered in Birmingham, UK. A cluster, randomized, controlled trial of Promoting Alternative Thinking Strategies (PATHS) was conducted with children aged four to six years in 56 schools for one hour per week for two school years. The sample was stratified by percentage of free school meals and size of school and then randomly allocated to PATHS (n=29 schools) and control (n=27 schools). PATHS was provided to all children in reception (i.e., kindergarten) and first grade resulting in a total sample of 5397 children in the schools at baseline attending 196 classes (n=102 in intervention and n=94 in control).

      Data were collected at three points: at baseline (n=183 classes and 5074 children), mid-intervention (n=176 classes and 4998 children) and at immediate posttest (n=178 classes and 4994 children) two years after the start of the study. Data gathered yielded 4006 complete cases at all three waves (26% attrition), however the study adhered to the intent to treat principle and included results from participants regardless of dose received or dropout status.

      Measures were gathered using five scales from the teacher-completed Strengths and Difficulties Questionnaire and nine scales from the PATHS Teacher Rating Survey.

      Data were analyzed using hierarchical linear models to account for clustering at the classroom and school levels as well as to control for covariates. Data were analyzed both with and without missing data imputed and results were similar.

      Results indicated that while there were some statistically significant improvements in the intervention group compared to the control group at mid-intervention, these improvements disappeared posttest and no statistically significant differences were observed between groups at immediate posttest.

      Schonfeld, D. J., Adams, R. E., Fredstrom, B. K., Weissberg, R. P., Gilman, R., Voyce, C., ... Speese-Linehan, D. (2014). Cluster-randomized trial demonstrating impact on academic achievement of elementary social-emotional learning. School Psychology Quarterly, advance online publication.

      Evaluation Methodology

      Design:

      Recruitment: All 24 schools in a large, high-risk, urban school district in the Northeast were included in the study.

      Assignment: Schools in the study were divided into two clusters balanced for race/ethnicity, proportion of students qualifying for free or reduced price lunch, and school size. Using a block randomization procedure, the study then assigned one cluster to the treatment group and the other cluster to the control group. The 12 schools in the treatment group had 692 students, and the 12 schools in the control group had 702 students.

      Attrition: The longitudinal study followed the same cohort of students from 3rd grade to 6th grade. Assessments occurred at the end of 4th, 5th, and 6th grade, with the 6th-grade assessment representing a posttest. The analysis included only students who remained in the same treatment group (although not necessarily the same school) for the entirety of the 4-year program. This excluded 49% of the originally assigned students.

      Sample: The sample was approximately half male (51%) and more than half (68%) received free or reduced price lunches. Black students made up 48% of the sample; Latino students made up 41%; White students made up 9%; and other races made up the remaining 2%. Although the study reports no socioeconomic information, the school district is described as high-risk.

      Measures: The State Mastery Test, a statewide achievement test administered annually in the spring in Grades 4 to 8, measured problem-solving skills for academic tasks separately for math, reading, and writing. The test showed good validity and reliability.

      Analysis: The study used multilevel logistic regressions to account for student-level information nested within schools (but not within the clusters used for assignment). Variability of the student defines level 1 analysis and variability of schools defines level 2. Because testing began in 4th grade, after the program start, the study did not have baseline test scores.

      Intent-to-Treat: The study included only students who participated in the full program period and therefore does not conform to intent-to-treat.

      Outcomes

      Implementation Fidelity: The study measured fidelity and the effect of exposure to the intervention on outcome scores. Teachers presented about two-thirds of the available lessons, and exposure to more lessons predicted achievement.

      Baseline Equivalence: At baseline, there were no significant differences between conditions in sociodemographic measures. There were no differences between control and treatment schools for achievement test results the year before the beginning of the program, but those scores did not include program participants.

      Differential Attrition: The study found no differences between completers and dropouts for four sociodemographic measures but could not test for differences by baseline outcomes and did not analyze attrition by condition.

      Posttest: Fourth grade students in the treatment group had significantly higher odds of attaining basic proficiency in reading and math, but not writing. Students in the treatment group in 5th and 6th grades had higher odds of attaining basic proficiency in writing, but not reading or math. None of the control measures moderated these intervention effects.

      Long-Term: The study did not conduct any long-term follow-up.

      Study 13

      Berry, V., Axford, N., Blower, S., Taylor, R. S., Edwards, R. T., Tobin, K., ... Bywater, T. (2016). The effectiveness and micro-costing analysis of a universal, school-based, social-emotional learning programme in the UK: A cluster-randomised controlled trial. School Mental Health, 8, 238-256.

      Evaluation Methodology

      Design:

      Recruitment: All mainstream primary schools in Birmingham (N=299) were invited to participate in the intervention, of which an initial 64 agreed, though only 56 completed baseline data and were ultimately randomized. At baseline, participants were children aged 4-7 in Reception (Kindergarten) and Year One (first grade).

      Assignment: A total of 56 were randomized to intervention (N=29) or control conditions (N=27), stratified by school size and percentage of students receiving free or reduced price lunch. All schools had access to the national SEAL curriculum in the previous year, but the control condition received no content from the PATHS intervention.

      Attrition: Assessments occurred at baseline, 12-month posttest (end of Year 1), and 24-month posttest (end of Year 2). A total of 5,074 participants were randomized, with 4,477 (88%) retained at the end of the first year and 4147 (82%) retained at the end of the second year. In total, there was data at all collection points for 4006 participants, for an overall attrition rate of 21%.

      Sample:

      Just over half of schools were considered “large” (53.6%), and half had a “high percentage” of students that qualified for free or reduced price lunch. The participants were split evenly by gender with an average age of 5.07 and an overall 68.1% identifying as non-White.

      Measures:

      All measures were completed by teachers, who were also the program deliverers.

      Conduct problems, emotional difficulties, hyperactivity and peer relationships, and prosocial behaviors were assessed using the 25-item Strengths and Difficulties Questionnaire, which displayed acceptable validity (α=.73).

      Emotion regulation, pro-social behavior, social competence, aggressive behavior, internalization, relational aggression, peer relations, hyperactivity, impulsivity, learning behaviors, and academic performance were measured with subscales from the PATHS Teacher Rating Scale. No psychometric properties were described.

      Classroom environmental characteristics across 9 domains (teacher positive behavior, teacher negative behavior, teacher praise, class compliance, class negative behavior to the teacher, class pro-social behavior, class off-task behavior, and an overall score) were assessed by independent observers blinded to condition with the Teacher-Pupil Observation Tool. Psychometric properties were not described.

      Analysis:

      The program was evaluated using mixed linear regression models, fitting both school and classroom as a random effect and controlling for baseline outcome, age, gender, ethnicity, qualifying for free school meals, and special education, as well as teacher reports of classroom climate.

      Intent-to-Treat: Eight schools refused to provide baseline data and were subsequently excluded from analysis. Multiple imputation was used to handle other missing data.

      Outcomes

      Implementation Fidelity:

      Fidelity was assessed with two different methods: for exposure/dose and adherence, teachers completed weekly logs documenting each lesson and their level of adherence to or deviation from the curriculum, and a semi-structured questionnaire completed by PATHS coaches evaluating teachers’ program delivery and pupil engagement. The first year of implementation had a very low rate of completion (27%) and was subsequently discarded. Overall, adherence in the second year was high, and 50% of teachers delivered the program with “high fidelity.”

      Baseline Equivalence:

      The authors report that there were no significant differences between groups on outcome measures and key demographic characteristics.

      Differential Attrition:

      No formal tests, though the authors report that the analysis sample of “56 schools showed a good balance between PATHS and control groups in terms of the stratification variables” (245), and the Appendix CONSORT diagram shows comparable rates of attrition in each condition.

      Posttest:

      At the end of Year 1, compared to the control group, intervention students improved on 6 of 16 behavioral outcome measures. All of the improvements were on measures considered to be “secondary” to the authors: social competence, aggressive behavior, hyperactivity, impulsivity, peer relations, and learning behaviors. However, none of these gains were sustained at the end of Year 2, and there were iatrogenic effects in the treatment group on emotional regulation and conduct problems relative to controls.

      Subgroup analyses at the end of Year 2 demonstrated that students identified as having emotional difficulties did significantly better on overall difficulties in PATHS schools than in control schools. No other subgroup effects were noted.

      Finally, analyses examining 9 classroom environmental outcomes at mid-Year 1 (6 months post-baseline) found significant benefits of the intervention over the control group on 3 measures: teacher total positive behaviors, class behavior negative to teacher, and class off-task behavior.

      Long-Term:

      The final assessment occurred 24 months post-baseline, but the intervention was ongoing over those 24 months, and there were no overall beneficial effects.

      Study 14

      Evaluation Methodology

      Design

      Recruitment: Four public elementary schools from highly disadvantaged Baltimore City neighborhoods with poor school readiness participated in the evaluation. Consent was sought from all children in the schools’ kindergarten classrooms. Of 464 eligible children, 327 provided consent and entered the trial, though children not receiving consent still received the intervention.

      Assignment: Schools were randomly assigned to treatment (n=2) or control conditions (n=2). The preschool/kindergarten version of the PATHS curriculum was used as the primary intervention, which was taught by all kindergarten teachers in treatment schools, while kindergarten teachers in control schools provided instruction as usual. Since the treatment was administered grade-wise within school, randomization could not occur by classroom.

      Assessment: Students were assessed using teacher-rated measures of behavior (e.g., attention, concentration, aggression) and peer-reported nominations (e.g., likability, aggression, acceptance) at baseline and posttest, in the spring of the academic year. Attrition was not described for the baseline sample.

      Sample

      The majority of students at participating schools were eligible for free lunches and nearly all students were African American. Household income in the areas served by the schools averaged about $40,000 a year, and areas had moderately high crime rates (~70/1000 residents).

      Measures

      All instruments were administered at the beginning of the fall semester and end of the spring semester, after the program had concluded.

      Kindergarten teachers completed a series of measures (listed below) assessing child competencies.

      • Aggression was assessed with seven modified items from the Teacher Observation of Child Adaptation-Revised. Despite the modifications, no psychometric properties were described.
      • Internalizing was measured using five items from the Teacher Observation of Child Adaptation-Revised. Reliability was not reported for the subscale.
      • Social Competence was defined using 13 items from the Social Competence Scale. No measures of validity or reliability were reported.
      • Emotion Regulation was assessed with six items regarding the selected student’s coping mechanisms and temper control (a=.88).
      • Prosocial Behavior was examined with a subscale of seven items that displayed good reliability (a=.96).
      • Child Impulsivity and Inattention were measured using the Diagnostic and Statistical Manual’s ADHD Rating Scale (a=.92-.94).
      • Student-Teacher Closeness and Conflict were assessed with eight items from the Student-Teacher Relationship Scale (a=.92).
      • Positive Peer Relationships were assessed with the Peer Relations Questionnaire (a=.79).
      • Academic Skill was measured with four items drawn from the Academic Competence Evaluation Scales. Psychometric properties for the instrument were not reported.

      Though not explicitly stated, it appears that child cognitive functioning outcomes (listed below) were administered by the research team. It was not clear whether these researchers were blind to condition.

      • Delayed Gratification was assessed across four dimensions using the Delay of Gratification tasks. Reliability was not reported for the measure.
      • Behavioral Inhibition was tested using the computerized Whack-A-Mole game, which yielded four total measures. No psychometric properties were described.
      • General Intelligence was measured using the KBIT-2 composite measure, which demonstrated high reliability (a=.89-.96).
      • Emotional Intelligence was assessed using “FACES task” at posttest, only. The validity of the measure was not described.
      • Motor-Skills Impulsivity was tested with the Peg-Tapping Task. No measures of validity or reliability were reported.

      Analysis

      Multilevel growth models were used to evaluate the intervention, with two observations nested in each student. The models control for student gender and inherently adjust for baseline outcomes. A secondary analysis applied similar methods to a propensity score matched subsample (114 of 327 cases) to adjust for baseline differences in baseline differences in several behavioral outcomes. There was no adjustment for clustering in classrooms, which were the unit of delivery, or schools (the unit of randomization) beyond the condition difference.

      Intent-to-Treat: Subject attrition was not discussed.

      Outcomes

      Implementation Fidelity: The teachers delivering the intervention all completed at least 80% of the lessons affiliated with the treatment. However, observation-based fidelity ratings made by the program coordinator suggested some deviation from content, with an average score of 3.8 (76%) across all classrooms on a 5-point scale.

      Baseline Equivalence: The groups were not equivalent at baseline, with “preliminary analyses reveal[ing] significant baseline differences across treatment condition for multiple behavioral outcomes,” (pp.503) and several apparent differences in school and neighborhood characteristics in Table 1 with no significance tests (pp.497); See, for example, household income (control mean=$31,053 vs. treatment mean=$50,592) and overall crime rates (control=73.69 per 1000 vs. treatment=45.17).

      Differential Attrition: There was no discussion of student attrition, though all schools were retained.

      Posttest: At posttest, the treatment group showed improvement on 13 of 13 teacher-rated outcomes compared to controls. Gains were observed for aggression, internalizing, social competence, emotion regulation, prosocial behavior, impulsivity, inattention, teacher-child closeness and conflict, peer relationship problems and academic skills, as rated by the teachers who delivered the intervention. All but two of these impacts (academic skills and teacher-student closeness) were maintained in the propensity score-matched subsample.

      In the overall sample, treatment students significantly improved 2 of 10 direct cognitive functioning outcomes over the control group, including inhibition task accuracy and motor-skills impulsivity performance. However, these gains were not evident in the matched subsample.

      Limitations:

      • No information on student-level attrition
      • No information on reliability/validity provided for 8 of the 14 outcome measures
      • Teachers who delivered the program also completed the assessments (with effects in favor of the treatment on 13 of the 13 teacher-rated measures)
      • There was an effect on 15 out of 23 measures, but only 2 of these effects were assessed using independent measures (and it still wasn’t clear whether those collecting these data were blind to condition)
      • Small sample size (n = 4 schools, and schools were the unit of assignment)
      • Incorrect level of analysis with no adjustment for unit of randomization (schools)