Please take our brief survey

Blueprints Programs = Positive Youth Development

Return to Search Results

Promising Program Seal

Wyman's Teen Outreach Program®

Blueprints Program Rating: Promising

A nine month program that engages high school students in a minimum of 20 hours of community service learning annually and weekly meetings using TOP's Changing Scenes curriculum, with a goal of reducing rates of teen pregnancy, course failure, and academic suspension.

  • Original Developer: Brenda Hostetler
  • Current Owner/Purveyor: Wyman Center
  • Academic Performance
  • Sexual Risk Behaviors
  • Teen Pregnancy

    Program Type

    • Civic Responsibility/Education
    • Leadership and Youth Development
    • School - Individual Strategies
    • Skills Training
    • Social Emotional Learning

    Program Setting

    • Community (e.g., religious, recreation)
    • Residential Facility
    • School

    Continuum of Intervention

    • Universal Prevention (Entire Population)

    A nine month program that engages high school students in a minimum of 20 hours of community service learning annually and weekly meetings using TOP's Changing Scenes curriculum, with a goal of reducing rates of teen pregnancy, course failure, and academic suspension.

      Population Demographics

      TOP is open to all 9th-12th graders who attend schools that have adopted the program. The developers have implemented and evaluated this program with middle school youth as well; however, these evaluations failed to meet Blueprints quality standards.

      Age

      • Early Adolescence (12-14) - Middle School
      • Late Adolescence (15-18) - High School

      Gender

      • Male and Female

      Race/Ethnicity

      • All Race/Ethnicity

      Race/Ethnicity/Gender Details

      Results provided by gender (male/female) and race (black/white/Hispanic/other). Bull et al. (2016) found a program effect only for Hispanic participants.

      Teen Outreach is not targeted to youth with specific risk factors. Rather, the program views adolescence itself as a risk factor. Protective factors include enhanced feelings of autonomy and connectedness with adults, community, and school through community service.

      • Individual
      • School
      • Neighborhood/Community
      Risk Factors
      • School: Low school commitment and attachment
      Protective Factors
      • Individual: Academic self-efficacy, Problem solving skills, Prosocial behavior*, Prosocial involvement*, Skills for social interaction
      • School: Opportunities for prosocial involvement in education*
      • Neighborhood/Community: Opportunities for prosocial involvement*, Rewards for prosocial involvement

      *Risk/Protective Factor was significantly impacted by the program.

      See also: Wyman's Teen Outreach Program® Logic Model (PDF)

      Wyman's evidence-based Teen Outreach Program® (TOP®) is a positive youth development program designed to build teens’ educational success, life and leadership skills, and healthy behaviors and relationships. As a result, teens are better able to navigate challenges during the teenage years—a time when decisions matter.

      TOP is designed to meet the developmental needs of middle (6th-8th grades) and high school (9th-12 grades) teens in a variety of settings, including in school, after-school, through community organizations or in systems and institutional settings. Curriculum topics include: emotion management, problem-solving, decision-making, goal-setting, health and wellness, healthy decision making, self-understanding, social identity, empathy, communication, relationships and community.

      Wyman’s evidence-based Teen Outreach Program® (TOP®) is a positive youth development program designed to build teens’ educational success, life and leadership skills, and healthy behaviors and relationships. TOP aims to achieve four goals:

      • Improve social, emotional, and life skills
      • Support development of a positive sense of self
      • Strengthen connections to others
      • Improve academic outcomes and decrease risky behavior

      These goals are accomplished through implementation of the following core program components:

      • High quality facilitation by caring, responsive and knowledgeable TOP staff who build strong, supportive relationships with teens, and create engaging and empowering “TOP Club” experiences
      • At least 25 weekly peer group meetings
      • At least 12 lessons of TOP curriculum provided by facilitators who tailor the content to the teens’ needs
      • At least 20 hours of meaningful community service learning, which includes planning, action and reflection

      The TOP curriculum includes lessons that address content on adolescent development: skill-building, connections with others, and learning about one's self. Within each area of content, the curriculum includes foundational, intermediate, and advanced lessons that are designed to meet the developmental needs of young people in grades 6 through 12. Each of the 120+ lessons is written for a 45-minute time period with an additional extension activity available if time allows. The TOP curriculum is designed to allow for sequenced programming over multiple years (e.g., as part of a whole-school middle school approach), with a minimum duration of 9 months.

      The community service component has four phases: preparation, action, reflection, and celebration. Most of the time is spent in action, with reflection happening continuously throughout the community service experience. Service can involve direct action (e.g., tutoring, building a playground or garden), indirect action (e.g., bake sales, blood drives), and civic action for older students (e.g., voter registration, public speaking, educational theatre and awareness programs). There is a process through which students are able to choose the specific type of service they want to engage in.

      Wyman’s national replication model involves partnering with school districts or community organizations who implement TOP in schools and other settings. Wyman uses a Training of Trainers (TOT) model which is offered quarterly in St. Louis, Missouri. In some cases with large school districts, Wyman will offer the training onsite at schools. The TOT training is five days long and is facilitated by two Wyman TOP Trainers. The first 2.5 days cover the facilitator training, during which participants learn how to directly deliver TOP to teens. The next 2.5 days focus on teaching participants how to train others in delivering TOP and how to structure their TOP implementation to ensure the fidelity and quality of their TOP implementation within their home organizations.

      Wyman’s National Network team provides technical assistance to TOP replication partners through a variety of means, including conference calls and webinars. In addition, outcome and implementation data collection and reporting are supported through Wyman Connect, Wyman’s online TOP data system. Wyman’s National Network team works directly with replication partners to regularly monitor fidelity and quality of implementation, with re-certification occurring at the end of each program year.

      Teen Outreach is a preventive program that aims to use a developmentally-based approach to achieve positive outcomes in preventing pregnancy and academic failure. A developmental perspective suggests that successful adolescent adjustment (i.e. fewer problem behaviors) requires a simultaneous sense of autonomy and relatedness among peers and adults. In Teen Outreach volunteer activities, adolescents find themselves in help-giving rather than help-receiving situations. This, in turn, enhances their sense of autonomy and self-efficacy while at the same time fosters bonds with the community and school. The qualities of autonomy and relatedness are viewed as essential tools in navigating adolescence and making sound decisions around school and sexuality.

      • Attachment - Bonding
      • Cognitive Behavioral
      • Self Efficacy
      • Skill Oriented

      The efficacy of the Teen Outreach Program has been typically studied using quasi-experimental designs. In these designs, students at each studied site were divided into Teen Outreach program groups and comparison groups. Youth in each group filled out confidential questionnaires within the first few weeks of the program/academic year and the last few weeks of the program/academic year. Data collected included sociodemographic characteristics, whether they had ever caused a pregnancy or been pregnant, whether they had failed any courses in the prior school year, and whether they had been suspended academically in the prior school year. The post-test changed the reference time period to the program/academic year itself. To determine the efficacy of the Teen Outreach program, the post-test data were compared across the two groups, controlling for past behaviors and individual characteristics.

      Walsh-Buhi et al., (2016) tested the program using a cluster randomized trial of 28 public schools in 12 nonmetropolitan Florida counties. The schools were paired on structural and demographic characteristics and randomized into the intervention or control groups. A total of 7,667 students across two cohorts participated in the program. Primary outcomes regarding sexual activity included ever having had sex, recent sex, recent risky sex, and intention to have risky sex. Participants were assessed at baseline in the fall of the school year and at follow-up the next spring, approximately 9 months after baseline, with a 62% retention rate.

      Robinson et al. (2016) evaluated the program in a community setting. The researchers randomly assigned 4769 youth from Louisiana and 966 youth from Rochester to the intervention and control conditions. The youth were recruited through convenience sampling methods. Measures of sexual initiation and use of birth control were collected through self-reports at baseline and immediately after the intervention period. In Louisiana the control condition received no programming, and in Rochester the control condition received a work-readiness intervention.

      Bull et al. (2016) examined subjects from eight Boys & Girls clubs in Denver over 4 years. The control condition consisted of TOP programming alone, and the intervention condition consisted of TOP programming plus the text message supplement. Participants completed self-report measures at baseline and immediately at program completion.

      Allen et al. (1997) found that the Teen Outreach Program had statistically weak to moderate effects on problem behavior outcomes. The risk of each of the problem behaviors was less than half for the Teen Outreach than for the comparison group. Specifically, risk of becoming pregnant was 41% as large as the comparison group, risk of course failure was 42% of the comparison group, and risk of suspension was 39% of the comparison group. The hypothesized psycho-social mechanism of an increased sense of autonomy and relatedness among peers and adults was found to be significant, but only among middle school youth (Allen, 1994). No theoretically sound mechanisms have been identified and successfully tested for high school youth.

      Allen and Philliber (2001) found that pregnancy prevention effects were similar across all youth except that those who were already parents benefited more than non-parents. With respect to preventing course failure, Teen Outreach was only effective for females, was more effective for minorities than for whites, and for those who had prior academic suspensions than for those without prior suspensions. No significant differences in the effectiveness of Teen Outreach in preventing academic suspension were found.

      Allen et al. (1990) sought to identify what Teen Outreach programmatic conditions were most effective in producing successful outcomes. No programmatic differences were found in effectiveness whether the program was offered during the normal class day or after-school or whether the program used a little or a lot of the Teen Outreach curriculum. However, the number of volunteer hours and number of classroom hours (especially for younger participants relative to older participants) had a significant and positive influence in reducing problem behaviors.

      Walsh-Buhi et al. (2016) found that at posttest, intervention participants in Cohort 1 were significantly less likely than control group youth to report recent sex, risky sex, and sexual intentions, and Cohort 2 program youth had significantly reduced risky sexual intentions; however, there were no significant impacts of the program across both cohorts, overall.

      Robinson et al. (2016) reported no significant effects.

      Bull et al. (2016) found no significant effects overall, but for Hispanic participants only, the intervention condition reported significantly fewer pregnancies than those in the intervention condition.

      Two studies assessed the effect of the Teen Outreach Program® on the three different problem behaviors (Allen et al, 1997; Allen and Philliber, 2001):

      • The risk of pregnancy among Teen Outreach participants was 41% and 53% as large as the comparison group. The effect was larger among those who were already parents than among non-parents.
      • The risk of course failure was 42% and 60% as large as the comparison group, respectively. The program was only successful for females. Also, minorities and those with prior academic suspensions benefitted more from the program than did whites and those without suspensions.
      • The risk of academic suspension was 39% and 52% as large as the comparison group, respectively.

      Studies with No Effects

      • Walsh-Buhi et al., (2016) found that, at follow-up, compared to the control group, participants in the intervention group showed no significant improvements in risky sexual behaviors overall.
      • Robinson et al. (2016) reported no significant effects on initiation of sexual intercourse or use of birth control.
      • Bull et al. (2016) found no effects overall but in comparison to the control condition, the intervention condition showed significantly fewer pregnancies for Hispanic participants only.

      Only one study investigated possible mediating effects. According to Allen et al. (1994), program effectiveness for middle schoolers was mediated by the degree to which youth viewed that the program enhanced their autonomy and relatedness among peers and adults.

      The program consistently demonstrated effect sizes in the weak to moderate range. The effect size of Teen Outreach in the odds of causing pregnancy or becoming pregnant were .53; the effect size in the odds of failing a course were .60; and the effect size in the odds of failing a course were .52. The effect size in the odds of causing or becoming pregnant were .34 for parents and .82 for non-parents. The effect size in the odds of failing a course were .52 for females (compared to no effect for males); .54 for minority students (compared to .74 for majority students); and .57 for youth with prior academic suspensions (.34 for those without prior suspensions). The effect size in the odds of being academically suspended was .52. In Walsh-Buhi et al., (2016), odds ratios ranged from .40-.71, indicating some small to medium program effects on risky sexual behaviors for one cohort receiving the program, though there were no impacts overall. Robinson et al. (2016) reported no significant effects. Bull et al. (2016) did not report an effect size for the significant effect found in the study.

      These studies are not generalizable to the nation or to non-studied Teen Outreach programs. This limitation is due to participant self-selection into the program (except for Allen et al., 1997 and Walsh-Buhi et al., 2016), lack of information on site selection and/or the impact of site selection, and because the demographic distributions of the students do not reflect distributions at other sites or across the nation.

      Otherwise, the program reduced course failure more among girls than boys and among minorities than non-minorities.

      • In two studies--Allen & Philliber (2009) and Allen et al. (1994)--the authors do not indicate how the studied sites were selected or from what years the data were drawn. A third study (Allen et al., 1990) provides the year of the data but does not indicate the site selection method.
      • Only one study randomly assigned all youth to treatment or comparison groups (Allen et al., 1997), although even in this study, the youth that made up the sampling frame had already expressed an interest in the program.
      • In each study, the youth self-selected into the Teen Outreach program. Thus, the results can only be generalized to the type of youth who would be inclined to join this type of program.
      • The authors (Allen & Philliber, 2001) do not indicate which years the data were collected; how the 60 sites were selected; or whether data was collected on all Teen Outreach students in each site, as opposed to sampling Teen Outreach students within sites.
      • Program participants select the program freely or respond affirmatively to a recommendation. None of the program participants are actually assigned to the program, and certainly not randomly assigned. Many in the comparison groups were nominated by students in the intervention group.

      Walsh-Buhi et al., (2016):

      • No overall impacts on behavioral outcomes
      • No tests for differential attrition
      • Some baseline differences
      • A few small iatrogenic effects in subgroups

      Robinson et al. (2016):

      • No information on reliability or validity of sexual behavior measures
      • Possible violation of intent to treat
      • Significant differences between groups in ethnicity at baseline
      • Tests for differential attrition used baseline equivalence with the analysis sample but without comparison to randomized sample
      • No effect on behavioral outcomes

      Bull et al. (2016):

      • No information on reliability or validity of sexual behavior measures
      • Significant differences between groups in pregnancy at baseline
      • Some effects of baseline measures on attrition
      • Effect on behavioral outcome for Hispanics only

      Another study of the program neither examined program outcomes nor used a control groups. A 4-week pilot program (Devine et al., 2014) was conducted that added text messages to the main program. A total of 96 teens were enrolled in the study and received on average 11 messages per week related to the program’s content with references to celebrities and musicians in whom the participants indicated they were interested. More than half of these texts asked for a response from the participant and an automated message was sent acknowledging the receipt of the response. Text reminders regarding upcoming sessions were also sent out. Teens without mobile phones were set up with a buddy system to read text messages with another participant. Satisfaction surveys were collected from the participants during and at the end of the study.

      • Blueprints: Promising
      • Crime Solutions: Promising
      • OJJDP Model Programs: Promising
      • SAMHSA: 2.2-2.3

      DeVonne Bernard
      Director, Teen Outreach Program
      Wyman Center
      1401 LaSalle Lane, Suite 220
      St. Louis, MO 63104
      (314) 471-3864
      Devonne.Bernard@wymancenter.org

      Alexandra Peralta
      Program Performance Officer
      Children’s Services Council of Palm Beach County
      2300 High Ridge Road
      Boynton Beach, FL 33426
      (561) 374-7612
      Alexandra.Peralta@cscpbc.org

      Allen, J., Kupermind, G., Philliber, S., & Herre, K. (1994). Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the Teen Outreach program. American Journal of Community Psychology, 22(5), 617-639.

      Allen, J. P. & Philliber, S. (2001). Who benefits most from a broadly targeted prevention program? Differential efficacy across populations in the Teen Outreach program. Journal of Community Psychology, 29(6), 637-655.

      Allen, J., Philliber, S., Herrling, S., & Kuperminc, G. (1997). Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach. Child Development, 64(4), 729-742.

      Allen, J., Philliber, S., & Hoggson, N. (1990). School-based prevention of teen-age pregnancy and school dropout: Process evaluation of the national replication of the Teen Outreach program. American Journal of Community Psychology, 29(4), 505-523.

      Devine, S., Bull, S., Dreisbach, S., & Shlay, J. (2014). Enhancing a teen pregnancy prevention program with text messaging: Engaging minority youth to develop TOP® plus text. Journal of Adolescent Health, 54(3), S78-S83.

      Christina Donald
      Wyman Center
      600 Kiwanis Drive
      St. Louis, Missouri 63025
      Tel: 314-717-2071
      Website 1: www.wymancenter.org
      Website 2: www.teenoutreachprogram.com

      Study 1

      Allen, J. P. & Philliber, S. (2001). Who benefits most from a broadly targeted prevention program? Differential efficacy across populations in the Teen Outreach program. Journal of Community Psychology, 29(6), 637-655.

      Evaluation Methodology

      Allen, J. P. & Philliber, S. (2001). Who benefits most from a broadly targeted prevention program? Differential efficacy across populations in the Teen Outreach program. Journal of Community Psychology, 29 (6),637-655.

      The motivation for this evaluation was to learn whether the Teen Outreach program had greater efficacy for those students who were more at-risk for problem behaviors such as pregnancy and academic difficulties.

      Design: In this mix of a quasi-experimental and randomized study, data were collected from 3,277 youth in grades 9-12 (1,673 program youth, 1,604 comparison youth) from 60 sites over the course of 4 years. The actual years of the data collection were not reported. No information is provided as to how the 60 sites were selected.

      About 20% of the study youth were randomly assigned to either the Teen Outreach group or the comparison group. The remaining Teen Outreach youth chose to participate in the program by requesting it after hearing about it during school announcements or agreeing to participate after having been referred by teachers or counselors. The remaining comparison group youth were selected through one of two methods: (a) program participants nominated youth whom they thought would fill out the questionnaire the same way they did; or (b) program participants nominated youth whom they thought were from similar backgrounds and encountered similar life problems. The authors reported that analysis found no statistically significant differences between the randomized and non-randomized samples.

      Youth in the Teen Outreach program and in the comparison group filled out self-report questionnaires within the first few weeks of the beginning of the program and academic year. Program youth did this in their Teen Outreach classroom, while the comparison group youth filled out the questionnaire in study hall or some other school setting. The youth reported demographic characteristics and whether they had ever been pregnant or caused a pregnancy, whether they had failed any courses in the prior school year, and whether they had been suspended academically in the prior school year. At the end of the program and academic year, the youth repeated the questionnaire but reported only on outcomes for the academic year of the program. The Teen Outreach sample experienced an overall attrition rate of 8.9% and the comparison group exhibited a rate of 7.8%.

      Sample characteristics: Of the 3,277 sampled youth, the mean age was 16 years old, with the majority in 9th or 10th grades. The average mother's education level was slightly over a high school graduate, and slightly over half (about 55%) of the youth lived in two-parent homes. About 19% of the sample had prior academic suspensions and 32% had failed at least one course. About 8% had either caused or been pregnant in the past and about 3% were teen parents. The samples differed only slightly on gender and race profiles.

      Measures: A simple self-report questionnaire was administered within the first two weeks of the beginning of school in the fall (usually August or September) and at program exit near the end of the academic year (typically May or June). Students were asked their age, their grade in school, their race/ethnicity (black, white, Hispanic, other), their mother's education level (1=not a high school graduate through 4=college graduate), whether they lived in a two-parent household, whether they had been suspended from school in the previous school year, whether they had failed courses in the previous school year, whether they had ever caused a pregnancy or been pregnant, and whether they were currently a parent.

      Analysis: This study used three logistic regressions in an intent-to-treat analysis. Each regression's binary dependent variable represented a particular problem behavior outcome at program exit (i.e., 1 if student became or caused a pregnancy in program year, 0 otherwise; 1 if student failed a class in program year, 0 otherwise; and 1 if student was academically suspended in program year, 0 otherwise). The independent variable that indicated program participation reflected the effect of the program on the three different outcome variables. Control variables included demographics and the baseline outcomes of pregnancy, course failure, and academic suspension status at the beginning of the year. In addition to general program effectiveness, interactions between program participation and other factors (demographic background, behavioral risks, familial risks) were tested to determine whether program effectiveness differed for students with different characteristics.

      Outcomes

      Implementation fidelity: Although program facilitators were trained, the content of the program (i.e., which components of the curriculum were used and to what extent) and the quality of the implementation at each site were not measured.

      Baseline equivalence: Only two relatively small but significant differences in sociodemographics and background risk factors existed between the program participants and the comparison groups. Females represented 29.1% of the comparison group and only 24.6% of the program participants, and 46.1% percent of the comparison group identified as black, compared with 44.3% percent of the program participants.

      Differential attrition: The Teen Outreach sample experienced an overall attrition rate of 8.9% and the comparison group exhibited a rate of 7.8%. Attriters were more likely than non-attriters to become pregnant or cause a pregnancy and/or fail courses in school. They were also more likely to be younger, male, members of a minority group, and living in single-parent households. However, attriters were equally likely to come from the Teen Outreach sample as from the comparison sample.

      Teen pregnancy: Self-reports showed that Teen Outreach participants had 53% (p<.01) of the risk of pregnancy exhibited by the comparison group. The only characteristic that resulted in significant differences by individual or behavioral characteristics was for teen parent status. Parents in Teen Outreach were 82% less likely to become or cause pregnancy than the comparison group, while non-parents in Teen Outreach were 34% less likely to become or cause pregnancy than non-parents in the comparison group. Note that Teen Outreach was no more or less effective for youth who had experienced prior pregnancy but were not parents.

      Course failure: Self-reports indicate that Teen Outreach participants had 60% (p<.001) of the risk of course failure exhibited by the comparison group. Significant differences in the effect of the program were found with respect to gender, race/ethnicity, and prior academic suspension status. The program was significantly effective for females (Teen Outreach females had 52% of the risk of course failure compared to females in the comparison group), but not for males. Teen Outreach was significantly effective for all races, but especially effective for minorities. Minority students in Teen Outreach exhibited 52% (p<.001) of the odds of course failure compared to minorities in the comparison group. Program youth with prior academic suspensions had 43% (p<.001) of the risk of course failure compared to the comparison group youth, but for youth without prior academic suspensions this figure was 66%.

      Academic suspension: Students participating in Teen Outreach exhibited 52% (p<.01) of the risk of academic suspension compared to the comparison group. No significant differences in the effectiveness of Teen Outreach were found by individual or behavior characteristics.

      Effects of differential attrition: To assess whether differential attrition affected the results, the researchers re-ran the models using multiple datasets in which the missing data were imputed based on assumptions. In the pregnancy model, all the main effects and interactions remained significant. In the course failure model, participation and interactions remained significant through the model that assumed problem behaviors at twice the baseline rate, but not in a model that assumed problem behaviors at four times the baseline rate.

      Allen, J., Philliber, S., Herrling, S., & Kuperminc, G. (1997). Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach. Child Development, 64 (4), 729-742.

      This evaluation addresses the general efficacy of the Teen Outreach program.

      Evaluation Methodology

      Design: From 1991 through 1995, 25 Teen Outreach programs participated in this evaluation of the Teen Outreach program. Total participation included 342 Teen Outreach Program youth and 353 control group youth in grades 9 through 12.

      In the spring prior to each academic year, all known Teen Outreach programs in the country were contacted and asked if they would participate in a random assignment evaluation of the program the following school year. Approximately 10% of the known sites (25 sites) eventually participated. Sites that refused to participate consistently reported three main reasons why they refused: a) random assignment of students was too awkward or unwieldy; b) they did not find enough interested students to fill both a control and a treatment group; and c) they found random assignment objectionable because it would require "turning away interested students."

      At the early stages of data analysis, two sites had attrition rates between 80 and 90 percent and one site showed a significantly higher rate of problem behaviors for the control group than the program group. These three sites were removed, and 22 sites were used in the final analysis.

      The 22 sites provided a sample of 342 students who participated in Teen Outreach and 353 who did not. Students indicated interest in the program by enrolling as part of the health curriculum, as an academic elective or as an after-school activity. After the students indicated their interest, they were randomly assigned to either the Teen Outreach program or the control group.

      Over the academic year duration, attrition was 5.3% for the Teen Outreach program and 8.4% for members of the control group.

      Sample characteristics: The typical student in the sample was about 16 years old, in 9th or 10th grade, female (roughly 85%), and black (about 67%). The high rate of female participation is explained by teachers and counselor's viewing the program as part of pregnancy prevention and therefore recommended more females than males.

      Measures: Students filled out brief self-report questionnaires within a few weeks of the start of the academic year and within a few weeks of the end of the academic year. Demographic data collected included age, grade level, race/ethnicity (black, white, Hispanic, other), household composition (one- or two-parent), and parents' education level (four categories ranging from not graduating high school to college graduate). The problem behaviors questions included whether they had ever been pregnant or caused a pregnancy, whether they had failed any courses during the prior year, and whether they had been suspended academically in the prior year. In the posttest, the problem behaviors referred to the academic year that they were completing at the time.

      Analysis: Logistic regression examined the effects of program participation on problem behaviors at exit (pregnancy, course failure, and academic suspension) while controlling for problem behaviors at entry and demographic characteristics.

      Outcomes

      Baseline Equivalence: No statistically significant differences in demographic characteristics existed across the groups. However, the control group exhibited significantly higher rates of problem behaviors at entry. The potential effects of this are discussed in the outcomes section.

      Differential Site Attrition: No analysis was made to determine whether the three sites that dropped out of the study had students who were significantly different than the other sites demographically or in terms of problem behaviors at entry. However, an alternative analysis was conducted with the three sites included and all effects of the program remained at the same significance levels. Unfortunately, the method used to impute data in order to run the alternative analysis was not reported.

      Differential Student Attrition: Comparing attriters with completers, the researchers found that students who dropped out of the remaining 22 program sites were not significantly different than those who completed the program except that they were more likely to have had or caused a prior pregnancy, been academically suspended, been younger and been male. Comparing attriters across group status found that attrition was not linked to being in the control or program group.

      Posttest: The Teen Outreach program had statistically significant and moderate effects on problem behavior outcomes. The risk of each of the problem behaviors was less than half for the Teen Outreach than for the control group. Specifically, risk of pregnancy was 41% as large as the control group, risk of course failure was 42% of the control group, and risk of suspension was 39% of the control group.

      Observed program effects could be the result of the Teen Outreach group having significantly lower rates of problem behavior at entry. Youth with fewer troubles may be more open to the program and may experience change more easily than the more troubled control group. To test this possibility, the researchers created a dichotomous variable that indicated whether the site had a higher number of problems at entry in the Teen Outreach group relative to the control group. The models were re-run with this new variable interacted with program participation. None of these interactions were significant, leading to the conclusion that the lack of equivalence between Teen Outreach and control group problem behaviors at entry was not responsible for the observed program effects.

      Dosage effects: Two indicators of program intensity were the number of classroom sessions attended and the number of hours of volunteer service. The models were re-run with continuous variables that represented dosage in place of the program participation dichotomous variable. No significant effects were found for program intensity on pregnancy or academic suspension, but a significant effect was found for number of volunteer hours on course failure.

      Limitations

      The following limitations were found in this evaluation:

      • The site sample was not random. Sites that were able to manage the random assignment of youth were more likely to agree, which may indicate that the participating sites are better run, which may bias the results towards higher effect sizes.
      • The site sample was small. Only 10% (25) of the invited sites chose to participate.
      • The method used to impute data when assessing the effects of removing three sites from the analysis was not described, nor were the results of the analysis of imputed data presented.
      • Program intensity factors were substituted for program participation and found to be mostly insignificant.

      Allen, J., Philliber, S., & Hoggson, N. (1990). School-based prevention of teen-age pregnancy and school dropout: Process evaluation of the national replication of the Teen Outreach program. American Journal of Community Psychology, 18 (4), 505-523.

      The purpose of this quasi-experimental evaluation was to discover the reasons for possible relative effectiveness among implementations of the Teen Outreach program. In addition to looking at the demographic characteristics of participating youth, the authors investigated several site-level programmatic differences that may induce different outcomes by site:

      • Structural factors, including meetings during school vs. after school and credit vs. not-for-credit;
      • Intensity factors including number of hours in the classroom and number of volunteer hours; and
      • Curriculum factors, including the overall degree to which the standard Teen Outreach curriculum was used and the degree to which different components of the curriculum were used.

      Evaluation Methodology

      Design: This evaluation was designed to assess both individual and program-level factors that may influence the effectiveness of the Teen Outreach program in reducing problem behaviors in youth. Data were collected at the site- and the individual-level during the 1986-1987 academic/program year. Of the 70 Teen Outreach sites in the nation, the study sampled 35 sites in 30 schools. The authors did not indicate how the evaluation sites were selected, but reported that the sample was geographically disperse and the sampled sites did not differ significantly from the non-sampled site. The authors did not indicate which site characteristics were used to draw this conclusion.

      Program participants could have entered the program by electing to participate in response to school-wide announcements or by agreeing to participate after they had been targeted by teachers or counselors. After the program youth were chosen, the comparison youth were quickly selected through one of two methods: (a) the program participants nominated youth whom they thought would fill out the questionnaire the same way they did; or (b) teachers and counselors who had made referrals to Teen Outreach identified other youth whom they thought had similar sociodemographic backgrounds and experienced similar problems as the youth chosen for the program.

      Participants in the study included 632 Teen Outreach participants and 855 comparison students in grades 7-12. Attrition over the course of the program year was 2.4% among Teen Outreach students and 4.8% among comparison students. An additional 6.5% of the Teen Outreach students did not complete exit questionnaires and were excluded from the analysis. Consistent with the intent-to-treat principle, the analysis used all subjects with available data.

      Sample characteristics: Students ranged in age from 11 to 19. The typical member of the sample was 16 years old and in 10th grade. Almost 70% of the sample was female, about half were white, and about a third were black. The mean parents' education for the sample was slightly more than a high-school graduate and slightly more than half of the youth lived in two-parent households.

      Measures: A simple self-report questionnaire was administered within the first two weeks of the beginning of school in the fall (usually August or September) and at program exit near the end of the academic year (typically May or June).

      In the entry questionnaire, students were asked a set of demographic questions (age, grade in school, race/ethnicity, mother's and father's education levels, and whether they lived in a two-parent household). Next, students were asked a set of questions about three specific problem behaviors (whether they had been suspended from school in the previous school year, whether they had failed courses in the previous school year, and whether they had ever caused a pregnancy or been pregnant).

      In the exit questionnaire, they were asked the same questions regarding problem behaviors, but the time reference period was the academic year just ending. In addition to the three primary problem behaviors, they were also asked if they intended not to return to school in the fall.

      The researchers chose to sum these behaviors into an entry problem behavior syndrome score (ranging from 0 to 3 problem behaviors) and an exit problem behavior syndrome score (ranging from 0 to 4 problem behaviors). By doing so, they addressed the difficulty of having low base rates on individual problem behaviors and also were consistent with the understanding at the time that a "syndrome" was more meaningful than individual inter-correlated behaviors. The mean entry problem behavior syndrome scores were .80 for the Teen Outreach group and .66 for the comparison group (p<.05).

      Site-level data were collected from facilitators at each site across the following dimensions:

      • Intensity measures: the average number of volunteer hours worked and the average number of classroom hours;
      • Structural measures: whether the program was offered during school or after school and whether it was for-credit or not-for-credit; and
      • Curriculum measures: a four point likert-scale measured how much (none, a little, a lot, almost all) facilitators used the curriculum overall and by curriculum category. The categories were identified by factor analysis to be an orientation section, a self-awareness section, a social-awareness section, and a developmental section.

      Analysis: The data used for this study were multi-level in nature, but the analysis was not. Individual students were nested within programs and each was assigned the average program characteristics from that site. This evaluation conducted two primary analyses. First, to determine the general effectiveness of the program, problem behavior outcomes were compared across the Teen Outreach youth and the comparison youth. Second, among Teen Outreach programs, researchers sought to identify the role of site-level factors in the efficacy of the program. As such, the regression models were run with individual-level characteristics as controls, with site-level intensity measures as explanatory factors, with site-level structural measures as explanatory factors, and with site-level curriculum used as explanatory factors.

      Outcomes

      Baseline Equivalence: Compared to the Teen Outreach participants, the comparison group youth had fathers with a slightly higher level of education, were more likely to live in a two-parent household, were less likely to have failed courses in the prior year, and had fewer problem behaviors on average.

      Differential Attrition: Without reporting figures, the authors report "no significant effects of loss-to-follow-up on demographic or problem behavior measures, nor were there significant interactions of loss-to-follow-up with membership in the program versus comparison group for any of these measures."

      Posttest: At the beginning of the program, the Teen Outreach participants had more problem behaviors, on average, than the comparison group (p<.001). At program exit, the Teen Outreach participants had fewer problem behaviors, on average, than the comparison group (p<.05). Thus, compared to non-participants, participating in Teen Outreach was associated with a reduced number of problem behaviors.

      The average number of volunteer hours at a site had a significant and positive but weak effect on reducing the number of problem behaviors (beta=-.09, p<.05). The number of classroom hours also had a positive effect in reducing the number of problem behaviors, but the effect was stronger for the younger participants (beta for grade x class hours = .10, p<.05). Whether a program was offered during the school day for credit or after-school as an activity had no significant impact in the effectiveness of the program in reducing problem behaviors. Also, the effectiveness of the program was not significantly altered by the amount of the overall curriculum used or by different usage of the categories of the curriculum.

      Limitations

      The major limitations of this study include:

      • The evaluation's overall lack of generalizability. This inability to generalize is due to student self-selection into the program, lack of clarity around site selection, and demographic characteristics of the sample that do not generalize to the population.
      • The estimates of program-level factors are made using the individual as the unit of analysis. Thus, standard errors are likely under-estimated and the results are likely biased upwards.
      • Non-equivalence exists between the treatment group and comparison group, possibly because neither group was randomly selected.
      • Control group participants were nominated by program participants or teachers rather than randomly selected.
      • Using a problem behavior syndrome approach assumes that each of the behaviors is weighted the same. Further, the impact of the program cannot be differentiated between reducing pregnancy or academic failures.

      Allen J., Kuperminc, G., Philliber, S. & Herre, K. (1994). Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the Teen Outreach program. American Journal of Community Psychology, 22 (5), 617-638.

      The intention of this Teen Outreach evaluation was to identify possible psycho-social mechanisms that may help explain why the Teen Outreach program helps prevent teen pregnancy and school failure. A developmental perspective suggests that successful adolescent adjustment (i.e., fewer problem behaviors) requires a simultaneous sense of autonomy and relatedness among peers and adults. The evaluators hypothesize that sites that successfully promoted student autonomy and relatedness and gave the students the chance to perform autonomy-enhancing volunteer work would be more successful in reducing problem behavior than other sites.

      Evaluation Methodology

      Design: This quasi-experimental study used individual and site-level data from 66 different Teen Outreach programs across the country. The researchers did not indicate how the 66 sites were selected or from which program year(s) the data were drawn.

      Participants enrolled in Teen Outreach by a variety of ways: it could have been part of the required curriculum, they could have chosen it as an elective or taken it as an after-school activity; or they could have agreed to take the program in response to a recommendation from a teacher or guidance counselor. Previous research has shown that the method of entry into the program is unrelated to outcomes. After choosing the program youth, the investigators quickly selected comparison youth through one of three methods:

      1. Program participants nominated youth who they thought would fill out the questionnaire the same way they did; or
      2. Program participants nominated youth whom they thought were from similar backgrounds and encountered similar life problems; or
      3. Comparison youth were randomly selected from a pool of youth who had wanted to enroll in the Teen Outreach but the program was already at capacity.

      Within the 66 sites, data were collected from 1,020 Teen Outreach youth and 1,013 comparison group youth in grades 7 - 12. Attrition was 2.4% among Teen Outreach students and 2.7% among comparison students.

      Sample characteristics: Students ranged in age from 11 to 19 years old. The typical member of the sample was 16 years old and in the 10th grade. About 70% of the sampled students were female, and roughly 40% were black and 40% were white. The mean parents' education level was just above a high school graduate and slightly more than half of the youth came from two-parent households.

      Measures: A simple self-report questionnaire was administered within the first two weeks of the beginning of school in the fall (usually August or September) and at program exit near the end of the academic year (typically May or June). Students were asked their age, their grade in school, their race/ethnicity, their parents' education level (1=not a high school graduate through 4=college graduate), whether they lived in a two-parent household, whether they had been academically suspended from school in the previous school year, whether they had failed courses in the previous school year, and whether they had ever caused a pregnancy or been pregnant. In the exit questionnaire, the reference time period was the academic year just ending.

      To measure the degree to which programs fostered senses of autonomy and relatedness, a questionnaire was administered to students and to facilitators during the final month of the program. This questionnaire was intended to measure (a) the extent to which the program was viewed as promoting adolescent autonomy and relatedness; and (b) whether the quality of the volunteer experience was likely to enhance the students’ sense of competence and autonomy.

      The questionnaire was structured similar to the Perceived Competence Scale for Children, which is intended to reduce the effects of a pull for social desirability. For each item, two contrasting stems were presented side by side (e.g., "some kids feel like their facilitators like them a lot” and “other kids feel like their facilitator just likes them OK.”). The youth were asked to pick one of the two statements that they most agreed with and then decide whether it was “really true” or “sort of true.” The autonomy and relatedness questions focused on whether the youth felt like they could make their own decisions within the program, whether their views were respected, etc. The quality of volunteer experience questions focused on whether the youth felt like they were learning new skills, whether they might like to continue volunteering after the program ended, etc. The two scales had adequate internal consistency (alpha=.87 and alpha = .72, respectively).

      The questionnaire was also given to facilitators who were asked to answer as they thought their class would answer. The adolescent autonomy and relatedness scale had an alpha of .81 and the quality of volunteer experience scale had an alpha of .52. The authors report that the “validity of this inventory approach has been established in other studies, in spite of its reduction of alpha levels”.

      Students were given the standard problem behavior questionnaire at the beginning of the program and at the end. Both program youth and facilitators were given the autonomy and relatedness questionnaire and the quality of volunteer experience questionnaire at some point in the last month of the program. Students were assured of confidentiality and that their data would not be shared with school officials.

      Analysis: The data used for this study were multi-level in nature. Individual students were nested within programs and each was assigned the average program values for program level measures. This evaluation conducted two primary analyses. First, to determine the general effectiveness of the program, problem behavior outcomes were compared across the Teen Outreach youth and the comparison youth. Second, among Teen Outreach programs, researchers sought to identify whether feelings of autonomy and connectedness and quality volunteer experiences mediated the program's effectiveness. Also, to control for potential unmeasured site-wide factors, the authors included residualized problem behavior change scores for the comparison group for each site.

      Linear regression models were run with the number of problem behaviors at exit as the dependent variable. The covariates were entered sequentially, as follows: individual-level data (number of problem behaviors at entry, grade level, residualized problem behavior change score); then the primary variable of interest (promotion of autonomy and relatedness scale in regression #1 and quality of volunteer experience in regression #2); and, finally, interactions of grade level with the primary variable of interest. Each model was run twice, once using student ratings and once using facilitator ratings.

      Outcomes

      Baseline Equivalency: The Teen Outreach participants and the comparison group were very similar with respect to demographic characteristics and program behaviors at entry. The only statistically significant difference was that the proportion of females was slightly higher among the Teen Outreach group. This difference can be attributed to adult referrals into a program that they view as primarily a pregnancy prevention program.

      Differential Attrition: Attrition was similarly low in both the treatment and comparison groups.

      Posttest: At program entry, there was no significant difference in the mean number of problem behaviors (ranging from 0 to 3) between the Teen Outreach youth and the comparison youth (.61 vs.64, respectively). After controlling for problem behaviors at program entry, the mean number of problem behaviors was significantly lower (p<.001) for youth in the Teen Outreach program (.48 vs.67, respectively).

      When testing for site-level effects, the researchers used averaged site-level data from both the students and facilitators. In the student rating model, programs with higher scores on the promotion of autonomy and relatedness scales were significantly more effective in reducing problem behaviors (p<.01). However, this effect was only true for middle school students (p<.05). Likewise, in the facilitator model, the only significant effect of promotion of autonomy and relatedness was for middle school students (p<.01).

      The regressions were run again with the quality of volunteer experience as the key explanatory variable. In the student rating model, programs with higher scores on the quality of volunteer experience scale were significantly more effective in reducing problem behaviors ( p<.05), but once again this effect only held for middle school students. In the facilitator rating model, no significant effects of the quality of volunteer experience was found. Additionally, the effect of the mean number of volunteer hours on program success was not significant.

      The authors conducted an analysis to determine if the small effect found for quality of volunteer experience (i.e., enhancing autonomy and relatedness) on reducing problem behaviors for middle school students was a mediating effect of the volunteer experience itself. The authors conclude that, in fact, a portion of the volunteer experience effect is mediated through the perceived autonomy and relatedness aspects of the program.

      In sum, the proposed psycho-social mechanisms were significant only for middle schoolers. This was surprising because previous research has shown that the overall Teen Outreach program is more effective, in general, for high school youth. The authors concluded that middle school youth are more in need of an environment that fosters this sort of development, while high school students may already have achieved a certain level of autonomy and relatedness. High school youth are therefore less dependent on getting this specific developmental competency within the program and instead benefit from other aspects of the program.

      Limitations

      The major limitations of this study include:

      • The evaluation's overall lack of generalizability. This inability to generalize is due to student self-selection into the program, lack of clarity around site selection, and demographic characteristics of the sample that do not generalize to the population.
      • The estimates of program-level factors are made using the individual as the unit of analysis. Thus, standard errors are likely under-estimated and the results are likely biased upwards.
      • Non-equivalence exists between the treatment group and comparison group because neither group was randomly selected.
      • Control group participants were nominated by program participants or teachers rather than randomly selected.
      • There is no independent measure of autonomy and relatedness. The measure used in this study substantially overlaps with the students' negative or positive experience of the program.
      • Success in the Teen Outreach program may influence the measure of autonomy and relatedness rather than vice-versa.

      Walsh-Buhi, E. R., Marhefka, S. L., Wang, W., Debate, R., Perrin, K., Singleton, A., Noble, C. A., Rahman, S., Maness, S. B., Mahony, H., Ziemba, R., Malmi, M., Marwah, E., Hall, K., Turner, D., Blunt-Vinti, H., Noble, S., & Daley, E. M. (2016). The impact of the Teen Outreach Program on sexual intentions and behaviors. Journal of Adolescent Health, 59, 283-290.

      Evaluation Methodology

      Design:

      Recruitment: Schools were recruited for participation if they were located in nonmetropolitan Florida counties (those with a population of less than 900,000), if they had the capacity to implement the program, and if they had higher rates of adolescent births, STIs, high school dropout, graduation, or out-of-school suspension rates compared to other communities. A total of 28 public schools within 12 counties were selected for program implementation, with two cohorts of 9th grade classes recruited for participation (n=9,220 students eligible, 7,667 consenting).

      Assignment: The 28 schools were matched into pairs based on county, courses offered, school size, region/proximity, and presence of block scheduling, and then randomly assigned to either the intervention (n=14 schools, 3,533 students) or control group (n=14, 4,134). In the intervention group, the intervention was delivered as a supplement to the usual health curriculum while control schools continued with health education as usual.

      Attrition: One pair of schools (1 treatment, 1 control) dropped out during the first year. At follow-up, cohort 1 had a retention rate of 85% (n=3,404) and cohort 2 had a retention rate of 75% (n=2,758).

      Sample:

      Across both cohorts the sample was evenly split on gender with a mean age of 14.6 at baseline. Nearly 60% of respondents were white, with 20% reporting as Hispanic, 10% as Black, and the remaining 10% as other races/ethnicities. Approximately 15% of the sample reported recent sexual activity; of those who reported recent sexual activity, 27% reported recent risky sex. About 32% of the entire sample intended to have sex within the next year, with 8% of those intending risky sex in that same time frame.

      Measures:

      Participants were given a series of self-report questions at baseline and at posttest, approximately 9 months after baseline. Outcomes of interest were: ever had sex, sexual intercourse in the previous 3 months, not using a condom in the previous 3 months, and ever been pregnant or gotten someone pregnant. Secondary outcomes included intention to have sex in the next year, and, among those who intended to have sex in the next year, their intention to use a condom or not.

      Analysis:

      The effects of the program were evaluated using mixed-effects logistic regression models with adjustments for clustering in schools, controlling for baseline demographics and behaviors. Many different subgroup impacts were examined, including differential impacts of the program across cohorts, genders, and by baseline behaviors.

      Intent-to-Treat: Multiple imputation was used to account for missing data, with all cases used in the analysis.

      Outcomes

      Implementation Fidelity:

      Fidelity was not assessed.

      Baseline Equivalence:

      There were significant differences between groups in racial/ethnic makeup, but no other baseline differences in demographics or behaviors.

      Differential Attrition:

      There was a higher rate of attrition in the second cohort than the first, and while the study states that they test for condition equivalence of those retained at posttest, the results of the attrition analyses were not presented.

      Posttest:

      Overall, at posttest, there were no significant differences between the treatment and control groups on any measures using the pooled Cohort 1 and Cohort 2 data. However, subgroup impacts were much more positive, especially in Cohort 1. Treatment participants in the first cohort were significantly less likely than control group youth to report recent sex, risky sex, and sexual intentions. Females showed lower odds of recent and risky sex, while males showed decreased odds of recent sex. Of those inexperienced at baseline, intervention group participants were significantly less likely to report both ever having had sex and recent sex; females were significantly less likely to report ever having been pregnant, ever having sex, and having recent sex. Of those in Cohort 1 considered experienced at baseline, intervention participants were significantly less likely to report risky sex; females were significantly less likely to report recent sex and risky sex. There was an iatrogenic effect of the program for Cohort 1 males who were sexually experienced at baseline, as they actually displayed greater odds of risky sex than their control group counterparts.

      The intervention participants in Cohort 2 reported significantly reduced intentions to have risky sex compared to the control group. However, of those considered experienced at baseline, intervention participants had significantly higher odds of recent sex than those in the control group—an iatrogenic effect.

      Long-Term:

      The follow-up assessment occurred shortly after treatment ended.

      Evaluation Methodology

      Design:

      Recruitment: The researchers recruited 4769 youth in Louisiana and 966 youth in Rochester via convenience sampling methods. Recruitment took place in school and community settings during the academic years of 2013-2015, and the sample contained three “annual cohorts.” Eligible participants included those who lived in the targets areas, spoke English well enough to complete the study materials, and had not previously participated. In Louisiana youth aged 12 to 17 years were included and in Rochester youth aged 11 to 14 years were included.

      Assignment: Individuals were randomly assigned to the conditions in Louisiana and 33 clusters were randomly assigned to the conditions in Rochester. In Louisiana the control condition received no programming, and in Rochester the control condition received a work-readiness intervention.

      Attrition: Of the 4769 youth in Louisiana, 2428 (51%) completed the follow-up survey, and of the 966 youth in Rochester, 824 (85%) completed the follow-up survey.

      Sample:

      The sample targeted areas with high rates of adolescent pregnancy and included multiple regions across Louisiana and Rochester. In Louisiana, the sample was 90% black and 60% female, with an average age of 13.9 years (SD = 1.59). In Rochester, the sample was 64% black and 52% female with a mean age of 12.3 years (SD = 1.11).

      Measures:

      Participants completed self-report outcome measures. Sexual activity was measured through a yes/no question about whether the participant had previously engaged in intercourse; the posttest initiation measure included only those who had not already had intercourse at baseline. A yes/no question asked about whether the participant had engaged in intercourse without birth control in the previous 3 months.

      Analysis:

      Logistic regression models were used to analyze predictors of the binary outcomes, with gender, age, race, ethnicity, and cohort year used as covariates. The baseline outcome was included in models of the use of birth controls but not for initiation of sexual intercourse (all analyzed subjects had not previously engaged in sexual intercourse). The analysis was done at the individual level with multivariable logistic regression in Louisiana, and at the clustered level with multilevel modeling in Rochester.

      Intent-to-Treat: The researchers stated that they used intent-to-treat analyses by including all individuals who completed the posttest. However, they also stated that “During the course of the study, 3 study sites were dropped from the program.”

      Outcomes

      Implementation Fidelity:

      The researchers reported that “process measures” were recorded after each session by facilitators to assess fidelity. Approximately 10% of sessions were observed and evaluated as well and high ratings were given by evaluators. However, only 21% of youth in Louisiana and 48% of youth in Rochester in the intervention condition attended at least 75% of the sessions.

      Baseline Equivalence:

      The researchers tested for baseline equivalence between the intervention and control conditions on demographic variables and outcome measures and found no significant differences in Louisiana. In Rochester, the intervention group included significantly more Hispanics. However, the tests used the analysis sample of those with posttest data rather than the randomized sample.

      Differential Attrition:

      Differential attrition was not analyzed directly. Tests for baseline equivalence for the analysis sample are relevant but lacked comparison to the randomized sample, included only 5-6 baseline measures, and showed one difference.

      Posttest:

      No significant effects on outcome variables were found.

      Long-Term:

      Long-term effects were not tested.

      Evaluation Methodology

      Design:

      Recruitment: Participants were recruited from eight Boys & Girls clubs in Denver over 4 years. Eligible participants included active club members between 14 and 18 years of age. A total of 3643 youth fit all eligibility criteria, and of these, 852 consented to participate in the study.

      Assignment: Randomization was clustered by site and by year, creating 32 clusters in total. Each site was assigned to the intervention condition for two of the years and to the control condition for the other two years. The intervention sites received TOP programming plus the text-message-based supplement, and control sites received just the TOP programming. The sample included 436 in the intervention condition and 315 in the control condition at baseline.

      Attrition: Participants completed self-report measures at baseline and immediately at program completion. At posttest, 72.7% of the intervention group and 75.7% of the control group completed the assessment.

      Sample:

      The clubs primarily served minority youth and those living in poverty. The mean age was 14.90 (SD = 1.02) in the intervention condition and 14.98 (SD = 1.14) in the control condition. The intervention condition was 42.41% Hispanic, 20.25% black, and 9.18% white, and the control condition was 43.81% Hispanic, 18.73% black, and 7.94% white.

      Measures:

      Measures were administered through self-reports. Participants reported how many times they had had sex in the previous three months and how many of these times were without protection. They also reported whether they had “accessed contraceptive or sexually transmitted infection services in the past 9 months” and whether they had gotten pregnant. The study reported no information on the validity and reliability of the measures.

      Analysis:

      Multilevel regression models were used to analyze the outcome variables, with a large number of covariates including the baseline outcomes. Sub-analyses compared Hispanic to non-Hispanic participants. Bonferroni-Hochner corrections were used to correct for multiple tests.

      Intent-to-Treat: The authors stated that subjects were used in the originally assigned conditions, regardless of the level of engagement with the program. Also, sensitivity analysis included missing data with full information maximum likelihood estimation and gave the same results.

      Outcomes

      Implementation Fidelity:

      No information on implementation fidelity was given, but the subjects attended an average of 11.9 sessions of the 25.

      Baseline Equivalence:

      Table 2 showed only 1 significant difference out of 22 tests, but the table used the analysis sample. The significant difference was for pregnancy.

      Differential Attrition:

      Differential attrition was tested by condition and by 22 baseline variables. Those remaining in the study reported higher rates of protected sex and living with both parents and lower likelihood of having sex. Additionally, intervention participants lost to follow-up were significantly older than those who remained in the study. However, tests for the interaction of the baseline variables and condition showed that the differential attrition was similar across conditions. The tests for baseline equivalence using the analysis sample also suggest a limited influence of attrition.

      Posttest:

      The program showed no significant effects for the full sample. Hispanic participants in the intervention condition reported significantly fewer pregnancies than those in the intervention condition, but the authors noted that this effect may be influenced by the significant baseline difference between conditions for pregnancy.

      Long-Term:

      No long-term effects were reported.