Please take our brief survey

Blueprints Programs = Positive Youth Development

Return to Search Results

Promising Program Seal

KEEP SAFE

Blueprints Program Rating: Promising

A six-session group-based intervention, facilitated by paraprofessionals, for youth in foster care as they transition to middle school to prevent internalizing and externalizing problems that may lead to more serious longer term outcomes such as delinquency, substance use, and high-risk sexual behavior. Foster parents also attend a six-session program.

  • Patricia Chamberlain and Leslie Leve
  • Oregon Social Learning Center
  • 10 Shelton McMurphey Boulevard
  • Eugene, OR 97401-4928
  • US
  • (541) 485-2711
  • pattic@oslc.org
  • lesliel@oslc.org
  • Illicit Drug Use
  • Positive Social/Prosocial Behavior
  • Sexual Risk Behaviors
  • Tobacco

    Program Type

    • Foster Care and Family Prevention
    • Parent Training
    • Skills Training

    Program Setting

    • Social Services

    Continuum of Intervention

    • Selective Prevention (Elevated Risk)

    A six-session group-based intervention, facilitated by paraprofessionals, for youth in foster care as they transition to middle school to prevent internalizing and externalizing problems that may lead to more serious longer term outcomes such as delinquency, substance use, and high-risk sexual behavior. Foster parents also attend a six-session program.

      Population Demographics

      KEEP SAFE is intended for middle school youth (ages 11-14) in foster care. The program has been evaluated with girls in foster care who have just left elementary school and are beginning middle school in the coming months. An ongoing study is evaluating the program for boys.

      Age

      • Early Adolescence (12-14) - Middle School

      Gender

      • Female only

      Race/Ethnicity

      • All Race/Ethnicity

      Race/Ethnicity/Gender Details

      Although a program for foster-care boys and girls, the KEEF SAFE program was evaluated with foster care girls. In the girls-only trial, the race distribution of the sample was 63% White, 10% Latino, 9% African American, 4% Native American, and 14% multiracial. The analysis did not examine results by race and ethnicity.

      Risk factors for developing internalizing and externalizing problems are especially acute for children with foster care histories because they are more likely to have been subject to childhood maltreatment. In addition, foster care girls are at increased risk compared to foster care boys because they have significantly higher rates of childhood abuse than boys and high rates of co-occurring physical abuse. Girls in foster care typically come from families with higher levels of stress and criminality than boys in foster care.

      Compared to non-foster girls, foster girls are more likely to experience an unstable rearing environment, which is associated with early pubertal onset, and early pubescence is associated with internalizing and externalizing behavior problems.

      • Individual
      • Peer
      • Family
      Risk Factors
      • Individual: Stress
      • Peer: Interaction with antisocial peers
      • Family: Family conflict/violence, Family history of problem behavior, Neglectful parenting, Parent stress, Poor family management
      Protective Factors
      • Individual: Problem solving skills, Prosocial involvement, Rewards for prosocial involvement, Skills for social interaction
      • Peer: Interaction with prosocial peers
      • Family: Opportunities for prosocial involvement with parents, Rewards for prosocial involvement with parents

      The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks during the summer. Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents. Sessions for foster parents focus on maintaining stability in the home, preparing the youth for middle school, and developing behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home. Continued sessions (i.e., ongoing training and support) are provided to foster parents (group-based) and youth (one-on-one sessions) once a week for two hours during the first year of middle school.

      The KEEP SAFE (Middle School) program begins during the summer prior to middle school entry and consists of two parallel components (both led by paraprofessionals): a six-session group-based intervention for the foster-care youth and a six-session, group-based intervention for the foster parents. The groups meet twice weekly for 3 weeks, with approximately 7 people per group.

      The parent sessions are led by a facilitator and a co-facilitator, each of whom are experienced foster parents with bachelor's degrees. Sessions are also often led by child welfare case workers and case work supervisors in “real world” implementation formats. The youth groups are led by a facilitator and three assistants. These individuals are supervised by a masters or doctoral-level clinician. Using paraprofessionals as the primary facilitators reduces the costs of the program dramatically.

      Continued sessions (i.e. ongoing training and support) are provided to foster parents and youth (one-on-one sessions) once a week for two hours during the first year of middle school.

      The content of the program is as follows:

      Focus for youth: Facilitators teach youth about setting goals, establishing positive relationships with peers and adults, building confidence, and developing decision-making skills. Problem-solving skills and opportunities are also provided so they can practice positive behaviors. A ceremony is conducted at the end of the summer in which the youth proclaim their goals and commitments to each other and their foster parents.

      Focus for foster parents: These sessions are primarily concerned with enabling foster parents to maintain stability in the home, prepare the youth for middle school, and develop behavioral reinforcement techniques and realistic expectations. Parents are given homework assignments designed to encourage them to practice their new skills at home.

      The developers based the program on "developmental theories and intervention work with at-risk youth" and uses a behavioral model of change, emphasizing reinforcement and contingency management.

      • Behavioral
      • Skill Oriented

      This randomized controlled trial of the program was conducted with girls in foster care from two counties (one urban, one rural) in Oregon. Researchers recruited foster girls who were finishing elementary school between 2004 and 2007. They were randomized into one of two groups – the KEEP SAFE (Middle School) intervention (n=48) or the control group (n=52), which received standard services from local child welfare organizations.

      Data were collected at five points in time: baseline, six months, 12 months (program completion), and one and two years post-intervention. A five- to seven-year post-intervention assessment is ongoing.

      The effects of the program on internalizing, externalizing, and prosocial behavior were examined at six months, as well as pathways through which the program may have affected delinquency, health-risking sexual behavior, foster-care placement stability, and substance use over the course of two years after program completion.

      At the six-month mid-program assessment, the KEEP SAFE (Middle School) program for foster girls was significantly related to improvements in internalizing and externalizing problem behaviors relative to the foster girls in the control group.

      At 12 months after baseline, the program significantly reduced placement changes and increased prosocial behavior.

      At 36 months after baseline, or two years after program completion, the intervention foster girls reported moderately lower levels of substance use (Cohen's d = -.47) than girls in the control group. The intervention also affected substance use via prosocial behavior and internalizing and externalizing behaviors. The program did not have a direct effect on delinquency, but indirectly affected it via prosocial behavior and internalizing and externalizing behaviors.

      Also at 36 months after baseline, or two years after program completion, the intervention foster girls reported significantly lower tobacco use, marijuana use, and health-risking sexual behavior. The reduction in rates of health-risking sexual behavior was mediated by the intervention-driven reduction in substance use (tobacco and marijuana use).

      Intervention girls, relative to control girls, demonstrated significant effects for:

      • Internalizing and externalizing problem behaviors at six months.
      • Prosocial behavior combined across the 6-month and 12-month assessments.
      • Placement stability at the 12-month posttest.
      • Substance use at 36 months beyond baseline (2 years post-intervention).
      • Health-risking sexual behavior at 36 months beyond baseline (2 years post-intervention).
      • Small to moderate mediating effects of prosocial, internalizing, and externalizing behaviors on long-term outcomes of substance use.
      • Small to moderate mediating effect of tobacco and marijuana use on long-term health-risking sexual behavior.

      The researchers found that there were small-to-moderate mediating effects of prosocial, internalizing, and externalizing behaviors on the long-term outcomes of substance use (beta = -.04) and delinquency (beta = -.07), and small-to-moderate mediating effects of tobacco and marijuana use on health-risking sexual behavior (beta = -.12).

      Effect sizes were generally moderate at the two-year follow-up, with Cohen’s d ranging from .45 to .50 for significant outcomes and with correlations of the intervention and outcomes ranging from -.22 to -.28.

      These studies are generalizable to urban and rural foster girls entering middle school in the northwestern U.S.

      Limitations of studies (Smith et al., 2011; Kim & Leve, 2011; Kim et al., 2013) included:

      • Models for long-term outcomes of substance use, delinquency, and sexual behavior lacked controls for baseline outcomes.
      • Results were typically reported at six months and 36 months post-baseline. There were no reports of posttest and one year posttest results for substance use, delinquency, and sexual behavior.
      • Although the measure for externalizing and internalizing at 6 months is a composite measure of caregivers and parents, the measurement at 2 years post intervention is by caregiver report only. Since the caregivers are direct recipients of the program, just as the girls, the reporting using this measure could be biased.

      • Blueprints: Promising

      Britany Binkowski
      Assistant to the Commissioner
      Office of Child Welfare Reform
      UBS Tower, 10th Floor
      315 Deaderick St., Nashville, TN 37243
      o. 615-741-1405
      c. 615-708-9084
      Britany.Binkowski@tn.gov

      Sylvia Rowlands
      Senior Vice-President
      Evidence-Based Programs
      590 Avenue of the Americas
      New York, New York 10011
      (212) 660-1342
      sylvia.rowlands@NYFoundling.org

      Smith, D., Leve, L., Chamberlain, P. (2011). Preventing internalizing and externalizing problems in girls in foster care as they enter middle school: Immediate impact of an intervention. Prevention Science, 12(3), 269-277.

      Kim, H. & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.

      Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse, 22(5), 370-387.

      Patricia Chamberlain
      Oregon Social Learning Center
      10 Shelton McMurphey Boulevard
      Eugene, OR 97401-4928

      Study 1

      Kim, H. & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79(6), 740-750.

      Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse, 22(5), 370-387.

      Smith, D., Leve, L., & Chamberlain, P. (2011). Preventing internalizing and externalizing problems in girls in foster care as they enter middle school: Immediate impact of an intervention. Prevention Science, 12(3), 269-277.

      Smith, D., Leve, L., & Chamberlain, P. (2011). Preventing internalizing and externalizing problems in girls in foster care as they enter middle school: Immediate impact of an intervention. Prevention Science, 12 (3), 269-277.

      Kim, H. & Leve, L. (2011). Substance use and delinquency among middle school girls in foster care: A three-year follow-up of a randomized controlled trial. Journal of Consulting and Clinical Psychology, 79 (6), 740-750.

      Kim, H. K., Pears, K. C., Leve, L. D., Chamberlain, P. C., & Smith, D. K. (2013). Intervention effects on health-risking sexual behavior among foster care girls: The role of placement disruption and substance use. Journal of Child and Adolescent Substance Abuse 22 (5), 370-387.

      Smith et al. (2011) examined internalizing, externalizing, and prosocial behavior about six months after baseline. Kim and Leve (2011) examined placement changes and prosocial behavior at posttest and substance use and delinquency two years after posttest. Kim et al. (in press) examined health-risking sexual behavior at two years after posttest. The latter two articles also examined several mediation models.

      Evaluation Methodology

      Design: This randomized controlled trial of the KEEP SAFE (Middle School) program was conducted with girls in foster care from two counties (one urban, one rural) in Oregon. Girls were eligible if they were finishing elementary school between 2004 and 2007 and were referred to foster care through the local child welfare system. Researchers recruited girls and their foster parents from the pool of eligible participants (N=145). Eligibility criteria required that girls not yet be reunited with their original family and that the foster parents provide consent. Recruitment occurred on a rolling basis and stopped when enrollment reached 100. The participants were randomly assigned to the intervention (n=48) and the control condition (n=52) which consisted of the usual services provided by the local child welfare organizations.

      After randomization the girls and their foster parents were assessed at baseline (T1) and at 6 months post baseline (T2). The girls were assessed again at 12-, 24-, and 36-months post baseline, which represent the periods shortly after the full program ended (T3), one year after the program ended (T4), and two years after program ended (T5), respectively. The baseline assessment consisted of a standardized interview and questionnaires for each girl and foster parent, an interview with the girl's caseworker, and the collection of child welfare records. The interviews lasted approximately two hours and were conducted in person by assessors who were blind to group assignments. The data collection instruments were designed to measure child and family characteristics, child behaviors, and parenting practices.

      The summer component of the program was implemented in the summer before commencement of middle school. The mean time between baseline (T1) and the 6-month follow-up (T2) was 147 days (s.d.= 45.6). When changes in placement/caregiver occurred between baseline and T2 (20% of sample), the girl was followed to her new placement and the girl's new caregiver was recruited to participate in the study. The authors did not report when the transitions to new caregivers occurred.

      Booster sessions took place during the first year of middle school. On average, parents attended ten weekly meetings between the completion of the summer session and the T2 assessment. The authors did not report on the specific frequency of the girls' attendance in follow-up sessions, but "participation rates mirrored those of their caregivers" (Kim and Leve, 2011).

      The baseline (T1) sample size for the intervention group was 48 and subsequent sample sizes were 48, 47, 44, and 45, respectively, for T2, T3, T4, and T5. Of the three subjects that were lost, one could not be located and two refused to participate. The baseline (T1) sample size for the control group was 52 and subsequent sample sizes were 50, 50, 48, 45, respectively, for T2, T3, T4, and T5. Of the seven subjects that were lost, three could not be located and four refused to participate. Thus, attrition was about 10% percent from baseline to the 2-year post-intervention follow-up.

      Sample Characteristics: The average age of the girls at baseline was 11.5 years. The ethnic distribution was 63% European American, 10% Latino, 9% African American, 4% Native American, and 14% multiracial. At baseline, the girls had been in foster care an average of 2.9 years and had experienced an average of 1.4 out-of-home placements. In this sample, 56% of the girls had at least one documented incident of physical abuse, 67% had at least one documented incident of sexual abuse, and 82% had at least one documented incident of physical or sexual abuse.

      Measures: Measures were divided into categories of short-term outcomes, long-term outcomes, and predictors.

      For short-term outcomes, the Parent Daily Report Checklist (PDR) was used to measure internalizing problems, externalizing problems, and prosocial behavior. This 34-item measure of child behavior problems was administered by telephone individually to foster parents and girls on three consecutive or closely spaced days. In each call, subjects were asked about the preceding 24-hours to improve reliability of the measure. The PDR has been shown to be both valid and reliable.

      • Internalizing problems: An internalizing problems composite was computed based on five of the PDR items. The scores were averaged across the three telephone calls at T2 (alpha=.72 for parents and .74 for girls). The parent and girl scores were significantly correlated and were thus combined into a composite internalizing problems score at 6 months. At two years, the report was by caregivers only.
      • Externalizing problems: An externalizing problems composite was computed based on 18 of the PDR items. The scores were averaged across the three telephone calls at T2 (alpha=.85 for parents and .81 for girls). The parent and girl scores were significantly correlated and were thus combined into a composite externalizing problems score at six months. At two years the report was by caregivers only.
      • Prosocial behavior: A prosocial behavior composite was computed based on 11 of the PDR items. The scores were averaged across the three telephone calls at T2 (alpha=.74 for parents and .75 for girls). The parent and girl scores were significantly correlated and were thus combined into a composite prosocial behavior score.

      Placement changes in foster care were measured from welfare system records as the sum from the start of the study through posttest. The number of placement changes ranged from 0 to 7 during this period, and the mean was .56.

      Long-term outcomes included substance use, delinquency, and health-risking sexual behavior.

      • Substance use: Three indicators were used to assess substance use at T5: How many times in the past year have you (a) smoked cigarettes or chewed tobacco; (b) drunk alcohol; and (c) used marijuana. The response scale ranged from 1 (never) through 9 (daily). Tobacco and marijuana use were combined to create a two-item composite score. The authors did not report on reliability or validity for this measure.
      • Delinquency: Delinquency was assessed with two indicators. First, girls' own delinquent behavior was measured with the 36 items from the general delinquency scale, which is a part of the Self-Report Delinquency Scale (SRD). Girls were asked how many times in the past year they had committed various acts (e.g., damaging or destroying property, stealing). The mean of the frequencies across the items was used to represent the level of delinquency. Internal reliability of this measure with this sample was alpha=.85. Second, the girls' association with delinquent peers was measured with 30 items from a modified version of the general delinquency scale from the SRD. Girls were asked how many of their friends were involved in delinquent acts (26 items scored 0="none" to 4="all") and how often their friends used alcohol, tobacco, and marijuana (4 items scored 0="none" to 4="a lot"). Because of different response scales for the first 26 items and the last 4 items, the two item sets were each standardized before combining to compute a mean value. The internal reliability of this scale was alpha=.96.
      • Health-risking sexual behavior: Eight items from the girls’ in-person interviews were used to assess health-risking sexual behavior at T5. The girls reported on items such as touching a boy’s body above or below the waist, having sexual intercourse, having sex with someone who they just met, or having sex with someone using drugs in the past 12 months. Positive answers to these items were totaled to represent the cumulative number of health-risking sexual behaviors. Internal reliability of the measure was .67.

      Predictor measures included:

      • Maltreatment History: The girls' cumulative maltreatment history at T1 was coded from child welfare case files using a modified version of the Maltreatment Classification System. Trained coders examined these case files to identify incidents of physical and sexual abuse. Two-thirds of the files were double-coded to compute interrater agreement. Agreement on the identification of the number of physical and sexual abuse incidents was high (85.7% and 86.2%, respectively). The maltreatment history variable consisted of the sum of the number of incidents prior to the study.
      • Pubertal Development: Pubertal development was measured at T1 using girl and foster parent reports on the Pederson Scales of Development. This measure has been shown to be reliable and valid. The Pederson Scales of Development uses a 4-point Likert scale to rate items such as body hair changes, skin changes, breast growth, and changes in height. Menstruation was measured as a dichotomous variable. A pubertal development score was computed using parent data and girl data. These scores were highly related and therefore averaged to create a mean pubertal development score.

      Analysis: For the 6-month assessment, Smith et al. (2011) ran separate stepwise hierarchical linear regression models, one for each T2 outcome (internalizing, externalizing, prosocial behaviors). The first step consisted of the general predictor variables (baseline age, maltreatment history, pubertal development, and behavior) and the second step included the intervention dummy. The study reported beta coefficients, but did not state that the coefficients were standardized (authors state in personal correspondence that coefficients were standardized).

      The two-year posttest follow-ups (Kim & Leve, 2011; Kim et al., in press) conducted structural equation modeling. To use the full intent-to-treat sample (n=100), full information maximum likelihood (FIML) estimation was used. This method has been shown to provide unbiased estimates when data are missing at random. Little's MCAR test indicated that the data were, in fact, missing completely at random. The composite tobacco and marijuana use variable was log transformed and the number of placement changes was square root transformed to more closely resemble normal distributions. Baseline outcomes of substance use, delinquency, and health-risking sexual behavior were not measured at baseline or used as predictors.

      Outcomes

      Fidelity: The interventionists were supervised weekly, and would regularly be shown videotapes of sessions and provided with feedback regarding adherence to the clinical model. The authors did not report how well the interventionists adhered to these guidelines. Caregivers attended 5.62 of the 6 summer sessions, and 20 of the 40 follow-up sessions during the year offered, on average. Participation rates of girls in the summer sessions mirrored those of their caregivers. Of the 40 follow-up sessions offered to youth, average attendance was 56.4%.

      Baseline equivalence: Kim and Leve (2011) found no significant differences on demographic characteristics, age at first placement, number of placements, foster care type, history of delinquency, history of special services, internalizing behavior, or externalizing behavior. Only the baseline measure of severity of neglect differed significantly, but it had no influence in any of the models.

      Differential attrition: The authors did not report on the characteristics of attriters vs. non-attriters, yet Little's MCAR test indicated that data were missing completely at random.

      Posttest: Smith et al. (2011) examined internalizing problems, externalizing problems, and prosocial behavior at about six months after baseline (conclusion of the summer component, but still during booster sessions occurring in the first middle school year). The program was significantly related to a decrease in internalizing (beta = -.28, p<.01) and externalizing (beta = -.21, p<.01) problems, but not significantly related to prosocial behavior.

      Kim and Leve (2011) examined placement changes, internalizing behavior, externalizing behavior, and prosocial behavior at 6-month and posttest combined. Girls in the intervention had significantly fewer placement changes (Cohen's d = .50) and significantly greater prosocial behavior (Cohen's d = .46). There were no significant differences on externalizing and internalizing symptoms (Cohen's d = .02).

      Long-term: In Kim and Leve (2011), the intervention group reported moderately lower levels of substance use (composite of tobacco, alcohol, and marijuana) than girls in the control group (Cohen's d = .47) at the two-year posttest follow-up. When the substances were examined separately, the program did not have a significant effect on alcohol use but did significantly affect tobacco use (Cohen's d = .45) and marijuana use (Cohen's d = .57). Delinquency (a composite of the girl's own delinquency and her association with delinquent peers) was marginally significantly lower for intervention girls.

      In Kim et al. (in press), girls from the intervention showed significantly lower levels health-risking sexual behavior (Cohen’s d = .48) than the control group. Post-hoc analysis indicated that only 4.4% of the girls in the program, as opposed to 17.8% of the girls in the control condition, reported having sexual intercourse in the past year (chi-square = 4.05, df = 1, p = .04).

      Mediation: First, the program had significant indirect effects (beta=-.04) on substance use through increased positive prosocial behavior at T2 and T3 and reduced internalizing and externalizing behaviors at T3 and T4. Second, the program did not have a direct effect on delinquency but did have an indirect effect (b=-.07) on delinquency through increased prosocial behavior at T2 and T3. Third, the program had a significant indirect effect (beta=-.12) on health-risking sexual behavior through reduced tobacco and marijuana use.