Check and Connect

Study: Maynard, Kjellstrand, & Thompson (2014)

Study Type: Group-Design

Participants: Unconvincing Evidence

Sample size: 189 students (89 program, 100 control)

Risk Status: All students met one or more of the criteria on the Texas Education Agency at-risk eligibility list or were referred for family crisis. The researchers did not specify whether these students had an IEP plan.

Demographics:

 

Program

Control

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

       

  Grade 1

       

  Grade 2

 

 

 

 

  Grade 3

 

 

 

 

  Grade 4

 

 

 

 

  Grade 5

 

 

 

 

  Grade 6

17

19%

20

20%

  Grade 7

13

15%

15

15%

  Grade 8

21

24%

19

19%

  Grade 9

19

21%

24

24%

  Grade 10

5

6%

8

8%

  Grade 11

7

8%

5

5%

  Grade 12

7

8%

9

9%

Mean Age

 

 

 

 

Race-ethnicity

  African-American

13

15%

7

7%

  American Indian

 

 

 

 

  Asian/Pacific Islander

 

 

 

 

  Hispanic

76

85%

93

93%

  White

 

 

 

 

  Other

 

 

 

 

Socioeconomic status

  Subsidized lunch

64

72%

76

76%

  No subsidized lunch

25

28%

24

24%

Disability status

  Speech-language impairments

       

  Learning disabilities

       

 Emotional disturbance

       

 Intellectual disability

       

  Other

 

 

 

 

  Not identified with a disability

 

 

 

 

ELL status

  English language learner

 

 

 

 

  Not English language learner

 

 

 

 

Gender

  Female

 

 

 

 

  Male

 

 

 

 

Training of Instructors: Mentors are CIS site coordinators who had been employed with CIS for a mean of 4.38 years who had a background in the field of psychology, counseling, or social work. 64% of the mentors had a master’s degree and 36% held a bachelor degree. All mentors received a full-day training on the C&C intervention, which consisted of didactic components and role-playing. Additionally, interventionists were also provided with a half-day booster training.

Design: Convincing Evidence

Did the study use random assignment?: Yes

If not, was it a tenable quasi-experiment?: Not applicable

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes? Yes

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Not applicable

Were the program and control groups demographically comparable?: Yes

Was there differential attrition for the program and the control groups?: No

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes

Implemented with Fidelity: Unconvincing Evidence

Description of when and how fidelity of treatment information was obtained: Fidelity was not measured. However, it was closely monitored over the course of the study to ensure the intervention was being implemented as intended. The researchers also noted that Check and Connect is a well-specified intervention with an intervention manual that provides for standardization, reduces variability of implementation, and provides sufficient information for the intervention to be replicated and compared across the studies, thus enhancing internal and external validity. In addition, the Check & Connect monitors in this study were trained and monitored to enhance the adherence of the implementation, reduce implementer drift, and connect any deviations from the intervention in real time over the course of the study.

Results on the fidelity of treatment implementation measure: Quantitative information on the fidelity of treatment implementation was not reported.

Measures Targeted: Partially Convincing Evidence

Targeted Measure

Reliability statistics

Relevance to program focus

Exposure to related support among control group

Academic performance (a composite score)

Cronbach’s alpha=.73

Directly related to objectives of study

Received Communities in Schools intervention only (no Check & Connect)

Total number of office referrals

NA

Directly related to objectives of study

Received Communities in Schools intervention only (no Check & Connect)

Attendance

NA

Directly related to objectives of study

Received Communities in Schools intervention only (no Check & Connect)

 

Broader Measure

Reliability statistics

Relevance to program focus

Exposure to related support among control group

N/A

 

 

 

 

Mean ES Targeted Outcomes: Data Unavailable

Mean ES Administrative Outcomes: N/A

Effect Size:

Targeted Measures

Measure

Effect Size

Academic performance (composite score)

Total number of office referral

Attendance

 

Broader Measures

Measure

Effect Size

None

 

 

Key

*        p ≤ .05

**      p ≤ .01

***    p ≤ .001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

u      Effect size is based on unadjusted means

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single-Subject Designs): N/A

Disaggregated Outcome Data Available for Demographic Subgroups: No

Target Behavior(s): Externalizing, Internalizing

Delivery: Individual

Fidelity of Implementation Check List Available: No

Minimum Interventionist Requirements: Paraprofessional, 1-2 days in-depth training required

Intervention Reviewed by What Works Clearinghouse: Yes – Intervention

What Works Clearinghouse Review

Dropout Prevention Protocol

Effectiveness: Check & Connect was found to have positive effects on staying in school, potentially positive effects on progressing in school, and no discernible effects on completing school for high school students with learning, behavioral, or emotional disabilities.

Studies Reviewed: 2 studies meet standards out of 3 studies total

Full Report

Other Research: Potentially Eligible for NCII Review: 4 studies

Maynard, B. R., Kjellstrand, E. K., & Thompson, A. M. (2013). Effects of Check and Connect on Attendance, Behavior, and Academics: A Randomized Effectiveness Trial. Research on Social Work Practice, 00(0)1-14.

Sinclair, M. F., Christenson, S. L., Evelo, D. L., & Hurley, C.M. (1998). Dropout Prevention for Youth with Disabilities: Efficacy of a Sustained School Engagement Procedure. Exceptional Children, 65(1), 7-21.

Sinclair, M. F., Christenson, S. L., & Thurlow, M. L. (2005). Promoting School Completion of Urban Secondary Youth with Emotional or Behavioral Disabilities. Exceptional Children, 71(4), 465-482.

Strand, P. S., & Lovrich, N. P. (2014). Graduation Outcomes for Truant Students: An Evaluation of a School-Based, Court-Engaged Community Truancy Board with Case Management. Children and Youth Services Review, 43, 138-144.