Check & Connect
Study: Maynard et al. (2014)

Describe how students were selected to participate in the study:

Students must have (1) not been previously enrolled in a dropout prevention program called Communities in Schools (CIS), a non-profit organization, (2) met one or more of the criteria on the Texas Education Agency at-risk eligibility list or referred for family crisis, (3) demonstrated absenteeism. These students came from schools that (1) had contracted with CIS to provide case management and dropout prevention services. In addition, the school was a either middle or high school and the school approved both the implementation of the C&C program.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

All students met one or more of the criteria on the Texas Education Agency at-risk eligibility list or referred for family crisis. The researchers did not specify whether these students had an IEP plan.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The treatment group received Check & Connect intervention, which consisted of Check and Connect components. The check component involves regularly monitoring student data related to risk indicators such as attendance. The connect component involves basic and intensive intervention based on students’ individual needs. The Check & Connect model is delivered by an adult “monitor” who uses a case management approach to work with students and their families during the intervention. The primary goal of the monitor is “to keep education a salient issue for the student, his or her family members, and teachers, and to reduce and prevent the occurrence of absenteeism, suspension, failing grades and other warning signs of school withdrawal” (Sinclaire et al., 1998, p.10). The monitor works with students and families on his or her case load to promote school management by building and maintaining relationships, monitoring student data related to alterable risk indicators, and implementing individualized interventions with students and families based on the data.

Specify which condition is the control condition:

Students in the control group were not exposed to C&C intervention.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

This study used a randomized block design. Eligible students were randomly assigned to the treatment or a control group.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Mentors are CIS sites coordinators who had been employed with CIS for a mean of 4.38 years who had a background in the field of psychology, counseling, or social work. 64% of the mentors had a master’s degree and 36% held a bachelor degree. All mentors received a full-day training on the C&C intervention, which consisted of didactic components and role-playing. Additionally, interventionists were also provided with a half-day booster training.

Describe when and how fidelity of treatment information was obtained.

Fidelity was not measured. However, it was closely monitored over the course of the study to ensure the intervention was being implemented as intended. The researchers also noted that Check and Connect is a well-specified intervention with an intervention manual that provides for standardization, reduces variability of implementation, and provides sufficient information for the intervention to be replicated and compared across the studies, thus enhancing internal and external validity. In addition, the Check & Connect monitors in this study were trained and monitored to enhance the adherence of the implementation, reduce implementer drift, and connect any deviations from the intervention in real time over the course of the study.

What were the results on the fidelity-of-treatment implementation measure?

Quantitative information on the fidelity of treatment implementation was not reported.

Was the fidelity measure also used in control classrooms?

Fidelity measure was not used in the control classrooms.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

A hierarchical linear modeling (HLM) was used for analysis to account for the fact that students are nested in schools. Note that the researchers did not reported adjusted and unadjusted means of posttreatment outcome data. However, the authors reported the β value as an indicator of effect size (see table 2 on page 8). For example, this study showed that Check & Connect was positively related to improvements in posttest student academic performance, β = 1.547 (p = .043), 95% CI [.047,3.048).