Reflex
Study: Sarrell (2014)

Describe how students were selected to participate in the study:

Students were invited to participate based on their being judged at-risk.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Students were identified based on being in pre-existing math RTI groups or math support classes. Those designations were based on failing the CRCT (Criterion Referenced Competency Test) along with a computerized math assessment. The CRCT is a statewide assessment used in Georgia to evaluate students' abilities relative to the Georgia Performance Standards)

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

100.0%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Users of Reflex

Specify which condition is the control condition:

Students who did not use Reflex

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Systematic

Please describe the assignment method or the process for defining treatment/comparison groups.

Existing, intact RTI groups were assigned to either treatment or comparison. Each grade level had its own treatment group and its own comparison group.

What was the unit of assignment?

Teachers

If other, please specify:

Please describe the unit of assignment:

At-risk students in the school were assigned to a number of "RTI groups," each of which was led by a highly qualified teacher. Several of these groups existed at each grade level. These RTI groups were assigned to either treatment or comparison.

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

Each student was tested individually.

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

All teachers for both groups were highly qualified teachers.

Describe when and how fidelity of treatment information was obtained.

All scheduled Reflex usage occurred during classroom time with monitoring teachers whose only requirement for the study was to ensure that the students were using the software.

What were the results on the fidelity-of-treatment implementation measure?

Was the fidelity measure also used in control classrooms?

No.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Dissertation author used an ANCOVA and then calculated an eta squared of 0.22 based on covariate-adjusted values. This is a large effect size, equivalent to a Cohen's d of 1.06 on covariate-adjusted data.