READ 180 Universal
Study: Kim et al. (2011)

Describe how students were selected to participate in the study:

The study recruited students from 4 elementary schools from grades 4 to 6 who scored below proficient on the Spring 2006 administration of the Massachusetts Comprehensive Assessment System (MCAS) in English Language Arts. A small percentage of students (5%) scoring above proficient were recruited to each group—treatment and control. Students with required to have active consent forms on file from a parent / guardian, then students were stratified by school and grade, and then randomly assigned to either READ 180 or the regular district after-school program.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

95% of students scored below proficient on the Massachusetts Comprehensive Assessment System (MCAS) in English Language Arts.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

READ 180, Enterprise edition, the comprehensive reading intervention program is considered the treatment condition, and the submitted program of this study.

Specify which condition is the control condition:

The control condition is a semi-structured enrichment program with little formal literacy instruction. Control teachers were encouraged to develop their own activities or choose from 16 activities, including informal arts projects, math games, and commercially developed math and literacy themed kits.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

A power analysis was conducted to determine the sample size needed to detect an effect size of .20, using a two-tailed test with alpha of .05. The results indicated that approximately 300 students were required to have an 80% chance of detecting an effect size of .20, using a covariate correlated to .70 to .80 with the posttest. Students were then randomly assigned to each group—treatment and control.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

READ 180 teachers received both pre-service, and in-service professional development.

Describe when and how fidelity of treatment information was obtained.

Trained observers conducted 3 observations (November, February, April) of the after-school program, along seven intervention components. Each component was rated on a 3-point scale, 1 = low fidelity and 3 = high fidelity.

What were the results on the fidelity-of-treatment implementation measure?

The mean fidelity score was 2.21, and scores for each individual component were: • Classroom Set-up, M=2.64 • Computers, M=2.65 • Headsets, M=2.71 • Computer rotation, M=2.16 • Small group rotation, M=1.94 • Independent reading rotation, M=2.29 • Whole group lessons, M=1.11

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Researchers conducted two separate analyses: 1) an ordinary least squares regression analysis to generate estimates to measure the impact of READ 180 regardless of dosage (number of days students attended the program) on students’ posttest reading scores; and 2) an instrumental variables analysis to determine whether treatment effects were larger for students with higher attendance rates compared to students with low attendance.