Enhanced Core Reading Instruction
Study: Fien et al. (2015); Smith et al. (2016)

Describe how students were selected to participate in the study:

Project ECRI was a cluster-randomized controlled trial that nested students and teachers within schools. In the first wave of the study (2009-2010), 18 schools in three school districts were randomly assigned to the treatment, 9 schools in each condition. Of these schools, two left after random assignment (1 treatment and 1 comparison). In the second wave of the study (2010-2011), 28 schools in six school districts were randomly assigned to the treatment, 14 schools in each condition.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Fall scores on the reading portion of the Stanford Achievement Test, 10th edition (SAT10) were used to assign students to Tier 1, Tier 2, or Tier 3 using the 2007 normative comparison. Students that scored between the 10th and 30th percentile on the SAT10 in the fall of first grade were assigned to Tier 2. Students that scored above the 30th percentile on the SAT10 or below the 10th percentile were assigned to Tier 1 and Tier 3, respectively.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

N/A

Specify which condition is the control condition:

N/A

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

In the first project wave, we recruited 22 schools in three Oregon school districts to participate in Project ECRI. Four of these schools elected to not participate in the study due to changes in school leadership between recruitment and the beginning of the study. The remaining 18 schools were randomly assigned to the treatment or comparison condition. We blocked on district before random assignment to control for core curricula and other important factors, ensuring similar schools in each condition. Because there was one district with only one participating school, that school joined another district for randomization. After random assignment, two schools participating in Wave 1 (one treatment and one comparison school) left the project, leaving 16 schools participating in Wave 1. In the second project wave, we recruited 20 schools in three districts in Oregon and eight schools in three districts in Massachusetts to participate and randomly assigned all schools to conditions. All recruited Wave 2 schools participated in the study, thus, the combined wave sample included 46 schools.

What was the unit of assignment?

Schools

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Participating teachers reported 13.49 total years of teaching experience, on average (M = 12.18 years for treatment teachers; M = 14.73 years for comparison teachers). Teachers in the treatment condition participated in three days of professional development on the ECRI instructional model prior to the beginning of the school year, and two days of follow-up activities in October. Professional development for teachers emphasized (a) instructional teaching routines; (b) overview of research on beginning reading content and skills including phonemic awareness, phonics, vocabulary, comprehension, and fluency in reading connected text; and (c) strategies for increasing student engagement in lessons. Small group instructors received two days of professional development in the fall and one day in January. Professional development for small group instructors emphasized instructional teaching routines and strategies for increasing student engagement in lessons. All professional development included teaching demonstrations by ECRI expert coaches and participant practice with coach feedback. Teachers and small group instructors also received comprehensive coaching support through classroom and small group visits once per month and regular study group meetings, facilitated by an ECRI coach.

Describe when and how fidelity of treatment information was obtained.

Observations were conducted during core reading instruction in all treatment and comparison classrooms in the fall, winter, and spring. Unless schools reported classrooms were providing less than 90 minutes of core reading instruction, all observations were scheduled for at least 90 minutes. Observations included whole group and small group instruction during the core reading block. The Quality of Explicit Instruction observation (Nelson-Walker et al., in press) was used to measure fidelity of implementation of the ECRI intervention and the quality of explicit reading instruction in first grade classrooms.

What were the results on the fidelity-of-treatment implementation measure?

Observations of implementation fidelity conducted by trained data collectors indicated nearly all treatment teachers used lesson maps (M = 0.90) and instructional templates (M = 0.89) during instruction, and instruction in treatment classrooms was consistent with the intent of the lesson maps and templates to improve the quality of explicit instruction (M = 0.77).

Was the fidelity measure also used in control classrooms?

As expected, observations conducted in comparison classrooms indicated comparison teachers rarely used lesson maps (M = .09) and instructional templates (M = .07) during instruction. Although treatment diffusion was minimal, some comparison teachers did have access to intervention materials.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

We assessed intervention effects on each of the primary outcomes with a mixed-model (multilevel) time × condition analysis (Murray, 1998) to account for students nested within schools, the level of random assignment. The analysis tests net differences between conditions on change in outcomes from the fall (T1) to spring (T2) of Grade 1, with gains for individual students clustered within schools. The test of net differences provides an unbiased and straightforward interpretation of the results (Cribbie & Jamieson, 2000, Fitzmaurice, Laird, & Ware, 2004), opposed to covariate adjusted scores, which can be ambiguous. The statistical model includes time, condition, and the time × condition interaction, with time coded 0 at T1 and 1 at T2 and condition coded 0 for control and 1 for ECRI. The time × condition interaction estimate provides for the test of condition effects. With 44 schools, tests of time × condition used 42 degrees of freedom. Primary analyses included the students in each school that scored between the 16th and 39th percentile on the SAT10 in the fall of first grade. This nested time × condition analysis accounts the intraclass correlation associated with multiple students nested within the same schools.