Responsive Reading Instruction
Study: Denton et al. (2010)

Describe how students were selected to participate in the study:

All first grade students at each participating school were screened to identify a pool of students at risk for serious reading difficulties. After identifying a pool of eligible at risk participants at each school, the researchers randomly selected 6 to 26 students within each school and randomly assigned them to one of two groups: RRI or typical school practice (TSP).

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

In a multistep screening process, the researchers assessed students with (a) letter sound, blending phonemes, and word reading screens from the Texas Primary Reading Inventory (TPRI); (b) a word reading task derived from the WJ-III Letter Word Identification subtest; and (c) a first grade oral reading fluency passage. In the first year of the study, students were eligible for inclusion if they failed the TPRI screen measures, had a score of 8 or less on the word-reading measure, and had oral reading fluency rates of 8 correct words per minute or less. In Year 2, the cutoffs for word reading and oral reading fluency were lowered to scores of 3 and 5 respectively. The researchers excluded students from the sample who received their primary reading instruction in a language other than English, those who had severe intellectual disabilities or emotional disturbance, and those with developmental disabilities such as autism. Students were included with mild to moderate disabilities who received their primary reading instruction in general education classrooms.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Responsive Reading Instruction (RRI)

Specify which condition is the control condition:

Typical School Practice (TSP)

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Systematic

Please describe the assignment method or the process for defining treatment/comparison groups.

After identifying the pool of eligible at-risk students at each school, we randomly selected 6 to 26 students within each school and randomly assigned them to one of two groups: RRI or typical school practice (TSP). The range in number of students assigned from school to school was due to differing numbers of at-risk first-grade students and available RRI teachers. In all, 507 students were randomized to the study conditions, 222 to RRI and 285 to TSR During implementation, 40 RRI (18%) and 45 TSP (16%) students left the study; the large majority attended the four schools that withdrew from the study. A chisquare analysis indicated no significant differences in levels of attrition between groups. Within both the RRI and TSP groups, students who left the study did not differ significantly from those who completed it on pretest word reading ability (see Table 1). After attrition, 182 RRI students and 240 TSP students had complete data and were included in the analysis.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

The principal at each school selected one or more certified teachers who were currently employed at the school to provide RRI to at least one group of first grade students each day, for a total of 40 teachers over the 2-year study. The interventionists had a variety of primary job assignments on their campuses with 31 full-time reading teachers, 2 classroom teachers, 2 librarians, 1 special educator, 1 physical education coach who had never taught reading, and 1 educational assistant (i.e., paraprofessional) with teaching credentials. All but one of the teachers held elementary education teaching credentials, and several were certified in other areas as well, including English as a second language/bilingual education (15), early childhood education (13), reading specialist/master reading teacher (13), special education (12), and secondary education (8). Teachers’ total years of experience as professional educators ranged from 2 to 31 years (M = 17.15; SD = 8.38), and they had from 0 to 21 years of experience as reading interventionists. (M = 4.93; SD = 5.12). Seven teachers had previous training in an explicit phonics-emphasis intervention, and 15 had previously been trained in the Reading Recovery program. Teachers participated in about 18 hours of professional development in RRI over 3 days at the beginning of the school year provided by the program developers and one experienced RRI teacher. It focused on (a) the research basis for the program, (b) RRI instructional procedures, and (c) the use of assessment results in lesson planning, and included videotaped modeling and mock teaching sessions to practice procedures. In Year 2, continuing teachers received about 6 hours (1 day) of professional development at the beginning of the school year. In each of the 2 years, all teachers participated in 12 hours of follow-up professional development over 2 days—one in late fall and another in early spring. Two experienced former RRI teachers provided coaching to some of the intervention teachers in the study. Each RRI teacher in the study had been randomly assigned to receive support in one of the three following formats: (a) traditional on-site coaching (n = 15), (b) technology-based coaching (n = 13), and (c) on-demand coaching in which teachers could contact a coach with questions or request visits from a coach but rarely did (n = 12) as part of a separate study. (See page 402)

Describe when and how fidelity of treatment information was obtained.

Two RRI coaches and one project coordinator observed intervention teachers and students for one entire teaching session, approximately once every 9 weeks for a total of three fidelity observations per teacher per year using a fidelity measure closely aligned with the intervention. The coaches did not observe their own teachers. Teachers did not receive feedback following these fidelity observations. Inter-observer reliability was established by co-viewing and independently rating lessons in schools, a procedure repeated within each cycle of data collection. Training of observers continued until they reached at least 80% agreement with program developers. Fidelity was rated on a 4 point Likert scale from 0 to 3 for each activity observed within each of the five components of a lesson. For each component, observers rated items addressing three domains: a) adherence to program procedures, (b) implementation quality, and (c) students’ on-task behaviors during the intervention.

What were the results on the fidelity-of-treatment implementation measure?

On average, teachers implemented RRI with adequate fidelity, although there was variation across teachers. The mean overall fidelity rating for all RRI teachers across all observations was 80.14% (SD = 8.43%; range, 57.33%--94.58%). Mean ratings (using the 0 to 3 scale with 0 meaning the element was expected but not observed and 3 being the highest rating) across all teachers for each of the three domains were 2.08 (SD = 0.47) for adherence to the RRI procedures; 2.26 (SD = 0.38) for quality of implementation, and 2.61 (SD = 0.22) for students’ on-task behavior.

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Group differences were modeled with multilevel techniques, where students were nested within school and measurement occasions were nested within students in schools as appropriate. (See page 405). For pretest-posttest measures, differences between groups were estimated by analyzing two-level hierarchical linear models, where students were nested within schools. For the ORF continuous progress monitoring measure, differences between groups were estimated by analyzing three-level linear growth models where measurement occasions were nested within students in schools. By centering the intercept at the last measurement occasion, the researchers were able to estimate the extent to which groups differed at the last phase of assessment. Group (RRI, TSP) was included as a student level variable. Covariates included Cohort because the study was conducted over 2 years with 2 different cohorts of students, Ethnicity and Gender. Each student measurement outcome was tested separately. (See page 405)