Responsive Reading Instruction
Study: Mathes et al. (2005)

Describe how students were selected to participate in the study:

During each of two years, we identified within these schools a sample of first graders who showed significant risk for reading difficulties. In order to determine which students were at risk for reading difficulty, classroom teachers and our research team screened all students at the end of kindergarten within the six participating schools using the kindergarten screening portion of the Texas Primary Reading Inventory (TPRI). At the beginning of the first-grade year we screened any students entering the school for the first time with the first-grade TPRI screen.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

(A) students were screened with the TPRI at the end of Kindergarten or the beginning of 1st grade; (B) the Woodcock Johnson III Word Identification subtest (any student who could read ≥ 5 words was eliminated) and Observation Survey of Early Literacy Achievement text reading subtest (any student who could read Level D texts or higher with 90% accuracy was eliminated) was administered; (C) 1 minute, end-of-1st grade-leveled Oral Reading Fluency passage was administered (any student who was reading > 5 words was eliminated from the sample). This multiple part identification of at-risk students was also used in the second study included in this protocol, Effectiveness of a Supplemental Early Reading Intervention Scaled Up in Multiple Schools by Denton, et al. (2010).

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Responsive Reading Instruction (RRI)

Specify which condition is the control condition:

Enhanced Classroom Instruction (EC)

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Proactive Reading

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Once identified, all students designated as at risk within a school were randomly assigned to one of three conditions: enhanced classroom + Proactive Reading, enhanced classroom + Responsive Reading, or enhanced classroom only. In addition, we identified a sample of typically achieving readers by randomly selecting them from among all students in the same classrooms who evidenced no risk for reading problems. The purpose of this typically achieving group was to provide a benchmark of typical reading development in these classrooms.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Background and Experience: bachelors and masters degrees, teaching certifications in multiple areas, 2 of the 6 were certified reading specialists. Mean years of experience = 9 years, ranging from 3 to 22 years (includes teachers not under review). Training and Ongoing Support: 42 hours of professional development from the author of the intervention prior to instruction (Year 1) and an additional 12 hours in Year 2 of the study. Monthly half-day in-service meetings during the school year, onsite coaching. Although the vendor does not directly require or provide training for implementation of Responsive Reading Instruction, the program developer strongly recommends training. Training is available from the program developer in the form of a 2-day initial professional development prior to the provision of intervention with two one-day follow-up sessions to be provided later in the school year. Each of these days comprises about 6 hours of training. The developer also strongly recommends some form of ongoing coaching support for teachers who provide RRI.

Describe when and how fidelity of treatment information was obtained.

Classroom observations were conducted every 8 weeks, for a total of four observations for each participating teacher. A 3-point rating scale was used (3 = exact, 1 = poor). Prior to each wave of data collection, two or more observers established reliability by co-observing lessons, rating them independently, and comparing and discussing the ratings during a consensus meeting. Reliability was calculated by comparing the ratio of absolute agreements and disagreements. These co-observations continued until agreement between raters reached at least 85%. This was repeated at each wave of data collection (4 times each year).

What were the results on the fidelity-of-treatment implementation measure?

Having materials ready 96% of the time; having students seated 96% of the time, being warm and enthusiastic 100% of the time. They were rated as having appropriate pacing (mean = 2.5), following procedures for each section of the lesson routine (mean = 2.6), scaffolding student responses appropriately (mean = 2.7), and maintaining student attentiveness mean = 2.7.

Was the fidelity measure also used in control classrooms?

The control classrooms were observed to better understand the nature of instruction provided to students in this condition. The same measure of fidelity was not used. See page 156, Enhanced classroom instruction section. Thirty teachers were observed 90 times. Observers reported the classroom was stimulating and motivating for 72.62% of the observations and that children were encouraged to express their ideas verbally in 90.27% of the observations. In terms of instructional content, instruction was provided in phonemic awareness during 34.65% of the observations, letter-sound correspondences in isolation during 72.72% of the observations, practice in reading words by sounding out during 82.36% of the observations, and decoding words using visual memory or context in 40.53% of the observations. Comprehension strategies instruction occurred in only 15.09% of the observations, vocabulary was presented in 83.75% of the observations, spelling was taught in 70.48% of the observations, and writing was included in 54.76% of the observations. While little comprehension instruction was provided, teachers frequently assessed students’ comprehension of text, asking literal questions in 83.53% of the observations and inferential questions in 61.60% of the observations.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Hierarchical Linear Modeling of both growth rate and end-of-year outcomes. Analyses took into account the shared variance within each student over repeated assessment times, and the shared variance from classrooms and schools inherent in the nested data structure. Analyses were able to effectively model individual student growth rates (slope estimates), and outcomes at the end of the year (intercepts). Finally, effect size estimates using HLM parameters are reported that compare the treatment and control group on growth rates and end of year outcomes. See pages 161-163 for further details on the analytical approach.