Responsive Reading Instruction

Study: Denton, Nimon, Mathes, Swanson, Kethley, et al. (2010)

Denton, C.A., Nimon, K., Mathes, P.G., Swanson, E.A., Kethley, C., Kutz, T.B., & Shih, M. (2010). Effectiveness of a supplemental early reading intervention scaled up in multiple schools. Exceptional Children, 76(4), 394-416.
Descriptive Information Usage Acquisition and Cost

Program Specifications and Requirements

Training

Responsive Reading Instruction (RRI) was created to help the lowest-performing first and second graders learn to read competently and within the average range for their age groups. RRI small-group lessons are based on six elements that research has shown are important for effective instruction of struggling readers:

  1. Instruction in Key Domains of Reading
  2. Explicit Instruction
  3. Opportunities to Practice
  4. Targeted Instruction Based on Assessment
  5. Scaffolding and Feedback
  6. The Home-School Connection

In the classroom, RRI teachers follow a consistent lesson structure that includes: Word Work, Print Concepts and Fluency, Assessment, Supported Reading, and Supported Writing. Optional DVD includes modeling of 34 of the RRI activities.

Responsive Reading Instruction is intended for use in first and second grades. It is designed for use with students with learning disabilities and any student at risk of academic failure. The academic area of focus is reading (including phonics/word study, comprehension, fluency, and vocabulary) and writing (including sentence construction, planning and revising, and writing in response to text).

Based on an analysis of purchased items, RRI has been implemented in approximately 50 locations since it was published in 2006. Implementations are primarily concentrated in Illinois and Texas.

Where to obtain:
Cambium Learning Sopris
4185 Salazar Way
Frederick, CO 80504
Phone: 800-547-6747
Website: www.soprislearning.com

Cost:
Initial cost per student for implementing program (2 options):

  • 5 Students & 1 Teacher: $19.59 per student (Includes teacher materials without DVD)
  • 5 Students & 1 Teacher: $24.70/student (Includes teachers materials with DVD)

Replacement cost per student for subsequent use: $ 0.00

Essentially, Responsive Reading Instruction comprises a single instructional manual/resource guide. This guide includes everything needed to begin instruction: embedded professional development, planning/implementation guides, and reproducible.

An optional component is a DVD that contains models of 34 RRI methods.

Manipulatives (e.g., sentence strips, sticky notes), big books, and leveled readers are not included.

Responsive Reading Instruction is designed for use with small groups of three to five students.

Responsive Reading Instruction takes 40 minutes per session with approximately five sessions per week for an unspecified number of weeks.

The program includes a highly specified teacher’s manual.

Technology is not required, but there is an optional DVD, which models instructional strategies, available for teachers.

Although training is not required, it is available if needed. Some districts have opted to have one day of initial training paired with on-going support (coaching/observations).

Instructors must be professionals. There are currently no training materials available; the professional development is embedded within the Teacher’s Guide.

Ongoing teacher support is not required, but is available.

 

Participants: Convincing Evidence

Sample size: 422 students from 31 elementary schools in 16 school districts in a southwestern state (507 were pretested; 182 in the treatment group and 240 in the control group). 

Risk Status: In a multistep screening process, the researchers assessed students with (a) letter sound, blending phonemes, and word reading screens from the Texas Primary Reading Inventory (TPRI); (b) a word reading task derived from the WJ-III Letter Word Identification subtest; and (c) a first grade oral reading fluency passage. In the first year of the study, students were eligible for inclusion if they failed the TPRI screen measures, had a score of 8 or less on the word-reading measure, and had oral reading fluency rates of 8 correct words per minute or less. In Year 2, the cutoffs for word reading and oral reading fluency were lowered to scores of 3 and 5 respectively. The researchers excluded students from the sample who received their primary reading instruction in a language other than English, those who had severe intellectual disabilities or emotional disturbance, and those with developmental disabilities such as autism. Students were included with mild to moderate disabilities who received their primary reading instruction in general education classrooms.

Demographics:

  Program Control p of chi square
Number Percentage Number Percentage

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

182

100%

240

100%

Not applicable

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Mean Age

         

Race-ethnicity

  African-American

33

18%

32

13%

Not significant

  American Indian

         

  Asian/Pacific Islander

         

  Hispanic

73

40%

109

46%

Not significant

  White*

76

42%

99

41%

Not significant

  Other

         

Socioeconomic status

  Subsidized lunch

 

 

 

 

 

  No subsidized lunch

 

 

 

 

 

Disability status

  Speech-language impairments

2

1%

4

1.6%

 

  Learning disabilities

1

0.5%

2

0.8%

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

1

0.5%

4

1.6%

 

  Not identified with a disability

178

98%

230

96%

Not significant

ELL status

  English language learner

9

7%

16

9%

Not significant

  Not English language learner

 

 

 

 

 

Gender

Female

84

46%

80

33%

p < 0.007

Male

98

54%

160

67%

p < 0.007

* The researchers combined the number of students classified as White and Asian

Training of Instructors: The principle at each school selected one or more certified teachers who were currently employed at the  school to provide RRI to at least one group of first grade students each day, for a total of 40 teachers over the 2-year study. The interventionists had a variety of primary job assignments on their campuses with 31 full-time reading teachers, 2 classroom teachers, 2 librarians, 1 special educator, 1 physical education coach who had never taught reading, and 1 educational assistant (i.e., paraprofessional) with teaching credentials. All but one of the teachers held elementary education teaching credentials, and several were certified in other areas as well, including English as a second language/bilingual education (15), early childhood education (13), reading specialist/master reading teacher (13), special education (12), and secondary education (8). Teachers’ total years of experience as professional educators ranged from 2 to 31 years (M = 17.15; SD = 8.38), and they had from 0 to 21 years of experience as reading interventionists. (M = 4.93; SD = 5.12). Seven teachers had previous training in an explicit phonics-emphasis intervention, and 15 had previously been trained in the Reading Recovery program. Teachers participated in about 18 hours of professional development in RRI over 3 days at the beginning of the school year, provided by the program developers and one experienced RRI teacher. It focused on (a) the research basis for the  program, (b) RRI instructional procedures, and (c) the use of assessment results in lesson planning, and included videotaped modeling and mock teaching sessions to practice procedures.  In Year 2, continuing  teachers received about 6 hours (1 day) of professional development at the beginning of the school year. In each of the 2 years, all teachers participated in 12 hours of follow-up professional development over 2 days—one in late fall and another in early spring. Two experienced former RRI teachers provided coaching to some of the intervention teachers in the study. Each RRI teacher in the study had been randomly assigned to receive support in one of the three following formats: (a) traditional on-site coaching (n = 15), (b) technology-based coaching (n = 13), and (c) on-demand coaching in which teachers could contact a coach with questions or request visits from a coach, but rarely did (n = 12) as part of a separate study.

Design: Partially Convincing Evidence

Did the study use random assignment?: No.

If not, was it a tenable quasi-experiment?: Yes.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Not applicable.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Yes.

Were the program and control groups demographically comparable at pretreatment?: Yes.

Was there attrition bias1? No.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Convincing Evidence

Describe when and how fidelity of treatment information was obtained: Two RRI coaches and one project coordinator observed intervention teachers and students for one entire teaching session, approximately once every 9 weeks for a total of three fidelity observations per teacher per year using a fidelity measure closely aligned with the intervention. The coaches did not observe their own teachers. Teachers did not receive feedback following these fidelity observations. Inter-observer reliability was established by co-viewing and independently rating lessons in schools, a procedure repeated within each cycle of data collection. Training of observers continued until they reached at least 80% agreement with program developers. Fidelity was rated on a 4 point Likert scale from 0 to 3 for each activity observed within each of the five components of a lesson. For each component, observers rated items addressing three domains: a) adherence to program procedures, (b) implementation quality, and (c) students’ on-task behaviors during the intervention.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: 
On average, teachers implemented RRI with adequate fidelity, although there was variation across teachers. The mean overall fidelity rating for all RRI teachers across all observations was 80.14% (SD = 8.43%; range, 57.33%--94.58%). Mean ratings (using the 0 to 3 scale with 0 meaning the element was expected but not observed and 3 being the highest rating) across all teachers for each of the three domains were 2.08 (SD = 0.47) for adherence to the RRI procedures; 2.26 (SD = 0.38) for quality of implementation, and 2.61 (SD = 0.22) for students’ on-task behavior.

Measures Targeted: Convincing Evidence

Measures Broader: Convincing Evidence

Targeted Measure Score type and range of measure Reliability statistics Relevance to program instructional content

Comprehensive Test of Phonological Processing (CTOPP) Blending Words

Raw score (0—20)

Internal consistency and alternate form reliability coefficients exceed .80; Test/retest coefficients range from 0.70 to 0.92.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in phonemic awareness.

CTOPP Segmenting Words

Raw score (0—20)

Internal consistency and alternate form reliability coefficients exceed .80; Test/retest coefficients range from 0.70 to 0.92.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in phonemic awareness.

Test of Word Reading Efficiency (TOWRE) Sight Word Efficiency

Raw score (0—104) and standard score (55—150)

Test retest reliabilities range from 0.83 to 0.92 for each subtest in the age range of the study.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in sight word recognition. 10 minutes is also spent in ‘Supported Reading’ as teachers provide scaffolding, feedback and instruction as students read increasingly difficult text.

TOWRE Phonemic Decoding

Raw score (0—63) and standard score (55—150)

Test retest reliabilities range from 0.83 to 0.92 for each subtest in the age range of the study.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in letter sound correspondences and phonemic decoding.

Woodcock Johnson-III (WJ-III) Letter Word Identification

WJ-II W score and standard score

The Basic Reading composite, composed of the WJ-III Letter Word ID and Word Attack subtests has split-half reliability of 0.91 to 0.97 in the age range of the study.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in letter sound correspondences, phonemic decoding, and sight word recognition. 10 minutes is also spent in ‘Supported Reading’ as teachers provide scaffolding, feedback and instruction as students read increasingly difficult text.

WJ-III Word Attack

WJ-II W score and standard score

The Basic Reading composite, composed of the WJ-III Letter Word ID and Word Attack subtests has split-half reliability of 0.91 to 0.97 in the age range of the study.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in letter sound correspondences and phonemic decoding.

WJ-III Spelling

WJ-II W score and standard score

Split-half reliabilities range from 0.87 to 0.91 in the age range of the study.

In each lesson, 10 minutes are devoted to Word Work. During this time, teachers use a variety of instructional activities to provide explicit and systematic instruction and practice in spelling, letter sound correspondences and phonemic decoding.

 

Broader Measure Score type and range of measure Reliability statistics Relevance to program instructional content

WJ-III Passage Comprehension

WJ-II W score and standard score

Split-half reliabilities range from 0.83 to 9.96 in the age range of the study.

10 minutes of every lesson is spent in Supported Reading. During this time, teachers provide scaffolding, feedback and comprehension instruction before, during and after students read increasingly difficult text.

Comprehensive Monitoring of Early Reading Skills Oral Reading Fluency (ORF)

Raw score (total number of correct words read)

High reliability for ORF curriculum based-measurement (ORF CMB) one-minute timings with elementary students was reported, (as cited in Fuchs, Fuchs, Hosp, & Jenkins, 2001; and Marston,1989). 

The Print Concepts/Fluency and assessment portion of the lesson consists of the teacher modeling fluent reading, repeated oral reading with feedback, and partner reading.

 

Number of Outcome Measures: 2 Prereading, 13 Reading

Mean ES - Targeted: Data Unavailable*u

Mean ES - Broader: Data Unavailable*u

Effect Size:

Targeted Measures

Construct Measure Effect Size
Prereading TOWRE Phonemic Decoding 0.42***, u
Prereading TOWRE Phonemic Decoding SS
Reading CTOPP Blending Words 0.25*, u
Reading CTOPP Segmenting Words 0.11 u
Reading TOWRE Sight Word Efficiency 0.44***, u
Reading TOWRE Sight Word Efficiency SS
Reading WJ-III Letter Word Identification W
Reading WJ-III Letter Word Identification SS 0.53***, u
Reading WJ-III Word Attack W 0.43***, u
Reading WJ-III Word Attack SS 0.39***, u
Reading WJ-III Spelling W 0.60***, u
Reading WJ-III Spelling SS 0.48***, u

 Broader Measures

Construct Measure Effect Size
Reading WJ-III Passage Comprehension W 0.53***, u
Reading WJ-III Passage Comprehension SS 0.49***, u
Reading Oral Reading Fluency

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Small Group, (n=3-4)

Duration of Intervention: 40 minutes, 5 times a week, 25 weeks

Minimum Interventionist Requirements: Paraprofessional, 18 hours of training

Reviewed by WWC or E-ESSA: E-ESSA

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

No studies considered met Evidence for ESSA's inclusion requirements.

Other Research: Potentially Eligible for NCII Review: 0 studies