Reading Recovery

Study: Center, Wheldall, Freeman, Outhred, & McNaught (1995)

Center, Y., Wheldall, K., Freeman, L., Outhred, L., & McNaught, M. (1995). An evaluation of Reading Recovery. Reading Research Quarterly, 30, 240-263.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

Reading Recovery is a highly effective, short-term intervention of daily one-to-one lessons that supplement good classroom teaching for the lowest-achieving first graders. The goal is to dramatically reduce the number of first- grade children with extreme difficulty learning to read and write and to reduce the cost of these learners to educational systems.

The lowest literacy achievers in Grade 1 receive a half-hour lesson each school day for 12-20 weeks with a specially trained Reading Recovery teacher. Daily and weekly records provide rich information for progress monitoring. As soon as children meet grade-level expectations and can continue to learn in the classroom, their lessons are discontinued, and new students begin individual instruction. Just as in the intent of RTI, two positive outcomes are possible: (a) responding to the intervention by meeting grade-level literacy expectations (approximately 75% of the children complete interventions) and (b) recommended for further evaluation and future support.

The intent of Reading Recovery lessons is to promote accelerated learning so that students can catch up with their peers and benefit from classroom instruction without supplemental help. Each lesson is individually designed and individually delivered by specially trained teachers and includes: reading familiar books, reading yesterday’s new book while the teacher takes a running record, working with letters and/or words using magnetic letters, composing a written message, assembling a cut-up story, and reading a new book.

Extensive training is required for the responsive, contingent teaching in Reading Recovery. Three levels of training form a network of expertise and support: (a) university-based trainers of teacher leaders; (b) site-based teacher leaders who train teachers; and (c) school-based teachers who deliver the intervention on their campuses.

Reading Recovery is intended for use in first grade. It is designed for any student at risk of academic failure. The academic area of focus is reading (including phonological awareness, phonics/word study, comprehension, fluency, and vocabulary) and writing (including spelling, sentence construction, phonemic awareness, letter-sound relationships, phonological and orthographic awareness, analogy, and writing vocabulary).

Reading Recovery has been implemented in the United States for 25 years. During the 2008-2009 school year, 1,909 school systems in 47 states were implementing Reading Recovery and 34 school systems in 8 states implemented Descubriendo la Lectura.

Where to obtain:
Reading Recovery Council of North America
500 W. Wilson Bridge Rd., Suite 250
Worthington, Ohio 43085-5218
Phone: 614-310-7323
Website: www.readingrecovery.org

Also available at University Training Centers: http://www.readingrecovery.org/development/centers/index.asp
          
Cost:
Initial per teacher costs for a new implementation in a school would include the cost of the year-long training (costs vary according to universities granting credit), the initial purchase of non-consumable children’s books to be used with students (about $2,500), professional books, materials, and supplies (approximately $250), and data processing fees (about $57).

Replacement cost per student for subsequent use: The cost of Reading Recovery per trained teacher is dramatically reduced after the training year. The only replacement costs are optional additions to the non-consumable set of books used with children and possible non-consumable professional books for teachers. Expenditures to the district/site include ongoing professional development, data processing fees, university affiliation fees if applicable, and miscellaneous supplies.

Reading Recovery is designed for use with individual students.

Reading Recovery takes 30 minutes per session with a recommended five sessions per week for 15-20 weeks.

The program includes a highly specified manual of teaching procedures from which teachers select to meet needs of each individual child. Technology is not required for the intervention with children. However, required data are sent electronically to the International Data Evaluation Center (IDEC).

An academic year of training with university credit is required for instructors.

For an academic year, teachers meet weekly with a registered Reading Recovery teacher leader who has completed a full year of training at a university training center to prepare for training teachers. While in training, teachers are also working with Reading Recovery students. Teachers learn to observe and discuss lessons behind a one-way mirror/glass.

Concurrent with the training, these teachers work with children in one-to-one lessons for part of the school day. Teacher leaders also make school visits and support implementation in schools.

Instructors must be professionals. When selecting Reading Recovery teachers, schools are encouraged to consider teaching experience, preferably with primary-grade students.

The training manuals have been trialed with thousands of children across the years and subsequent revisions followed. Books used for training Reading Recovery professionals also have evolved after trials with children.

Ongoing professional development is a requirement for all Reading Recovery professionals. Several professional development sessions are scheduled yearly and include continued observation of one-to-one teaching sessions. Reading Recovery teacher leaders also provide ongoing support to teachers at their schools.

 

Participants: Convincing Evidence

Sample size: 109 students pre-tested across ten schools first grade. (31 students in the treatment group and 39 students in the control group. There was also a comparison [buffer] group of 39 students.) 28 treatment and 34 control students were post-tested.

Risk Status: Participants in the RR treatment and the comparison schools were identified by their teachers as children whom they consider to be at great risk of reading failure after one year at school. In each of these schools the RR teacher administered the Clay Diagnostic Survey to these children, and then the 12 lowest achieving students from each of these schools were selected to participate in the research.

Demographics:

 

Program

Control

p of chi square

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

31

100

39

100

 

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity                        

  African-American

 

 

 

 

 

  American Indian

 

 

 

 

 

  Asian/Pacific Islander

 

 

 

 

 

  Hispanic

 

 

 

 

 

  White

 

 

 

 

 

  Other

 

 

 

 

 

Socioeconomic status        

  Subsidized lunch

 

 

 

 

 

  No subsidized lunch

 

 

 

 

 

Disability status          

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

 

 

 

 

 

ELL status                            

  English language learner

 

 

 

 

 

  Not English language learner

 

 

 

 

 

Gender                               

Female

 

 

 

 

 

Male

 

 

 

 

 

Training of Instructors: This was a standard Reading Recovery implementation. Teachers had participated in the initial training year and ongoing professional development as described above.

Design: Unconvincing Evidence

Did the study use random assignment?: No.

If not, was it a tenable quasi-experiment?: Yes.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Not applicable.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? There were some statistically significant differences between treated and comparison groups at pretest, and it is unclear whether differences were adjusted for in the analysis.

Were the program and control groups demographically comparable at pretreatment?: Although the study states that students were demographically similar, no data are provided to show this.

Was there attrition bias1? Attrition did occur; it was unclear whether this had a differential impact on group composition.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Unconvincing Evidence

Describe when and how fidelity of treatment information was obtained: Reading Recovery teachers follow a standard lesson format, but make individual decisions within that framework to accelerate the progress of individual students. Training ensures fidelity of treatment. Teacher leaders also observe lessons and monitor fidelity as part of their roles in providing support to the teachers. All of the teachers participating in this study had completed the professional development experiences required to be certified Reading Recovery teachers and were continuing to participate in ongoing professional development, which includes visits by Reading Recovery teacher leaders to observe lessons with students.

Systematic observation of each RR teacher for one session with each of 4 students, by trained researcher, was undertaken in April 1991 when teachers had spent 6 weeks with their students. A short observation form covering the seven components of the program was used and specific and general instances of positive/negative reinforcement were also recorded.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: The observers indicated that all RR teachers included the seven mandatory components in their individualized sessions, all students remained on task for almost the entire intervention period, and a large amount of positive reinforcement was provided by all teachers to all of their students.

Measures Targeted: Convincing Evidence

Measures Broader: Partially Convincing Evidence

Targeted Measure Score type & range of measure Reliability statistics Relevance to program instructional content

Text Level task

The Ohio stanines for text level indicated an average of Level 2 for the fall of first grade, with a range of Level 9 to Level 12 for average performance in the spring of first grade. The National Data Evaluation Center random sample data indicated an average end of first-grade text level of 20 for a stratified national sample.

The scoring of running records, on which the text level decisions were based, was reliable across two scorings by a trained recorder over a 2-year interval (r = 0.98).

The gradient of difficulty reflected in these texts was similar to instructional materials used in the RR program and many early literacy classroom programs.

Letter Identification task

Students were asked to respond to 26 upper and 28 lowercase letter forms. The additional lowercase letters included two forms of a and g. The child could respond with a letter name, a sound, or a word beginning with that letter (maximum score = 54).

Cronbach’s α = 0.95

 

Concepts About Print task

The adult read one of the booklets to the child. The child was asked to help by responding to questions or requests related to book handling, directional behavior, visual scanning, and specific concepts related to printed language, like punctuation, and the relationship of letters and words within sentences (maximum score = 24).

Cronbach’s α =  0.78; split-half r = 0.95

 

Word Tests

Scoring was based on the number of words read correctly (maximum score = 20).

Cronbach’s α = 0.92

 

Writing Vocabulary

Scoring was a count of the number of words correctly generated.

Test–retest r = 0.62 and 0.97

 

Hearing and Recording Sounds in Words (HRSW) task

The task was scored on the number of phonemes correctly recorded (maximum score = 37).

Cronbach’s α = 0.96

 

 

Broader Measure Score type & range of measure Reliability statistics Relevance to program instructional content

Passage Reading Test

Standardized and criterion-referenced test.

The test measures the median number of words read correctly in 1 minute, from each of three passages selected from a basal reader. There were high correlations (ranging from .73 to .91) between rate of reading words aloud from text accurately and reading comprehension scores (from published standardized tests).

Test-retest coefficients for Passage 1 = 0.96, for Passage 2 = 0.96, and for Passage 3 = 0.95

 

Phonemic Awareness Test

Standardized and criterion-referenced test.

Includes: recognition and supply of rhyming pairs; alliteration – sound-to-word and word-to-word matching of sounds spoken 1 second apart, in one syllable words; blending the Yopp-Singer phoneme segmentation test to measure ability to articulate the individual sounds of a word; and phoneme deletion to measure ability to delete initial, medial, and final sounds in words.

Test-retest coefficient = 0.91

 

Word Attack Skills Test

Standardized and criterion-referenced test.

Test-retest coefficient = 0.93

Measures a student’s phonological recoding, which is one aspect of meta-linguistic skills.
Neale Analysis of Reading Ability      
Waddington Diagnostic Spelling Test      
Cloze Test      
Burt Word Reading Test      

 

Number of Outcome Measures: 1 Prereading, 7 Reading

Mean ES - Targeted: 1.48*u

Mean ES - Broader: Data Unavailable*u

Effect Size:

Targeted Measures

Construct Measure Effect Size
Reading Clay’s Book Level Test 1.48***, u

 Broader Measures

Construct Measure Effect Size
Prereading Phonemic Awareness Test 0.77**,u
Reading Neale Analysis of Reading Ability 1.25***,u
Reading Passage Reading Text 1.24***,u
Reading Waddington Diagnostic Spelling Test 1.00***,u
Reading Cloze Test 0.41u
Reading Word Attack Skills Test
Reading Burt Word Reading Test

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual

Duration of Intervention: 30 minutes, 5 times a week, 15 weeks

Minimum Interventionist Requirements: Professional, An academic year of training, with university credit

Reviewed by WWC or E-ESSA: WWC & E-ESSA

What Works Clearinghouse Review

Beginning Readers Protocol

EffectivenessReading Recovery® was found to have positive effects on alphabetics and general reading achievement and potentially positive effects on fluency and comprehension.

Studies Reviewed: 3 studies meet standards out of 79 studies total

Full Report

English Language Learners Protocol

Effectiveness: No studies of Reading Recovery® that fall within the scope of the English Language Learners (ELL) review protocol meet What Works Clearinghouse (WWC) evidence standards. The lack of studies meeting WWC evidence standards means that, at this time, the WWC is unable to draw any conclusions based on research about the effectiveness or ineffectiveness of Reading Recovery® on ELL.

Studies Reviewed: N/A

Full Report

 

Evidence for ESSA

Program Outcomes: Reading Recovery has been evaluated in four qualifying studies (many other studies did not qualify because they only reported outcomes for successful students, or used developer-made measures). In comparison to control groups, the average effect size across the four studies was +0.43 on measures such as ITBS, CAT, Woodcock, and Gates. These outcomes qualify Reading Recovery for the ESSA “Strong” category, and for the “Solid Outcomes” rating (at least two studies with effect sizes of at least +0.20).

Number of Studies: 4

Average Effect Size: 0.43

Full Report

Other Research: Potentially Eligible for NCII Review: 3 studies

Baenen, N., Bernhole, A., Dulaney, C., & Banks, K. (1997). Reading Recovery: Long-term progress after three cohorts. Journal of Education for Students Placed at Risk, 2(2), 161.
 

Pinnell, G. S., DeFord, D. E., & Lyons, C. A. (1988). Reading Recovery: Early intervention for at-risk first graders (Educational Research Service Monograph). Arlington, VA: Educational Research Service.
 

Pinnell, G. S., Lyons, C. A., DeFord, D. E., Bryk, A. S., & Seltzer, M. (1994). Comparing instructional models for the literacy education of high-risk first graders. Reading Research Quarterly, 29(1), 8–39.