Reading Recovery

Study: Schwartz (2005)

Schwartz, R. M. (2005). Literacy learning of at-risk first grade students in the Reading Recovery early intervention. Journal of Educational Psychology, 97(2), 257-267.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

Reading Recovery is a highly effective short-term intervention of daily one-to-one lessons that supplement good classroom teaching for the lowest-achieving first graders. The goal is to dramatically reduce the number of first- grade children with extreme difficulty learning to read and write and to reduce the cost of these learners to educational systems.

The lowest literacy achievers in Grade 1 receive a half-hour lesson each school day for 12-20 weeks with a specially trained Reading Recovery teacher. Daily and weekly records provide rich information for progress monitoring. As soon as children meet grade-level expectations and can continue to learn in the classroom, their lessons are discontinued, and new students begin individual instruction. Just as in the intent of RTI, two positive outcomes are possible: (a) responding to the intervention by meeting grade-level literacy expectations (approximately 75% of the children complete interventions) and (b) recommended for further evaluation and future support.

The intent of Reading Recovery lessons is to promote accelerated learning so that students can catch up with their peers and benefit from classroom instruction without supplemental help. Each lesson is individually designed and individually delivered by specially trained teachers and includes: reading familiar books, reading yesterday’s new book while the teacher takes a running record, working with letters and/or words using magnetic letters, composing a written message, assembling a cut-up story, and reading a new book.

Extensive training is required for the responsive, contingent teaching in Reading Recovery. Three levels of training form a network of expertise and support: (a) university-based trainers of teacher leaders; (b) site-based teacher leaders who train teachers; and (c) school-based teachers who deliver the intervention on their campuses.

Reading Recovery is intended for use in first grade. It is designed for any student at risk of academic failure. The academic area of focus is reading (including phonological awareness, phonics/word study, comprehension, fluency, and vocabulary) and writing (including spelling, sentence construction, phonemic awareness, letter-sound relationships, phonological and orthographic awareness, analogy, and writing vocabulary).

Reading Recovery has been implemented in the United States for 25 years. During the 2008-2009 school year, 1,909 school systems in 47 states were implementing Reading Recovery and 34 school systems in 8 states implemented Descubriendo la Lectura.

Where to obtain:
Reading Recovery Council of North America
500 W. Wilson Bridge Rd., Suite 250
Worthington, Ohio 43085-5218
Phone: 614-310-7323
Website: www.readingrecovery.org

Also available at University Training Centers: http://www.readingrecovery.org/development/centers/index.asp
          
Cost:
Initial per teacher costs for a new implementation in a school would include the cost of the year-long training (costs vary according to universities granting credit), the initial purchase of non-consumable children’s books to be used with students (about $2,500), professional books, materials, and supplies (approximately $250), and data processing fees (about $57).

Replacement cost per student for subsequent use: The cost of Reading Recovery per trained teacher is dramatically reduced after the training year. The only replacement costs are optional additions to the non-consumable set of books used with children and possible non-consumable professional books for teachers. Expenditures to the district/site include ongoing professional development, data processing fees, university affiliation fees if applicable, and miscellaneous supplies.

Reading Recovery is designed for use with individual students.

Reading Recovery takes 30 minutes per session with a recommended five sessions per week for 15-20 weeks.

The program includes a highly specified manual of teaching procedures from which teachers select to meet needs of each individual child. Technology is not required for the intervention with children. However, required data are sent electronically to the International Data Evaluation Center (IDEC).

An academic year of training with university credit is required for instructors.

For an academic year, teachers meet weekly with a registered Reading Recovery teacher leader who has completed a full year of training at a university training center to prepare for training teachers. While in training, teachers are also working with Reading Recovery students. Teachers learn to observe and discuss lessons behind a one-way mirror/glass.

Concurrent with the training, these teachers work with children in one-to-one lessons for part of the school day. Teacher leaders also make school visits and support implementation in schools.

Instructors must be professionals. When selecting Reading Recovery teachers, schools are encouraged to consider teaching experience, preferably with primary-grade students.

The training manuals have been trialed with thousands of children across the years and subsequent revisions followed. Books used for training Reading Recovery professionals also have evolved after trials with children.

Ongoing professional development is a requirement for all Reading Recovery professionals. Several professional development sessions are scheduled yearly and include continued observation of one-to-one teaching sessions. Reading Recovery teacher leaders also provide ongoing support to teachers at their schools.

 

Participants: Convincing Evidence

Sample size: 72 students in first grade (36 students in the treatment group and 36 students in the control group). 

Risk Status: The Reading Recovery teachers followed the program procedures for identifying the lowest-performing first-grade students. These procedures included recommendations from the first-grade classroom teachers and then assessment with Clay’s An Observation Survey of Early Literacy Achievement (2002, 2006).

Demographics:

 

Program

Control

p of chi square

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

36

100

36

100

 

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity

  African-American

 

47

 

38

 

  American Indian

 

 

 

 

 

  Asian/Pacific Islander

 

3

 

0

 

  Hispanic

 

12

 

15

 

  White

 

38

 

47

 

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

 

60

 

57

 

  No subsidized lunch

 

40

 

43

 

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

 

 

 

 

 

ELL status

  English language learner

 

 

 

 

 

  Not English language learner

 

 

 

 

 

Gender

Female

 

39

 

59

 

Male

 

61

 

41

 

Training of Instructors: This was a standard Reading Recovery implementation. Teachers had participated in the initial training year and ongoing professional development.

Design: Convincing Evidence

Did the study use random assignment?: Yes.

If not, was it a tenable quasi-experiment?: Not applicable.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Yes.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Not applicable.

Were the program and control groups demographically comparable at pretreatment?: Yes.

Was there attrition bias1? No

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Unconvincing Evidence

Describe when and how fidelity of treatment information was obtained: Reading Recovery teachers follow a standard lesson format, but make individual decisions within that framework to accelerate the progress of individual students. Training ensures fidelity of treatment. Teacher leaders also observe lessons and monitor fidelity as part of their roles. All of the teachers participating in this study had completed the professional development experiences required to be certified Reading Recovery teachers and were continuing to participate in ongoing professional development, which includes visits by Reading Recovery teacher leaders to observe lessons with students.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: This is a study of an intervention that has been brought to scale at a national and international level and has 25 years of standard implementation protocols in the United States. The lesson framework and extensive professional development ensure a high level of treatment fidelity. Individual differences among teachers and decisions within the lesson that are intended to accelerate a particular child’s learning are part of the intervention.

Measures Targeted: Convincing Evidence

Measures Broader: Convincing Evidence

Targeted Measure Score type & range of measure Reliability statistics Relevance to program instructional content

Text Level task

The Ohio stanines for text level indicated an average of Level 2 for the fall of first grade, with a range of Level 9 to Level 12 for average performance in the spring of first grade. The National Data Evaluation Center random sample data indicated an average end of first-grade text level of 20 for a stratified national sample.

The scoring of running records, on which the text level decisions were based, was reliable across two scorings by a trained recorder over a 2-year interval (r = 0.98).

The gradient of difficulty reflected in these texts was similar to instructional materials used in the RR program and many early literacy classroom programs.

Letter Identification task

Students were asked to respond to 26 upper and 28 lowercase letter forms. The additional lowercase letters included two forms of a and g. The child could respond with a letter name, a sound, or a word beginning with that letter (maximum score = 54).

Cronbach’s α = 0.95

 

Concepts About Print task

The adult read one of the booklets to the child. The child was asked to help by responding to questions or requests related to book handling, directional behavior, visual scanning, and specific concepts related to printed language, like punctuation, and the relationship of letters and words within sentences (maximum score = 24).

Cronbach’s α =  0.78; split-half r = 0.95

 

The Ohio Word Test

Scoring was based on the number of words read correctly (maximum score = 20).

Cronbach’s α = 0.92

 

Writing Vocabulary task

Scoring was a count of the number of words correctly generated.

Test–retest r = 0.62 and 0.97

 

Hearing and Recording Sounds in Words (HRSW) task

The task was scored on the number of phonemes correctly recorded (maximum score = 37).

Cronbach’s α = 0.96

 

 

Broader Measure Score type & range of measure Reliability statistics Relevance to program instructional content

Phoneme Segmentation Test

The task consisted of 4 practice items and 22 test items. Each item was scored as 1 point if all phonemes were separated and articulated (maximum score = 22).

Cronbach’s α = 0.95

 

Deletion Task

Two items ask for syllable deletion, with the remaining item requiring phoneme deletion from an initial, medial, or final position. The phoneme or syllable deletion resulted in a different word (maximum score = 10).

Cronbach’s α = 0.78

 

Slosson Oral Reading Test—Revised

This was a standardized, norm referenced measure (maximum score = 200).

Kuder–Richardson 21 for ages 6 to 7 = 0.98

 

The Degrees of Reading Power Test

This was a standardized, norm referenced measure. The primary forms were recommended for the end of first grade and the beginning of second grade (maximum score = 28).

Kuder–Richardson 20 at second grade level = 0.92)

 

 

Number of Outcome Measures: 3 Prereading , 7 Reading

Mean ES - Targeted: 1.15*u

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct Measure Effect Size
Prereading Concepts About Print 1.08***, u
Prereading Letter ID 0.48*, u
Reading Text Reading Level 2.06***, u
Reading Ohio Word Test 1.35***, u
Reading Writing Vocabulary 0.90***, u
Reading Hearing and Recording Sounds in Words 1.03***, u

 Broader Measures

Construct Measure Effect Size
Prereading Phoneme Segmentation
Reading Slosson Oral Reading Test - Revised
Reading Degrees of Reading Power
Reading Deletion

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual

Duration of Intervention: 30 minutes, 5 times a week, 12-20 weeks

Minimum Interventionist Requirements: Professional, An academic year of training, with university credit

Reviewed by WWC or E-ESSA: WWC & E-ESSA

What Works Clearinghouse Review

Beginning Readers Protocol

EffectivenessReading Recovery® was found to have positive effects on alphabetics and general reading achievement and potentially positive effects on fluency and comprehension.

Studies Reviewed: 3 studies meet standards out of 79 studies total

Full Report

English Language Learners Protocol

Effectiveness: No studies of Reading Recovery® that fall within the scope of the English Language Learners (ELL) review protocol meet What Works Clearinghouse (WWC) evidence standards. The lack of studies meeting WWC evidence standards means that, at this time, the WWC is unable to draw any conclusions based on research about the effectiveness or ineffectiveness of Reading Recovery® on ELL.

Studies Reviewed: N/A

Full Report

 

Evidence for ESSA

Program Outcomes: Reading Recovery has been evaluated in four qualifying studies (many other studies did not qualify because they only reported outcomes for successful students, or used developer-made measures). In comparison to control groups, the average effect size across the four studies was +0.43 on measures such as ITBS, CAT, Woodcock, and Gates. These outcomes qualify Reading Recovery for the ESSA “Strong” category, and for the “Solid Outcomes” rating (at least two studies with effect sizes of at least +0.20).

Number of Studies: 4

Average Effect Size: 0.43

Full Report

Other Research: Potentially Eligible for NCII Review: 3 studies

Baenen, N., Bernhole, A., Dulaney, C., & Banks, K. (1997). Reading Recovery: Long-term progress after three cohorts. Journal of Education for Students Placed at Risk, 2(2), 161.
 

Pinnell, G. S., DeFord, D. E., & Lyons, C. A. (1988). Reading Recovery: Early intervention for at-risk first graders (Educational Research Service Monograph). Arlington, VA: Educational Research Service.
 

Pinnell, G. S., Lyons, C. A., DeFord, D. E., Bryk, A. S., & Seltzer, M. (1994). Comparing instructional models for the literacy education of high-risk first graders. Reading Research Quarterly, 29(1), 8–39.