Read Naturally

Study: Heistad (2005)

Heistad, D. (2005). The Effects of READ NATURALLY on fluency and reading comprehension: A supplemental service intervention. Technical Report.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

Read Naturally programs develop fluency, support vocabulary development, and promote comprehension using the research-based strategies of teacher modeling, repeated reading, and progress monitoring. Three versions are available. Read Naturally Encore Edition uses printed stories with audio support on audio CDs. Read Naturally Software Edition (SE) uses computer software that guides students through the steps of the program.Read Live is our web-based intervention version and assessment program.

Read Naturally is intended for use in grades 1 through high school. It is designed for use with students with disabilities, (including learning disabilities, intellectual disabilities, and behavioral disabilities), English language learners, and any student at risk of academic failure. The academic area of focus is reading comprehension and fluency.

Read Naturally is used in all 50 states and several countries. Thousands of districts have implemented Read Naturally since 1991.

Where to obtain:
Read Naturally
2945 Lone Oak Dr. #190
St. Paul, MN 55121-4902
Phone: 800-788-4085
Website: www.readnaturally.com
Email:  info@readnaturally.com

Cost:
One time pricing:

Read Naturally Encore: $129 per level including license for all students. Encore audio CD level contains 12 audio CDs, 24 nonfiction high interest stories, and Teachers Manual and Glossary.

Software Edition: $125 stand alone per level, Server (school $200 or district $1000) license for all students, $399 school site license per level for all students in school.

Read Live subscription: 
1 seat $149
6 seats $299
30 seats $599
130 seats $1999

Read Naturally is designed for use with individual students or small groups of six to eight students.

Read Naturally takes 30 minutes per session with a recommended three to five sessions per week for 36 weeks.

The program includes a highly specified teacher’s manual.

The Encore Edition requires a timer, CD player, and headphones.

The Software Edition requires a computer.

Read Live requires a computer and high speed internet access.

Four to eight hours of training is required for the instructor.

Instruction training options include: full day seminars; SE self study course; school district training; virtual seminars; video workshop for teachers; and video workshop for assistants and Read Live
Hands-on Training Workbook.

Instructors must have expertise in reading instruction and may be professionals, paraprofessionals or volunteers.

Customer service support, virtual seminars, and follow-up training are available.

 

Participants: Unconvincing Evidence

Sample size: 156 students across four schools in grades 3, 4, and 5. (78 students in the treatment group and 78 students in the control group)

Risk Status: Students in the program conditions were selected for participation based on the recommendations of school-based teams and/or parent requests/nominations. The students who were selected were considered to be “not on course” in reading achievement and were identified because the teams believed that they would not achieve proficiency as measured by the MN Comprehensive Assessment, which was administered in the spring of Grade 3 and Grade 5.

Students in the control (business as usual) condition were selected as a matched sample from the available population in Minneapolis Public Schools.

Demographics:

 

Program

Control

p of chi square

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

 

 

 

 

 

  Grade 2

 

 

 

 

 

  Grade 3

23

14

23

14

p > 0.05

  Grade 4

33

21

33

21

p > 0.05

  Grade 5

22

14

22

14

p > 0.05

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity

  African-American

28

36

30

38

p > 0.05

  American Indian

4

5

4

5

p > 0.05

  Asian/Pacific Islander

0

0

0

0

p > 0.05

  Hispanic

20

30

20

30

p > 0.05

  White

13

20

13

20

p > 0.05

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

50

34

50

34

p > 0.05

  No subsidized lunch

28

16

28

16

p > 0.05

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

9

12

9

12

p > 0.05

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

 

 

 

 

 

ELL status

  English language learner

28

36

28

36

p > 0.05

  Not English language learner

50

64

50

64

p > 0.05

Gender

Female

34

44

37

47

p > 0.05

Male

44

56

41

53

 

Training of Instructors: One lead teacher in each school site was provided training from the vendor, which was a standard 6-hour training.

Design: Partially Convincing Evidence

Did the study use random assignment?: No.

If not, was it a tenable quasi-experiment?: Yes.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Not applicable.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Yes.

Were the program and control groups demographically comparable at pretreatment?: Yes.

Was there attrition bias1? No.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Unconvincing Evidence

Describe when and how fidelity of treatment information was obtained: Three visits from a trained Reading Fluency Monitor (RFM) assessor were made during the year. While collecting the RFMs the assessor made informal observations of the degree of student engagement and use of RN materials.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Feedback was provided to teachers to ensure fidelity, but data were not collected and analyzed.

Measures Targeted: Convincing Evidence

Measures Broader: Data Unavailable

Targeted Measure Score type & range of measure Reliability statistics Relevance to program instructional content

Northwest Achievement Levels Tests (NALT)

Rasche scores using a scale of 140 to 280; also reported in normal percentile equivalents

Test-retest reliability = Spring to Fall stability for MN aligned version ranges from 0.84 to 0.90 with a median of 0.88. Internal consistency ranges from 0.93 to 0.96 with a median of 0.94.

Criterion related validity coefficients with the Minnesota Comprehensive Test ranges from 0.85 to 0.87 with a median of 0.86.

 Read Naturally Fluency Monitor (Curriculum Based Measurement of Oral Reading)

Raw scores in units of Words Read Correctly per minute. The possible range for CBM-R is approximately 0 to 300+

Test-retest and alternate form coefficients approximate0.91.

Criterion related validity coefficients with the Minnesota Comprehensive Test approximate 0.83 in 3rd Grade and 0.75in 5th Grade; coefficients with NALT approximate 0.91 across grades, with similar coefficients with other criterion measures.

Minnesota Comprehensive Assessment in Reading (MCA)

Rasche model used to calibrate. Standard scores reported ranging from 1200 to 1750 with 1420 as proficient for each grade level. “Partially Proficient” = 1280; “Advanced” = 1500

No test-retest reported. Internal consistency reported at 0.95.

Aligned to the frameworks and curriculum standards for Minnesota

 

Broader Measure Score type & range of measure Reliability statistics Relevance to program instructional content

Not Applicable

 

 

 

 

Number of Outcome Measures: 3 Reading

Mean ES - Targeted: Data Unavailableu

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct Measure Effect Size
Reading Read Naturally Fluency Monitor 0.14u
Reading NALT 0.24u
Reading MCA (Matching variable = NALT)

 Broader Measures

Construct Measure Effect Size
  Not Applicable  

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Small Group, (n=5-10)

Duration of Intervention: 30 minutes, 3-5 times a week, 30 weeks

Minimum Interventionist Requirements: Paraprofessional / Volunteer, 6 hours of training

Reviewed by WWC or E-ESSA: WWC

What Works Clearinghouse Review

Adolescent Literacy Evidence Protocol

Effectiveness: Read Naturally® was found to have potentially positive effects on general literacy achievement for adolescent readers.

Studies Reviewed: 1 study meets standards out of 4 studies total

Full Report

Beginning Reading Protocol

EffectivenessRead Naturally® was found to have no discernible effects on fluency and reading comprehension.

Studies Reviewed: 5 studies meet standards out of 11 studies total

Full Report

English Language Learners Protocol

EffectivenessRead Naturally® was found to have no discernible effects on reading achievement and English language development of elementary school English language learners.

Studies Reviewed: 2 studies meet standards out of 3 studies total

Full Report

Students with Learning Disabilities Protocol

Effectiveness: Read Naturally® was found to have no discernible effects on reading fluency and potentially positive effects on writing for students with learning disabilities.

Studies Reviewed: 2 studies meet standards out of 3 studies total

Full Report

 

Evidence for ESSA

This program was not reviewed by Evidence for ESSA.

Other Research: Potentially Eligible for NCII Review: 6 studies

Arvans, R. (2010). Improving reading fluency and comprehension in elementary students using Read Naturally. Dissertation Abstracts International, 71(01B), 74-649.
 

Chenault, B., Thomson, J., Abbott, R. D., & Berninger, V. W. (2006). Effects of prior attention training on child dyslexics’ response to composition instruction. Developmental Neuropsychology, 29(1), 243–260.
 

Denton, C. A., Anthony, J. L., Parker, R., & Hasbrouck, J. E. (2004). Effects of two tutoring programs on the English reading development of Spanish-English bilingual students. The Elementary School Journal, 104(4), 289–305.
 

Hancock, C. M. (2002). Accelerating reading trajectories: The effects of dynamic research-based instruction. Dissertation Abstracts International, 63(6), 2139A. (UMI No. 3055690)
 

Kemp, S. C. (2006). Teaching to Read Naturally: Examination of a fluency training program for third grade students (Doctoral dissertation, University of California, Irvine and University of California, Los Angeles, 2006). Dissertation Abstracts International, 67(7A), 95-2447.
 

Mesa, C. L. (2004). Effect of Read Naturally software on reading fluency and comprehension. Unpublished master’s thesis, Piedmont College, Demorest, GA.