Read Naturally
Study: Heistad (2005)

Describe how students were selected to participate in the study:

Schools were recruited from among Minneapolis Public Schools to volunteer for the study. Four schools agreed to participate. Students were selected for participation based on the recommendations of school-based teams and/or parent requests/nominations. The students who were selected were considered to be “not on course” in reading achievement and were identified because the teams believed that they would not achievement proficiency as measured by the MN Comprehensive Assessment, which was administered in the spring of Grade 3 and Grade 5. Ninety-six students across grades were nominated to participate in the program condition (Read Naturally) and 78 of those students were included all analysis. The 18 students excluded for analysis were missing either pre-test or post-test data. An additional 78 students were identified from other Minneapolis Schools with the same AYP status. A match procedure was used to identify and select students based on pre-test and demographic variables. Students were matched first on NALT pretest score from spring of 2003 (i.e. matches needed to be within 3 scale score points of the target student) and then by the following demographic factors: 1) Grade 2) English Language Learner status 3) Special Education status 4) Free or reduced price lunch 5) Racial/Ethnic category 6) Home Language 7) Sex Perfect matches of RN and control students were accomplished for 73 (93%) of the pairs, 3 (4%) pairs were matched on 7 of 8 variables, and 2 pairs (3%) were matched on 6 of 8 variables.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Students in the program conditions were selected for participation based on the recommendations of school-based teams and/or parent requests/nominations. The students who were selected were considered to be “not on course” in reading achievement and were identified because the teams believed that they would not achieve proficiency as measured by the MN Comprehensive Assessment, which was administered in the spring of Grade 3 and Grade 5. Students in the control (business as usual) condition were selected as a matched sample from the available population in Minneapolis Public Schools.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Read Naturally fluency intervention The first section of READ NATURALLY instruction involves a student choosing a story from his/her individual instructional level and making a prediction. The student then writes what s/he already knows about the subject of the story. Next, the student takes a “cold timing” on the passage where s/he reads for one minute and records difficult words. The student then graphs the number of words read correctly per minute. During the next component of instruction the student reads along with prerecorded audio of a fluent reader on the same passage three consecutive times, with each reading slightly faster than the previous reading. The student then reads the story independently without audio support. The student sets the timer for one minute for each reading and practices the passage several times until the predetermined rate (i.e. words read correctly) is reached. The final part of the process occurs once the target fluency is reached. The student then answers multiple choice and constructed response questions that pertain to the story. Passages at each grade level include non-fiction themes. The questions tap inferential and literal passage comprehension. After answering the questions, the student retells the story in writing. The entire process is monitored by the instructor with corrective feedback and guided practice provided as needed.

Specify which condition is the control condition:

No treatment, business as usual.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Systematic

Please describe the assignment method or the process for defining treatment/comparison groups.

Each student receiving READ NATURALLY services was matched with a student not receiving READ NATURALLY (RN) services. Students were matched first on NALT pretest score from spring of 2003 (i.e. matches needed to be within 3 scale score points of the target student) by the following demographic factors: Pretest Score, Grade, English Language Learner status, Special Education status, Free or reduced price lunch, Racial/Ethnic category, Home Language, Sex, Birth Date (to break ties). Matching was accomplished by sorting the merged file of RN students with the entire grade 3-5 population of students. The file was sorted hierarchically by the demographic factors above. The best match to the RN student was typically the student directly above or below in the record. The number of variables was computed and if the student directly above and directly below the RN student had the same number of matches, the student birth date closest to the RN student was used to break the tie. Perfect matches of RN and control students were accomplished for 73 (93%) of the pairs, 3 (4%) pairs were matched on 7 of 8 variables, and 2 pairs (3%) were matched on 6 of 8 variables.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

study used matched sample by student

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

One lead teacher in each school site was provided training from the vendor, which was a standard 6-hour training.

Describe when and how fidelity of treatment information was obtained.

Three visits from a trained Reading Fluency Monitor (RFM) assessor were made during the year. While collecting the RFMs the assessor made informal observations of the degree of student engagement and use of RN materials.

What were the results on the fidelity-of-treatment implementation measure?

Feedback was provided to teachers to ensure fidelity, but data were not collected and analyzed.

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Ran a series of paired sample t-tests and a chi-square analysis of Level III proficiency on the state test (MCA)