Failure Free Reading
Study: Torgesen et al. (2006)
Summary
Failure Free Reading (FFR) is a research based, scientifically validated diagnostic/prescriptive reading intervention documented in numerous quality design research studies to improve reading outcomes for students who have not responded to regular and/or remedial reading instruction. The program’s unique combination of scaffolds enable the lowest level readers to immediately experience success with age- and grade level- appropriate passages regardless of current reading ability. FFR’s approach has consistently been shown to rapidly build comprehension, vocabulary and fluency, along with efficacy, confidence and esteem. Multi-sensory, and based on intensive language development, the print-based FFR methodology was developed in the late 1980’s by program author Dr. Joseph Lockavitch for special education and at-risk students, along with other “treatment resisters” (including those with language and/or audio phonological deficits). The software component was added in the 1990s. Over the years, numerous studies involving FFR as treatment (including 8 published studies) reported rapid and significant gains in comprehension and other measures. For example, two of the published studies reported a more than 40% reduction in the numbers of students who, post treatment, would have no longer qualified for special education services based on IQ-Achievement discrepancies. Unfortunately, neither of those studies incorporated control groups so they are not being submitted as part of this response. However, in part because of FFR’s documented evidence in reducing the number of students unnecessarily placed in special education, the publisher believes FFR is an appropriate Response to Intervention (RtI) program for Tier ll and Tier lll level students performing in the bottom 15 percent (below Basic) on standardized reading tests. FFR can rapidly move Level 1 English Language Learners (ELLs), learning disabled, cognitively challenged, at-risk, and treatment resisters up the proficiency scale, to levels 2 & 3, so that they can participate in and benefit from core and remedial instruction. In support, the What Works the National Reading Research Clearing House (WWC) reported an improvement index average of 10 percentile points (Power4Kids, 2006 – one of the studies submitted as part of this response) for comprehension measures and concludes that the average effect size across the two comprehension outcomes was “large enough to be considered substantively important,” though it was not statistically significant (note: Power4Kids focused on striving readers and not the typical group of bottom 15% performing students that FFR targets). In the interest of “collecting more evidence on the program’s effects” (see attached Narrative document in PDF format), MDRC of NYC applied for an Institute of Education Sciences (IES) grant in June 2009 to fund a two year national validation randomized trial study (goal 3 reading topic) to involve a total sample of approximately 1,200 grades 3,4, & 5 students in 25 schools. The authors of Power4Kids (Torgesen et al., 2007) later reported the following outcomes in their one year follow up evaluation (note: Note: for students in the third-grade cohort, FFR (the only word level plus comprehension program) had an impact on one measure of phonemic decoding, two of the three measures of word reading accuracy and fluency, and one measure of comprehension: • Alphabetics: FFR had a statistically significant effect on one measure of phonemic decoding and one measure of word reading accuracy. The average effect size across the Word Attack was statistically significant and large enough one year later to be considered substantively important according to the WWC criteria (that is, an effect size of .33). In addition, the authors found statistically significant and large effect sizes to occur with a significant number of their 3rd grade subgroups: Students with High Baseline Word Attack Scores, Students with Low Vocabulary, and Students Eligible for Free and Reduced Lunch (effect sizes of : .62, .46 and.45 respectively). The average effect size for the Test of Word Reading Efficiency (TOWRE) for Students Eligible for Free and Reduced Lunch was statistically significant and/or large enough one year later to be considered substantively important according to the WWC criteria (that is, an effect size of .24). • Fluency. One outcome in this domain (the Oral Reading Fluency test) was reported and the authors reported a statistically significant effect for FFR. While the overall average effect size of .20 was not large enough to be considered substantively important according to WCC criteria, the authors reported that FFR did have a statistically significant effect and/or large enough effective size to be considered substantively important for two 3rd grade cohorts: those with High Baseline Word Attack Scores, and those with Low Vocabulary (effect size of .25 and .27 respectively). • Comprehension. Two outcomes were reported for this domain (WRMT–R: Passage Comprehension subtest and Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest) and again reported statistically significant effects similar to those found in the first year of the initial year report (Torgesen et al., 2006). FFR did have a statistically significant effect and/or large enough effective size in the Passage Comprehension subtest to be statistically significant and/or considered substantively important for five 3rd grade cohorts: High Baseline Word Attack, High Vocabulary, Low Baseline Word Attack with High Vocabulary, High Baseline Word Attack with High Vocabulary and Students Not Eligible for Free and Reduced Lunch (effect sizes of: .40, .36, .26, .50,and .51 respectively). The authors also reported that Failure Free Reading did have a statistically significant effect and/or large enough effective size in the Group Reading Assessment and Diagnostic Evaluation (GRADE) subtest to be statistically significant and/or considered substantively important for four 3rd grade cohorts: High Baseline Word Attack, Low Baseline Word Attack with High Vocabulary, Low Baseline Word Attack with High Vocabulary and Students Eligible for Free and Reduced Lunch (effect sizes of: .35, .42, .53, and .37 respectively). Unlike most other commercial reading interventions, FFR combines word level instruction with reading comprehension/vocabulary instruction. The program does not incorporate phonics or phonemic decoding instruction, attempt to remediate deficits, or “teach skills to mastery”. Instead, FFR’s unique non-phonic methodology compensates for the environmental, language and audio-phonological deficits that prevent so many students from reading age-level texts with fluency and comprehension success, and developing grade level vocabularies. FFR is less concerned with etiology (why a student is a “non-reader”) and more what the utilization of effective instructional strategies found to impact non-readers, such as the use of multiple exposures within multiple contexts (McCormick,1994). Students who are engaged in highly sequenced cumulative learning opportunities presented to them in a consistent multi-contextual format show significant growth. The program employs multiple exposures within multiple contexts through the use of teacher-guided language/literacy lessons, talking software and pencil & paper reinforcement activities. Included in the program are thematic Lexile-based reading Illustrated Booklets, Instructional Readers, flashcards and take home parent communication reports. The lessons in the FFR Elementary, Secondary and Life Skills Solutions – and Rapid Response to Reading Intervention offerings – are all based on the same highly structured pedagogy. Each lesson takes 45-60 minutes to deliver, and is taught in a one-to-one or small group format. Each day, the instruction begins with scripted direct teacher-led lesson that involves five steps: previewing material, listening to the instructor read, answering factual, inferential, and leading questions, reading the material, and reviewing the material. Following the teacher-led lesson, students complete a talking software session, and then pencil & paper activities, both of which are designed to extend and reinforce the teacher lesson. FFR materials are highly controlled semantically and syntactically, and provide students with sufficient repetition regardless of individual need. All content is age and developmentally appropriate content. Students rapidly acquire Dolch words and EDL Core Vocabulary in the context of simple stories they can access and relate to. The methodology is designed to immediately enable students to experience success, maximal time on task, and a sense of challenge. As a result, even secondary age “non-readers” become capable of rapidly building self-efficacy and esteem, and experiencing success and independent engagement. FFR presents these age appropriate materials within a multi-sensory format using practices based on direct instruction, scaffolding, repeated readings and meta-cognitive strategies. More importantly, Failure Free Reading has created instructional materials which incorporate the principles of cumulative learning (McCormick, 1994), and stress the therapeutic necessity of (1.) repetition within multiple instructional contexts (Gates, 1930, Hargis et. al., 1992, McCormick, 1994), (2.) control for syntax and semantics (Harper, 1979, Wigg and Semel,1980), and (3.) providing immediate performance feedback. Key to the efficacy of FFR is its instructional integration, cognitive chunking and multiple exposures within multiple contexts. Assessment is on-going and utilizes Lexile-based diagnostic prescriptive software, paper and pencil criterion referenced CBM tests measuring discrete growth in: word recognition, comprehension, fluency and changes in attitude and behavior as seen by the student, parent and teacher. Lessons and tasks within each unit frequently involve concepts from more than one strand and actively explore their interconnections and relationships. For example, each lesson presented within aThematic Series is a logical extension of the lessons that preceded it. Students are taught one paragraph at a time. Students learn the first unit on the first day, review the first unit on the second day before being introduced to the new unit for that day. On the third day, the instructor reviews days one and two before introducing the new material for day three. This process repeats until the students have mastered the entire theme. This abundant repetition in a logical and meaning based environment is what sets FFR material apart. Every single task includes teacher guided questions designed to produce deductive and inductive thinking strategies . Students are also actively engage in higher-order thinking activities to promote knowledge and growth in areas such as vocabulary and syntax. Finally, FFR is taught by certified and non-certified instructors in regular classrooms, extended day programs, pull-out situations, and lab settings. For additional information, visit www.failurefreeonline.com, or contact the program author Dr. Joseph Lockavitch at Failure Free Reading. 1-800-542-2170.
- Target Grades:
- 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
- Target Populations:
-
- Students with disabilities only
- Students with learning disabilities
- Students with intellectual disabilities
- Students with emotional or behavioral disabilities
- English language learners
- Any student at risk for academic failure
- Other: students scoring 0-15% on standardized reading tests
- Area(s) of Focus:
-
- Comprehension
- Fluency
- Vocabulary
- Spelling
- Spelling
- Where to Obtain:
- Failure Free Reading
- Initial Cost:
- $3,000.00 per subscription to digital library
- Replacement Cost:
- $100.00 per seat per year
-
The Failure Free Reading Elementary, Secondary and Life Solution (Tier lll) programs each consist of a $3,000 site-level curriculum library (containing all teacher and student print materials, all of which are non-consumable), a 4% shipping & handling fee (on just print materials), plus either online software @ $100 per twelve month subscription seat, or the standalone/network edition of the same software @ $300 per thirty six month licensed seat. Product training is available either half-day onsite ($900) or 3 hour webcast ($300). Note: for sites with more than 2 or 3 teachers using the program, additional Teacher Manuals and student materials may be needed for implementation. For sites with several students, the programs described above are also available in the Rapid Response to Reading Intervention (Tier lll) edition @ $300 per student which includes a 12 month online only software seat, with all teacher and student materials made available in blackline master CD-ROM format only. The Rapid Response to Reading Intervention (Tier ll) is a 20 hour online software only offering designed to identify students for whom Failure Free’s intensive language development approach to intervention is appropriate, and indicated.
- Staff Qualified to Administer Include:
-
- Special Education Teacher
- General Education Teacher
- Reading Specialist
- Math Specialist
- EL Specialist
- Interventionist
- Student Support Services Personnel (e.g., counselor, social worker, school psychologist, etc.)
- Paraprofessional
- Other:
- Training Requirements:
- 1-4 hours of training
-
Training is onsite, and hands-on. Each participant is required to have access to a computer (with the software installed, or available via an Internet connection), and the print materials must be onsite. The training involves each participant going completely through the placement process, and an entire instructional cycle (scripted teacher lesson, talking software, print activities) plus reporting, grouping and classroom management.
They were developed over time based on interaction with educators using the program.
- Access to Technical Support:
- Free and for-charge webinars and in person presentations by the program author are available upon request. CEU credits can be made available.
- Recommended Administration Formats Include:
-
- Individual students
- Small group of students
- Minimum Number of Minutes Per Session:
- 45
- Minimum Number of Sessions Per Week:
- 5
- Minimum Number of Weeks:
- 4
- Detailed Implementation Manual or Instructions Available:
- Yes
- Is Technology Required?
-
- Computer or tablet
- Internet connection
Program Information
Descriptive Information
Please provide a description of program, including intended use:
Failure Free Reading (FFR) is a research based, scientifically validated diagnostic/prescriptive reading intervention documented in numerous quality design research studies to improve reading outcomes for students who have not responded to regular and/or remedial reading instruction. The program’s unique combination of scaffolds enable the lowest level readers to immediately experience success with age- and grade level- appropriate passages regardless of current reading ability. FFR’s approach has consistently been shown to rapidly build comprehension, vocabulary and fluency, along with efficacy, confidence and esteem. Multi-sensory, and based on intensive language development, the print-based FFR methodology was developed in the late 1980’s by program author Dr. Joseph Lockavitch for special education and at-risk students, along with other “treatment resisters” (including those with language and/or audio phonological deficits). The software component was added in the 1990s. Over the years, numerous studies involving FFR as treatment (including 8 published studies) reported rapid and significant gains in comprehension and other measures. For example, two of the published studies reported a more than 40% reduction in the numbers of students who, post treatment, would have no longer qualified for special education services based on IQ-Achievement discrepancies. Unfortunately, neither of those studies incorporated control groups so they are not being submitted as part of this response. However, in part because of FFR’s documented evidence in reducing the number of students unnecessarily placed in special education, the publisher believes FFR is an appropriate Response to Intervention (RtI) program for Tier ll and Tier lll level students performing in the bottom 15 percent (below Basic) on standardized reading tests. FFR can rapidly move Level 1 English Language Learners (ELLs), learning disabled, cognitively challenged, at-risk, and treatment resisters up the proficiency scale, to levels 2 & 3, so that they can participate in and benefit from core and remedial instruction. In support, the What Works the National Reading Research Clearing House (WWC) reported an improvement index average of 10 percentile points (Power4Kids, 2006 – one of the studies submitted as part of this response) for comprehension measures and concludes that the average effect size across the two comprehension outcomes was “large enough to be considered substantively important,” though it was not statistically significant (note: Power4Kids focused on striving readers and not the typical group of bottom 15% performing students that FFR targets). In the interest of “collecting more evidence on the program’s effects” (see attached Narrative document in PDF format), MDRC of NYC applied for an Institute of Education Sciences (IES) grant in June 2009 to fund a two year national validation randomized trial study (goal 3 reading topic) to involve a total sample of approximately 1,200 grades 3,4, & 5 students in 25 schools. The authors of Power4Kids (Torgesen et al., 2007) later reported the following outcomes in their one year follow up evaluation (note: Note: for students in the third-grade cohort, FFR (the only word level plus comprehension program) had an impact on one measure of phonemic decoding, two of the three measures of word reading accuracy and fluency, and one measure of comprehension: • Alphabetics: FFR had a statistically significant effect on one measure of phonemic decoding and one measure of word reading accuracy. The average effect size across the Word Attack was statistically significant and large enough one year later to be considered substantively important according to the WWC criteria (that is, an effect size of .33). In addition, the authors found statistically significant and large effect sizes to occur with a significant number of their 3rd grade subgroups: Students with High Baseline Word Attack Scores, Students with Low Vocabulary, and Students Eligible for Free and Reduced Lunch (effect sizes of : .62, .46 and.45 respectively). The average effect size for the Test of Word Reading Efficiency (TOWRE) for Students Eligible for Free and Reduced Lunch was statistically significant and/or large enough one year later to be considered substantively important according to the WWC criteria (that is, an effect size of .24). • Fluency. One outcome in this domain (the Oral Reading Fluency test) was reported and the authors reported a statistically significant effect for FFR. While the overall average effect size of .20 was not large enough to be considered substantively important according to WCC criteria, the authors reported that FFR did have a statistically significant effect and/or large enough effective size to be considered substantively important for two 3rd grade cohorts: those with High Baseline Word Attack Scores, and those with Low Vocabulary (effect size of .25 and .27 respectively). • Comprehension. Two outcomes were reported for this domain (WRMT–R: Passage Comprehension subtest and Group Reading Assessment and Diagnostic Evaluation (GRADE): Passage Comprehension subtest) and again reported statistically significant effects similar to those found in the first year of the initial year report (Torgesen et al., 2006). FFR did have a statistically significant effect and/or large enough effective size in the Passage Comprehension subtest to be statistically significant and/or considered substantively important for five 3rd grade cohorts: High Baseline Word Attack, High Vocabulary, Low Baseline Word Attack with High Vocabulary, High Baseline Word Attack with High Vocabulary and Students Not Eligible for Free and Reduced Lunch (effect sizes of: .40, .36, .26, .50,and .51 respectively). The authors also reported that Failure Free Reading did have a statistically significant effect and/or large enough effective size in the Group Reading Assessment and Diagnostic Evaluation (GRADE) subtest to be statistically significant and/or considered substantively important for four 3rd grade cohorts: High Baseline Word Attack, Low Baseline Word Attack with High Vocabulary, Low Baseline Word Attack with High Vocabulary and Students Eligible for Free and Reduced Lunch (effect sizes of: .35, .42, .53, and .37 respectively). Unlike most other commercial reading interventions, FFR combines word level instruction with reading comprehension/vocabulary instruction. The program does not incorporate phonics or phonemic decoding instruction, attempt to remediate deficits, or “teach skills to mastery”. Instead, FFR’s unique non-phonic methodology compensates for the environmental, language and audio-phonological deficits that prevent so many students from reading age-level texts with fluency and comprehension success, and developing grade level vocabularies. FFR is less concerned with etiology (why a student is a “non-reader”) and more what the utilization of effective instructional strategies found to impact non-readers, such as the use of multiple exposures within multiple contexts (McCormick,1994). Students who are engaged in highly sequenced cumulative learning opportunities presented to them in a consistent multi-contextual format show significant growth. The program employs multiple exposures within multiple contexts through the use of teacher-guided language/literacy lessons, talking software and pencil & paper reinforcement activities. Included in the program are thematic Lexile-based reading Illustrated Booklets, Instructional Readers, flashcards and take home parent communication reports. The lessons in the FFR Elementary, Secondary and Life Skills Solutions – and Rapid Response to Reading Intervention offerings – are all based on the same highly structured pedagogy. Each lesson takes 45-60 minutes to deliver, and is taught in a one-to-one or small group format. Each day, the instruction begins with scripted direct teacher-led lesson that involves five steps: previewing material, listening to the instructor read, answering factual, inferential, and leading questions, reading the material, and reviewing the material. Following the teacher-led lesson, students complete a talking software session, and then pencil & paper activities, both of which are designed to extend and reinforce the teacher lesson. FFR materials are highly controlled semantically and syntactically, and provide students with sufficient repetition regardless of individual need. All content is age and developmentally appropriate content. Students rapidly acquire Dolch words and EDL Core Vocabulary in the context of simple stories they can access and relate to. The methodology is designed to immediately enable students to experience success, maximal time on task, and a sense of challenge. As a result, even secondary age “non-readers” become capable of rapidly building self-efficacy and esteem, and experiencing success and independent engagement. FFR presents these age appropriate materials within a multi-sensory format using practices based on direct instruction, scaffolding, repeated readings and meta-cognitive strategies. More importantly, Failure Free Reading has created instructional materials which incorporate the principles of cumulative learning (McCormick, 1994), and stress the therapeutic necessity of (1.) repetition within multiple instructional contexts (Gates, 1930, Hargis et. al., 1992, McCormick, 1994), (2.) control for syntax and semantics (Harper, 1979, Wigg and Semel,1980), and (3.) providing immediate performance feedback. Key to the efficacy of FFR is its instructional integration, cognitive chunking and multiple exposures within multiple contexts. Assessment is on-going and utilizes Lexile-based diagnostic prescriptive software, paper and pencil criterion referenced CBM tests measuring discrete growth in: word recognition, comprehension, fluency and changes in attitude and behavior as seen by the student, parent and teacher. Lessons and tasks within each unit frequently involve concepts from more than one strand and actively explore their interconnections and relationships. For example, each lesson presented within aThematic Series is a logical extension of the lessons that preceded it. Students are taught one paragraph at a time. Students learn the first unit on the first day, review the first unit on the second day before being introduced to the new unit for that day. On the third day, the instructor reviews days one and two before introducing the new material for day three. This process repeats until the students have mastered the entire theme. This abundant repetition in a logical and meaning based environment is what sets FFR material apart. Every single task includes teacher guided questions designed to produce deductive and inductive thinking strategies . Students are also actively engage in higher-order thinking activities to promote knowledge and growth in areas such as vocabulary and syntax. Finally, FFR is taught by certified and non-certified instructors in regular classrooms, extended day programs, pull-out situations, and lab settings. For additional information, visit www.failurefreeonline.com, or contact the program author Dr. Joseph Lockavitch at Failure Free Reading. 1-800-542-2170.
The program is intended for use in the following age(s) and/or grade(s).
Age 3-5
Kindergarten
First grade
Second grade
Third grade
Fourth grade
Fifth grade
Sixth grade
Seventh grade
Eighth grade
Ninth grade
Tenth grade
Eleventh grade
Twelth grade
The program is intended for use with the following groups.
Students with learning disabilities
Students with intellectual disabilities
Students with emotional or behavioral disabilities
English language learners
Any student at risk for academic failure
Any student at risk for emotional and/or behavioral difficulties
Other
If other, please describe:
students scoring 0-15% on standardized reading tests
ACADEMIC INTERVENTION: Please indicate the academic area of focus.
Early Literacy
Alphabet knowledge
Phonological awareness
Phonological awarenessEarly writing
Early decoding abilities
Other
If other, please describe:
Language
Grammar
Syntax
Listening comprehension
Other
If other, please describe:
Reading
Phonics/word study
Comprehension
Fluency
Vocabulary
Spelling
Other
If other, please describe:
Mathematics
Concepts and/or word problems
Whole number arithmetic
Comprehensive: Includes computation/procedures, problem solving, and mathematical concepts
Algebra
Fractions, decimals (rational number)
Geometry and measurement
Other
If other, please describe:
Writing
Spelling
Sentence construction
Planning and revising
Other
If other, please describe:
BEHAVIORAL INTERVENTION: Please indicate the behavior area of focus.
Externalizing Behavior
Verbal Threats
Property Destruction
Noncompliance
High Levels of Disengagement
Disruptive Behavior
Social Behavior (e.g., Peer interactions, Adult interactions)
Other
If other, please describe:
Internalizing Behavior
Anxiety
Social Difficulties (e.g., withdrawal)
School Phobia
Other
If other, please describe:
Acquisition and cost information
Where to obtain:
- Address
- Phone Number
- Website
Initial cost for implementing program:
- Cost
- $3000.00
- Unit of cost
- subscription to digital library
Replacement cost per unit for subsequent use:
- Cost
- $100.00
- Unit of cost
- seat
- Duration of license
- year
Additional cost information:
Describe basic pricing plan and structure of the program. Also, provide information on what is included in the published program, as well as what is not included but required for implementation (e.g., computer and/or internet access)
The Failure Free Reading Elementary, Secondary and Life Solution (Tier lll) programs each consist of a $3,000 site-level curriculum library (containing all teacher and student print materials, all of which are non-consumable), a 4% shipping & handling fee (on just print materials), plus either online software @ $100 per twelve month subscription seat, or the standalone/network edition of the same software @ $300 per thirty six month licensed seat. Product training is available either half-day onsite ($900) or 3 hour webcast ($300). Note: for sites with more than 2 or 3 teachers using the program, additional Teacher Manuals and student materials may be needed for implementation. For sites with several students, the programs described above are also available in the Rapid Response to Reading Intervention (Tier lll) edition @ $300 per student which includes a 12 month online only software seat, with all teacher and student materials made available in blackline master CD-ROM format only. The Rapid Response to Reading Intervention (Tier ll) is a 20 hour online software only offering designed to identify students for whom Failure Free’s intensive language development approach to intervention is appropriate, and indicated.Program Specifications
Setting for which the program is designed.
Small group of students
BI ONLY: A classroom of students
If group-delivered, how many students compose a small group?
2-6Program administration time
- Minimum number of minutes per session
- 45
- Minimum number of sessions per week
- 5
- Minimum number of weeks
- 4
- If intervention program is intended to occur over less frequently than 60 minutes a week for approximately 8 weeks, justify the level of intensity:
Does the program include highly specified teacher manuals or step by step instructions for implementation?- Yes
BEHAVIORAL INTERVENTION: Is the program affiliated with a broad school- or class-wide management program?-
If yes, please identify and describe the broader school- or class-wide management program: -
Does the program require technology? - Yes
-
If yes, what technology is required to implement your program? -
Computer or tablet
Internet connection
Other technology (please specify)
If your program requires additional technology not listed above, please describe the required technology and the extent to which it is combined with teacher small-group instruction/intervention:
Online software requires Internet-enabled computer with MAC OS8 or newer, or PC Windows XP or newer; Flash; audio card and speakers. Optional standalone/network edition of same software requires same hardware but no Internet connection.
Training
- How many people are needed to implement the program ?
Is training for the instructor or interventionist required?- Yes
- If yes, is the necessary training free or at-cost?
- At-cost
Describe the time required for instructor or interventionist training:- 1-4 hours of training
Describe the format and content of the instructor or interventionist training:- Training is onsite, and hands-on. Each participant is required to have access to a computer (with the software installed, or available via an Internet connection), and the print materials must be onsite. The training involves each participant going completely through the placement process, and an entire instructional cycle (scripted teacher lesson, talking software, print activities) plus reporting, grouping and classroom management.
What types or professionals are qualified to administer your program?
General Education Teacher
Reading Specialist
Math Specialist
EL Specialist
Interventionist
Student Support Services Personnel (e.g., counselor, social worker, school psychologist, etc.)
Applied Behavior Analysis (ABA) Therapist or Board Certified Behavior Analyst (BCBA)
Paraprofessional
Other
If other, please describe:
- Does the program assume that the instructor or interventionist has expertise in a given area?
-
No
If yes, please describe:
Are training manuals and materials available?- Yes
-
Describe how the training manuals or materials were field-tested with the target population of instructors or interventionist and students: - They were developed over time based on interaction with educators using the program.
Do you provide fidelity of implementation guidance such as a checklist for implementation in your manual?-
Can practitioners obtain ongoing professional and technical support? -
Yes
If yes, please specify where/how practitioners can obtain support:
Free and for-charge webinars and in person presentations by the program author are available upon request. CEU credits can be made available.
Summary of Evidence Base
- Please identify, to the best of your knowledge, all the research studies that have been conducted to date supporting the efficacy of your program, including studies currently or previously submitted to NCII for review. Please provide citations only (in APA format); do not include any descriptive information on these studies. NCII staff will also conduct a search to confirm that the list you provide is accurate.
Study Information
Study Citations
Torgesen, J., Myers, D., Schirm, A., Stuart, E., Vartivarian, S., Mansfield, W., Stancavage, F., Durno, D., Javorsky, R. & Haan, C. (2007). National Assessment of Title I Final Report Volume II: Closing the Reading Gap: Findings from a Randomized Trial of Four Reading Interventions for Striving Readers. Institute of Education Sciences.
Participants
- Describe how students were selected to participate in the study:
- Within each of 50 schools, 3rd and 5th grade students were identified as struggling readers by their teachers.
- Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):
- The students identified by their teachers as struggling readers were tested and were eligible for the study if they scored at or below the 30th percentile on the combination of the Sight Word Efficiency (SWE) and phonological decoding subtests of the Test of Word Reading Efficiency (TOWRE), and at or above the 5th percentile on the Peabody Picture Vocabulary Test (PPVT).
-
ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:
- below the 30th percentile on local or national norm, or
- identified disability related to the focus of the intervention?
- %
-
BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:
- emotional disability label,
- placed in an alternative school/classroom,
- non-responsive to Tiers 1 and 2, or
- designation of severe problem behaviors on a validated scale or through observation?
- %
- Specify which condition is the submitted intervention:
- Failure Free Reading
- Specify which condition is the control condition:
- Control group students received their school’s normal classroom and individual instruction. Competing interventions were: Corrective Reading, SpellRead P.A.T., and Wilson Reading.
- If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):
Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.
Grade Level
Demographic | Program Number |
Control Number |
Effect Size: Cox Index for Binary Differences |
---|---|---|---|
Age less than 1 | |||
Age 1 | |||
Age 2 | |||
Age 3 | |||
Age 4 | |||
Age 5 | |||
Kindergarten | |||
Grade 1 | |||
Grade 2 | |||
Grade 3 | 47.4% | 36.9% | 0.25 |
Grade 4 | |||
Grade 5 | 52.6% | 63.1% | 0.25 |
Grade 6 | |||
Grade 7 | |||
Grade 8 | |||
Grade 9 | |||
Grade 10 | |||
Grade 11 | |||
Grade 12 |
Race–Ethnicity
Demographic | Program Number |
Control Number |
Effect Size: Cox Index for Binary Differences |
---|---|---|---|
African American | 22.4% | 17.5% | 0.19 |
American Indian | |||
Asian/Pacific Islander | |||
Hispanic | |||
White | 77.6% | 82.5% | 0.19 |
Other |
Socioeconomic Status
Demographic | Program Number |
Control Number |
Effect Size: Cox Index for Binary Differences |
---|---|---|---|
Subsidized Lunch | 44.0% | 46.6% | 0.07 |
No Subsidized Lunch | 56.0% | 53.4% | 0.07 |
Disability Status
Demographic | Program Number |
Control Number |
Effect Size: Cox Index for Binary Differences |
---|---|---|---|
Speech-Language Impairments | |||
Learning Disabilities | |||
Behavior Disorders | |||
Emotional Disturbance | |||
Intellectual Disabilities | |||
Other | |||
Not Identified With a Disability |
ELL Status
Demographic | Program Number |
Control Number |
Effect Size: Cox Index for Binary Differences |
---|---|---|---|
English Language Learner | |||
Not English Language Learner |
Gender
Demographic | Program Number |
Control Number |
Effect Size: Cox Index for Binary Differences |
---|---|---|---|
Female | 52.6% | 53.4% | 0.00 |
Male | 47.4% | 46.6% | 0.00 |
Mean Effect Size
For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences between groups in the descriptions below, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not demographic characteristics, please describe the results of those analyses here.
Design
- What method was used to determine students' placement in treatment/control groups?
- Random
- Please describe the assignment method or the process for defining treatment/comparison groups.
- We implemented the intervention in the Allegheny Intermediate Unit (AIU), located just outside Pittsburgh, Pennsylvania. The AIU consists of 42 school districts and about 125 elementary schools. Not all schools that agreed to participate in the study had sufficient numbers of eligible third- and fifth-grade students, and some schools had only third or fifth grade, not both. Thus, we partnered some schools to form “school units” such that each school unit would have two third-grade and two fifth-grade instructional groups consisting of three students per instructional group. From a pool of 52 schools, we formed 32 school units, and randomly assigned the 32 school units to the four interventions, within four strata defined by the percentage of students eligible for free or reduced-price school lunch. One school unit (consisting of two schools) dropped out of the study after randomization, but before it learned of its random assignment, leaving 31 school units and 50 schools in the study
-
What was the unit of assignment? - Other
- If other, please specify:
- Schools, Teachers, Students
-
Please describe the unit of assignment: -
What unit(s) were used for primary data analysis? -
Schools
Teachers
Students
Classes
Other
If other, please specify:
-
Please describe the unit(s) used for primary data analysis:
Fidelity of Implementation
- How was the program delivered?
-
Individually
Small Group
Classroom
If small group, answer the following:
- Average group size
- 3
- Minimum group size
- 3
- Maximum group size
- 3
What was the duration of the intervention (If duration differed across participants, settings, or behaviors, describe for each.)?
- Weeks
- 18.00
- Sessions per week
- 5.00
- Duration of sessions in minutes
- 60.00
- What were the background, experience, training, and ongoing support of the instructors or interventionists?
- The teachers were randomly selected from volunteers at each school. They received approximately 70 hours of professional development and support during the year.
- Describe when and how fidelity of treatment information was obtained.
- Trainers rated teachers twice: in the fall (at the end of the practice period) and in the spring (near the end of the intervention period). The trainers provided ratings on eight dimensions of the teacher’s delivery of the program. The first five dimensions specifically address intervention fidelity while the remainder deal with general teacher quality.
- What were the results on the fidelity-of-treatment implementation measure?
- The below summarizes the ratings on eight dimensions of program delivery. The ratings used a sevenpoint scale ranging from 1 = unsatisfactory performance through 3 = satisfactory performance to 7 = expert performance. The average ratings on all eight dimensions in both fall and spring generally ranged from about 4.0 to 6.8—well above the satisfactory (3) level.
- Was the fidelity measure also used in control classrooms?
Measures and Results
Measures Broader :
Targeted Measure | Reverse Coded? | Reliability | Relevance | Exposure |
---|
Broader Measure | Reverse Coded? | Reliability | Relevance | Exposure |
---|
Administrative Data Measure | Reverse Coded? | Relevance |
---|
Effect Size
Effect size represents the how much performance changed because of the intervention. The larger the effect size, the greater the impact participating in the intervention had.
According to guidelines from the What Works Clearinghouse, an effect size of 0.25 or greater is “substantively important.” Additionally, effect sizes that are statistically significant are more trustworthy than effect sizes of the same magnitude that are not statistically significant.
Effect Size Dial
The purpose of the effect size dial is to help users understand the strength of a tool relative to other tools on the Tools Chart.
- The range represents where most effect sizes fall within reading or math based on effect sizes from tools on the Tools Chart.
- The orange pointer shows the average effect size for this study.
Targeted Measures (Full Sample)
Average Reading Effect Size
Measure | Sample Type | Effect Size |
---|---|---|
Average across all targeted measures | Full Sample | 0.08 |
* = p ≤ 0.05; † = Vendor did not provide necessary data for NCII to calculate effect sizes. |
Broader Measures (Full Sample)
Average Reading Effect Size
Measure | Sample Type | Effect Size |
---|---|---|
Average across all broader measures | Full Sample | -0.03 |
* = p ≤ 0.05; † = Vendor did not provide necessary data for NCII to calculate effect sizes. |
Administrative Measures (Full Sample)
Measure | Sample Type | Effect Size |
---|---|---|
Average across all admin measures | Full Sample | -- |
* = p ≤ 0.05; † = Vendor did not provide necessary data for NCII to calculate effect sizes. |
Targeted Measures (Subgroups)
Measure | Sample Type | Effect Size |
---|---|---|
* = p ≤ 0.05; † = Vendor did not provide necessary data for NCII to calculate effect sizes. |
Broader Measures (Subgroups)
Measure | Sample Type | Effect Size |
---|---|---|
* = p ≤ 0.05; † = Vendor did not provide necessary data for NCII to calculate effect sizes. |
Administrative Measures (Subgroups)
Measure | Sample Type | Effect Size |
---|---|---|
* = p ≤ 0.05; † = Vendor did not provide necessary data for NCII to calculate effect sizes. |
- For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.
- Please explain any missing data or instances of measures with incomplete pre- or post-test data.
- If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:
- Describe the analyses used to determine whether the intervention produced changes in student outcomes:
- The authors used a randomized blocks experimental design with random assignment carried out at two levels (32 school units randomly assigned to the four interventions within blocking strata determined by the percentage of students eligible for free or reduced-price school lunch; and within schools, eligible 3rd and 5th grade students to treatment or control). The main objective of was to estimate the impacts of the four interventions on students’ reading skills, specifically, the impacts of the four interventions combined, the three word level interventions combined, and each of the four individual interventions for not only all third-grade and all fifth-grade students eligible for the interventions, but also several key subgroups of students. For a detailed presentation of the findings, and description of the estimation methods (with technical and contextual issue) see Chapter lV pages 53-68.
Additional Research
- Is the program reviewed by WWC or E-ESSA?
- WWC & E-ESSA
- Summary of WWC / E-ESSA Findings :
What Works Clearinghouse Review
Beginning Reading Protocol
Effectiveness: Failure Free Reading was found to have no discernible effects on alphabetics and fluency, and potentially positive effects on comprehension.
Studies Reviewed: 1 study meets standards out of 1 study total
Evidence for ESSA
Qualifying studies found no significant positive outcomes for Elementary reading. No studies met inclusion requirements for Secondary reading.
- How many additional research studies are potentially eligible for NCII review?
- 0
- Citations for Additional Research Studies :
Data Collection Practices
Most tools and programs evaluated by the NCII are branded products which have been submitted by the companies, organizations, or individuals that disseminate these products. These entities supply the textual information shown above, but not the ratings accompanying the text. NCII administrators and members of our Technical Review Committees have reviewed the content on this page, but NCII cannot guarantee that this information is free from error or reflective of recent changes to the product. Tools and programs have the opportunity to be updated annually or upon request.