Fast ForWord Language Series
Study: Slattery (2003)

Describe how students were selected to participate in the study:

The study was conducted at Freemansburg Elementary School, Bethlehem Area School District, Freemansburg, Pennsylvania. Study participants were selected from the 62 students in grades three, four, or five who were reading below grade level. From this population, 30 students were randomly assigned to the Fast ForWord treatment group and 30 students were randomly assigned to the no-treatment control group.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

The student population of Freemansburg Elementary School is multicultural, with a high percentage of low socioeconomic status families and a high percentage of Limited English Proficient students. The free or reduced-price lunch rate at Freemansburg Elementary School is 61%. Given the student population, the students attending Freemansburg Elementary are considered at-risk within the district. In addition to these risk factors that affect the school’s population in general, the students selected to participate in the study were all reading below grade level, and were all eligible for Title One services.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The treatment group used the Fast ForWord Language Series (specifically, the Fast ForWord Language product).

Specify which condition is the control condition:

The control group was not provided with Fast ForWord or any other intervention during the study.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Thirty students, randomly selected from that population of students in grades three, four, and five, participated in the Fast ForWord program during the after-school program at Freemansburg Elementary School. This random selection of participants yielded eight third graders, seven fourth graders, and fifteen fifth graders. Four of these students had both a special education identification of specific learning disability as well as speech and language impaired. Ten of these students had a full program of Reading Recovery- a research-based one-to-one early intervention edification program for the lowest achieving first graders to build a strong foundation of reading and writing strategies – in first grade. Twenty of these students had attended preschool. Thirty students randomly selected from that same population of students in grades three, four and five, served as the control group. This random selection of the control group yielded seven third graders, nine fourth graders, and fourteen fifth graders. Three of these student shad both a special education identification of specific learning disability as well as speech and language impaired, and three of these students had a special education identification of specific learning disability. Twenty of these students had a full program of Reading Recovery in first grade. Nineteen of these students had attended preschool.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Two qualified coaches were present in the computer lab daily during the intervention sessions. The coaches were Freemansburg Elementary School staff members, including the academic support teacher, reading specialist, speech therapist, and two first-grade teachers. The same staff members served as coaches for all three of the small groups receiving the intervention.

Describe when and how fidelity of treatment information was obtained.

Fast ForWord is an individually adaptive computer-based intervention. Consequently, participation rate and progress through the content are the two primary methods of evaluating the fidelity of the implementation. Instructors are primarily tasked with providing a focused training environment and helping students use the exercises properly – strong completion rates indicate that both students and lab instructors are implementing with fidelity.

What were the results on the fidelity-of-treatment implementation measure?

Fidelity of treatment implementation was measured in terms of student compliance to the recommended training schedule, and in terms of meeting a criterion for product completion. Compliance scores were calculated as follows: (actual training minutes/ expected training minutes) *10. This calculation yields a score between 1 and 10 for each student, with acceptable compliance (per the program publisher) indicated by scores of 8.0 and above. Of the 30 students in the treatment group, 70% attained acceptable compliance scores. Acceptable Compliance N=21; Percentage=70%. Not Acceptable Compliance N=9; Percentage=30%. The criterion set for acceptable completion was 80% through at least four of the seven exercises in the product. For each exercise, a percent through score is calculated based on how far a participant has advanced through the levels of that exercise. Of the 30 students in the treatment group, 77% attained the criterion for product completion. Acceptable Completion N=23; Percentage= 77%. Not Acceptable Completion N=7; Percentage=23%. During this study, compliance was a primary consideration and the coaches ensured that the participants adhered to the training schedule when present. At the time of this study, Scientific Learning considered a compliance score of 8.0 - 10.0 to be acceptable (these calculations have been revised over time). As the report states, 21 (70%) of the students reached this level of compliance indicating that, by the Scientific Learning’s standards, the implementation was satisfactory for 70% of the students. Seven of the nine remaining students missed significant time due to illness, but otherwise had satisfactory participation – their time out sick means their participation numbers do not reflect their true usage behavior. Only one had a behavioral problem that prevented satisfactory product implementation. Therefore, by Scientific Learning’s standards, 97% (29 out of 30) of the students had satisfactory product use.

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

• Paired samples t-tests were conducted to compare performance of the two groups at pre-test and at post-test on each measure. • Additional paired samples t-tests were conducted to evaluate how much the students’ raw scores changed from pre-test to post-test, looking at each group and each measure seperately. • Analyses of covariance (ANCOVA) were performed using pre-test scores as the covariate. • For all analyses, p < 0.05 was used as the criterion for significance.