Lexia Core5 Reading
Study: Macaruso & Rodman (2009)

Describe how students were selected to participate in the study:

Students were enrolled in a remedial reading program with sixth and seventh grade students in a public middle school in St. George, Utah. Students with IQ score 70 or below were omitted from analyses. Three other students were also omitted from analysis: two suffered from a traumatic brain injury and the third had a diagnosis of autism.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Students were originally selected for the program based on low scores on the Developmental Reading Assessment, coupled with teacher referrals. The Developmental Reading Assessment assesses reading accuracy, fluency, and comprehension.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The teacher selected at random two classes to be the treatment classes. In addition to the curriculum used in the class (Language! combined with Read Right in small groups), these classes used Lexia Reading’s Strategies for Older Students for 20-30 minutes per session, two or three times per week.

Specify which condition is the control condition:

The control class received all instruction with Language! combined with Read Right in a whole-class format (not small groups, which was required to accommodate the Lexia Reading instruction). There were no obvious differences between classes in amount of direct instruction time with the core curriculum.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

The sample consisted of 47 sixth-grade and seventh-grade students attending a public middle school in St George, Utah. The students were enrolled in a remedial reading programme, which was taught by the same teacher in three sessions. Students were originally selected for the programme on the basis of low scores on the Developmental Reading Assessment (Beaver and Carter 2003), coupled with teacher referrals. The Developmental Reading Assessment assesses reading accuracy, fluency and comprehension. Students are asked to read passages aloud and provide written responses to comprehension questions. Students who met the criteria for remedial instruction were randomly placed in one of the three classes. All classes were determined by the teacher to have comparatively similar skill distributions among students. The teacher then selected at random two classes to be treatment classes, and designated the third class as a control class. There were 32 students in the treatment classes and 15 students in the control class.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

The teacher had used the Lexia S.O.S. program during the previous school year and thus was familiar with the program and management of CAI in the classroom.

Describe when and how fidelity of treatment information was obtained.

Lexia Reading software provided specific information about the amount of time and when students used the software.

What were the results on the fidelity-of-treatment implementation measure?

Treatment students averaged 50 sessions over the school year. Lexia Reading has established 45 sessions over the school year as a minimum use criterion for effective implementation. This criterion is based on research findings comparing reading gains of students who have or have not shown adequate use (see Macaruso & Hook, 2007). Of the 28 students in the treatment group, 75% met the 45 session criterion. Sessions were missed due to absences, school-wide activities, and students attending outside services such as speech therapy.

Was the fidelity measure also used in control classrooms?

Study measures are classified as targeted, broader, or administrative data according to the following definitions:

• Targeted measures
  Assess outcomes, such as competencies or skills that the program was directly targeted to improve.
  • In the academic domain, targeted measures typically are not the very items taught but rather novel items structured similarly to the content addressed in the program. For example, if a program taught word-attack skills, a targeted measure would be decoding of pseudo words. If a program taught comprehension of cause-effect passages, a targeted measure would be answering questions about cause-effect passages structured similarly to those used during intervention, but not including the very passages used for intervention.
  • In the behavioral domain, targeted measures evaluate aspects of external or internal behavior the program was directly targeted to improve and are operationally defined.

• Broader measures
  Assess outcomes that are related to the competencies or skills targeted by the program but not directly taught in the program.
  • In the academic domain, if a program taught word-level reading skill, a broader measure would be answering questions about passages the student reads. If a program taught calculation skill, a broader measure would be solving word problems that require the same kinds of calculation skill taught in the program.
  • In the behavioral domain, if a program taught a specific skill like on-task behavior in one classroom, a broader measure would be academic performance in that setting or on-task behavior in another setting.

• Administrative data measures apply only to behavioral intervention tools and are measures such as office discipline referrals (ODRs) and graduation rates which do not have psychometric properties as do other, more traditional targeted or broader measures.

Click here for more information on effect size.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Analyses of covariance comparing post-test means with pre-test means serving as covariates confirmed a significant difference on the Word Attack subtest.