Lexia Core5 Reading
Study: Macaruso et al. (2020)

Describe how students were selected to participate in the study:

Administrators in three charter schools had concerns about the less proficient readers in their schools and decided to make a change to their ELA curriculum and adopt Lexia Core5 Reading. The administrators were also willing to have their students serve as participants in this study.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

One of the intervention schools had a high percentages of Hispanic students (69%), another school had a high percentage of students from low socioeconomic households (63%), and the third school had high percentages of both (Hispanic: 72%; low socioeconomic households: 80%). These demographic characteristics are known indicators of students at risk for academic failure.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Three schools engaged in using Lexia Core5 Reading combined with a standard ELA curriculum served as the intervention condition.

Specify which condition is the control condition:

Three schools engaged in using a standard ELA curriculum (without Lexia Core5 Reading) served as the control condition. The standard ELA curriculum used in all six schools included Wonders in grades K - 2, and both Wonders and Reading Plus in grades 3 - 5.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Systematic

Please describe the assignment method or the process for defining treatment/comparison groups.

As indicated in the published study, administrators in the charter school network chose which schools would use Core5 and – understandably – selected schools that had the lower performing readers. So, given that an RCT was not possible, we decided on a quasi-experimental design, comparing treatment schools with comparable control schools in the network. However, none of the non-Core5 schools in the network available to us had students with pretest scores similar to treatment schools. To maintain the spirit of a quasi-experimental design, we carefully selected control schools comparable to the treatment schools in terms of key demographic characteristics. And as a further control, taking into account the discrepancy in pretest scores across groups, we ran an ANCOVA comparing groups at posttest using pretest scores as a covariate.

What was the unit of assignment?

Schools

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Teachers in the intervention schools were using Lexia Reading Core5 for the first time. To support implementation of Core5, these teachers participated in Lexia’s Implementation Support Service Package, which contains three training modules. The first is a “launch” module which instructs teachers on how to efficiently get started with Core5 and reviews key components of Core5 including its scope-and-sequence and instructional materials. The second module called “data coaching” addresses progress monitoring in Core5, identifying students at-risk for reading difficulties, and planning instruction to target skill gaps. The third module called “instructional materials” focuses on implementing Core5 as part of a blended learning model and looks in more depth at the instructional materials (e.g., Lexia Lessons) available for teachers when planning offline lessons. Two treatment schools received the launch and data coaching modules, and the third treatment school received the data coaching and instructional materials modules.

Describe when and how fidelity of treatment information was obtained.

Fidelity of treatment was identified mainly through tracking students' minutes using the online component of Lexia Core5 Reading.

What were the results on the fidelity-of-treatment implementation measure?

Students used the online component of Lexia Core5 Reading for an average of 78 minutes per week. This average is close to the maximum usage recommendation (80 minutes per week) for the program.

Was the fidelity measure also used in control classrooms?

No

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

Students with missing MAP data points at posttest were omitted from the data set and left unanalyzed. The proposition of students omitted was 2.4% for the intervention condition and 1.6% for the control condition.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

An analysis of covariance was conducted with main effects of treatment group, pretest, grade and ethnic category, and two-way interactions between treatment group and grade and between treatment group and ethnic category. The dependent variable was posttest RIT scores. The analysis revealed a significant main effect of treatment group but there were no significant interactions between treatment group and grade and between treatment group and ethnic category. That is, the treatment effect applied equally across grades and across students in different ethnic categories. Thus, separate analyses for each ethnic group were not carried out.