Lexia Core5 Reading
Study: Schechter et al. (2015)

Describe how students were selected to participate in the study:

The study was conducted in an urban elementary school in western Massachusetts. Students in first grade classes and second grade classes participated in this study. For each grade, one class was randomly assigned to the treatment group and the second class was assigned to the control group.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

The school was identified as having students that were at risk for academic failure and was targeted for use of the intervention due to the poor performance on state reading tests. For example, in 2012 the percent of students who scored as Proficient or Advanced on the 3rd and 5th grade reading tests were 30% and 32% respectively.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Lexia Reading Core5 is the submitted program

Specify which condition is the control condition:

The control condition did not use Lexia Reading Core5.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

The randomization procedure was not described in the study report. A computer program, Excel, was used to generate a number between 0 and 1 for each pair of classes (one pair per grade). The lower number was assigned to the control group and the higher number was assigned to the treatment group.

What was the unit of assignment?

Teachers

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Teachers in the treatment and control classes had comparable qualifications. Three teachers have master degrees in education (one treatment, two control) and one teacher has a Bachelor of Science degree. The two treatment teachers had six and 12 years of teaching experience, and the two control teachers had nine and 15 years of teaching experience. Teachers took part in a half-day Core5 orientation session prior to implementation where they were trained in best practices for integrating the online learning and offline instructional materials of Core5 into their classroom instruction. Each treatment classroom had six computers and students used the online program as a center activity following a rotating schedule. Teachers were instructed to have students use the online program in accordance with recommended minutes (20 to 100 per week, depending on risk) based on the Prescriptions of Intensity indicated in myLexia.com. Teachers received a teacher training guide that they could refer back to at any time. In addition, all staff members had direct contact information for the research coordinator, as well as the Lexia Customer Support team to support them with any questions or issues that came up, available Monday-Friday 8am-6pm EST.

Describe when and how fidelity of treatment information was obtained.

As part of ongoing monitoring, two members of the research team visited the treatment classes in February 2013 to observe students using the online program. During the session, twelve students were observed in each classroom. Independent observations were made at two time points in a session. In addition to observations and data analysis, interviews were conducted to assess use of Core5 as part of classroom instruction. In addition to observation and teacher interviews, the Core5 online component tracks the number of sessions completed for each student (number of sessions and the length of sessions).

What were the results on the fidelity-of-treatment implementation measure?

The observations agreed upon by both raters were as follows: 15 cases in which students were engaged with the program, four cases in which students were seated but not engaged, and two cases in which students were not seated. Raters disagreed on three cases, where students were seated but it was unclear if they were engaged with the program. For the first grade class, there was 88% (21/24) agreement across time points. For the second grade class there was 100% (24/24) agreement that all students were engaged with the program. Interviews revealed that both teachers of the treatment classrooms reported that the online program was a key center activity and they used the Lexia Lessons when students were struggling with specific skills in the online program. The first grade treatment teacher said she printed out lessons (with directions) for parents to use at home. The paper and pencil tasks (Skill Builders) were used as morning activities in the first grade class and as homework for second graders. The second grade teacher reported that when all students had mastered a skill in the online activities, she used that information to direct her whole class instruction to other skill areas. Computer logs indicated that students showed strong use of Core5’s online program: their average login time was 85 minutes per week. Looking across all student sessions, the minimum and maximum times were 28 and 203 minutes per week, respectively. These strong use patterns resulted in over 90% of students meeting usage recommendations for at least three months and 62% met usage recommendations for five or more months.

Was the fidelity measure also used in control classrooms?

The fidelity measure was not applicable to the control classrooms because the measure determines students’ engagement with their online independent practice time with Core5 and not their general engagement with the LEAD21 curriculum. All treatment and control classes used the same curriculum for their core ELA instruction. The school had been using the curriculum for a few years before this study.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

The GRADE subtests scores can be organized into a Total Test score and domain scores in Vocabulary and Comprehension. These three combined scores were used in the analyses. One-sample t-tests revealed that both groups showed significant gains in Total Test scores: treatment (t(44) = 7.50, p < .01); control (t(37) = 5.10, p < .01). However, the extent of gain was larger for the treatment group (15.6) than the control group (9.0). An independent sample t-test showed that the difference in gain scores favoring the treatment group was significant (t(81) = 2.38, p = .02). In addition, an analysis of covariance comparing Total Test scores at post-test using Total Test pretest scores as covariates confirmed the significant group effect (F(1,80) = 5.23, p = .03). One-sample t-tests were also used to examine standard score gains in each domain separately. The treatment group showed significant gains in Vocabulary (t(44) = 4.51, p < .01) and Comprehension (t(44) = 6.59, p < .01). The control group also showed significant gains in both domains – Vocabulary (t(37) = 6.42, p < .01) and Comprehension (t(37) = 3.24, p < .01). The two groups showed similar gains in Vocabulary (treatment: 11.9; control: 10.6), whereas gains were greater for the treatment group (14.2) than for the control group (7.0) in Comprehension. Independent sample t-tests showed no significant group difference in Vocabulary (t(81) = 0.42, p = .67) but a significant group difference in Comprehension (t(81) = 2.33, p = .02). Analyses of covariance comparing post-test scores using pretest scores as covariates confirmed the significant group effect in Comprehension: F(1,80) = 4.89, p = .03. Effect sizes comparing mean gain scores for the two groups were .09 and .52 in Vocabulary and Comprehension, respectively. While the effect size in Vocabulary is low, the effect size in Comprehension is moderate.