Words Their Way: Word Study In Action Developmental Model
Study: Eddy et al. (2011)

Describe how students were selected to participate in the study:

Students were initially screened with scores on state standardized tests. Of those students screened, students were selected to participate if they scored at or below the 30th national percentile on the AIMSweb R-CBM screening diagnostic reading fluency assessment. (Pages 16-17)

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Students were identified through standardized test scores and other teacher recommendations; students qualified for participation in the study based on their initial AIMSweb R-CBM reading fluency scores.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The program used for the treatment condition was Words Their Way: Word Study in Action Developmental Model (Page 17)

Specify which condition is the control condition:

Control students used a variety of other phonics, spelling, and vocabulary programs, depending on the school site. Some control students were considered part of a “true control” group in which they stayed in their regular language arts classroom and did not receive any intervention program that was comparable to Words Their Way. However, other students used programs such as Read Naturally, Passports, PLAID Phonics, Voyager, Literacy Place, Leveled Literacy, and Phonics for Reading. (Page 17)

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

The Words Their Way study was conducted during the 2010-2011 school year. The efficacy study was designed as a Randomized Controlled Trial (RCT) in which qualified students were randomly assigned to either the treatment group, using the Words Their Way program or a control group (using the existing reading intervention program, if any, at their schools or continuing with business as usual). Teachers and their students used their respective language arts programs in their classes for the duration of the 2010-11 school year.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Intervention teachers taught reading for an average of 11.8 years. 16 of 23 teachers had received their Master’s degree. Interventionists received an initial training and a follow-up training covering the Words Their Way program. Interventionists were also provided with trainers’ contact information and could contact them at any time to receive additional support or ask any questions about the program.

Describe when and how fidelity of treatment information was obtained.

The fidelity of implementation was achieved by observing each classroom twice during the study, as well as teacher's completion of online weekly logs.

What were the results on the fidelity-of-treatment implementation measure?

The average number of minutes teachers were able to implement every week ranged from 74-138 minutes. Teachers were able to implement, on average, 104% of the recommended time each week, demonstrating that most teachers implemented more than expected. On average, teachers covered 20.7 sorts. Additionally teachers were asked to self-rate their level of adherence to the program on a scale of 1-9 each week (1=not at all, 5=somewhat, 9=fully). The average self-reported degree of implementation was 6.7.

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

A combination of hierarchical linear modeling (HLM), repeated measures ANOVA, and independent t-tests were used to determine the difference between the control group and the treatment group. (Page 45)