Learning Strategies Curriculum: LINCS Vocabulary Strategy
Study: Harris, Schumaker, & Deshler (2011)

Describe how students were selected to participate in the study:

Participants were 230 public school ninth graders who were regularly enrolled in nine inclusive English classes, including students with and without disabilities. Three of the classes were randomly selected to receive instruction in the Vocabulary LINCing Strategy. Three classes were randomly selected to receive instruction in the Word Mapping Strategy, and three of the classes were randomly selected as the normative comparison classes. The students' parents signed informed consent forms to allow the students' participation.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

The students with disabilities had active IEPs. After informed consent was obtained, their IEPs and test scores were reviewed to determine whether the students fit the criteria for having a disability. (Their standardized achievement and IQ data were collected from school records to facilitate this process.)

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

10.0%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The three classes that received instruction in the Vocabulary LINCing Strategy (the submitted intervention) was the experimental group (79 students).

Specify which condition is the control condition:

The three classes that received "business as usual" vocabulary instruction was the comparison condition (72 students).

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

The three classes that received instruction in the Word Mapping Strategy took part in the third condition (79 students).

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

The nine classes were each randomly assigned to one of the three groups ( the Vocabulary LINCing, Word Mapping, or control groups).

What was the unit of assignment?

Classes

If other, please specify:

Please describe the unit of assignment:

The classes were public school ninth-grade English classes in which students with and without disabilities were enrolled and expected fully to engage in the instruction.

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

Students' scores on vocabulary assessments were used in the data analyses.

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

The instructor of the Vocabulary LINCing Strategy (and the Word Mapping Strategy) was a graduate student who was completing her Ph.D. She had 10 years of teaching experience at the secondary level. Her advisors provided her support.

Describe when and how fidelity of treatment information was obtained.

The fidelity of treatment information was obtained for the Vocabulary LINCing Strategy instruction with a checklist that listed the instructional practices associated with teaching the strategy. Each item on the checklist was a teacher behavior. An observer used the checklists during class sessions in which the strategy was taught. The sessions were also videotaped for reliability purposes.

What were the results on the fidelity-of-treatment implementation measure?

The primary scorer and the reliability observer for the fidelity data were blind to the purpose of the study. Items on the fidelity checklist were compared item-by-item. The scorers agreed on 111 out of 120 opportunities to agree for 93% agreement. (The range on individual lessons was 80% to 100%.) Results indicated that the instructor implemented an average of 96% of the required instructional behaviors across the strategy lessons.

Was the fidelity measure also used in control classrooms?

No, the control classrooms were engaged in business-as-usual English classes. The vocabulary instruction on the Vocabulary LINCing fidelity checklists would not have pertained to those classes.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Pretest scores were compared across the three groups using ANOVAs. No statistically significant difference was found between the pretest scores for any of the three tests. To determine whether there were any differences across the three groups regarding their demographic data and standardized test data, chi-square tests and one-way ANOVAs were conducted. The Pearson's chi-square results showed that there was no significant difference between the groups with respect to gender or ethnicity. The ANOVAs revealed no significant differences among the groups on their vocabulary achievement standard scores on the SAT-10 and full-scale score from the WISC-III. (See the article for the statistical statements on p. 21.)

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

There were no missing data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

If a variable only pertained to the Word Mapping Strategy, it was excluded. Thus, the percentage of points earned on the Word Mapping Strategy test was excluded. That test was only administered to students assigned to the group that received instruction in the Word Mapping Strategy.

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

For the Vocabulary LINCing Strategy Test, which measured students' knowledge and use of the strategy, a repeated measures ANOVA with one between-subjects factor (SWDs versus non-SWDS) was used to examine changes from the pretest to the posttest. Then a separate paired sample t-test was performed to determine whether gains made by the subgroups were significant. For the Word Knowledge Test and the Meaning Prediction Test, separate repeated-measures ANOVAs were used with two between-subjects factors (condition and subgroup) to examine changes from pretest to posttest. To examine differences between the posttest scores of the three groups, a two-way ANCOVA with two between-subjects factors (condition x subgroup) was conducted, with the pretest scores serving as the covariate. The results showed that with regard to the performance of the Vocabulary LINCing group on the Vocabulary LINCing Test, the main effect of time was significant. Thus, the posttest scores were significantly higher than the pretest scores, representing a large effect size (eta squared = .738). This means that the students learned to use the Vocabulary LINCing Strategy. When the students were tested over the meaning of the words that had been directly taught during the instruction, both students with (SWDS) and without disabilities (NSWDs) in the Vocabulary LINCing Group made significant gains, representing large effect sizes. On the Meaning Prediction Test, no significant differences were found between the pretest and posttest scores for either the experimental or control groups. Likewise, there was no difference between the posttest scores of the experimental and control groups. This means that teaching the Vocabulary LINCing Strategy to students did not enable them to predict the meaning of unknown words.