Learning Strategies Curriculum: Fundamentals in the Sentence Writing Strategy
Study: Bui, Schumaker, & Deshler (2006)

Describe how students were selected to participate in the study:

Participants were 113 fifth-grade students enrolled in two low-performing elementary schools. The students were regularly enrolled in five intact general education classes in which students with learning disabilities, other heath impairments, and emotional disturbance were fully included in all aspects of instruction along with other students who had no disabilities. The district in which the students were enrolled has the policy of randomly selecting students to be placed in each classroom. Then the classes were randomly selected to participate as either the experimental classes or the control classes. Thus, the students were randomly selected into the groups because they were randomly selected by the district to enroll in each class, and then the classes were randomly selected into the experimental and control groups.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

The five classes were selected because at least 10% of the students had been identified with disabilities, the students with disabilities had IEPs, and the classification of students with disabilities met federal and state requirements. Fourteen of the students had been formally classified as having learning disabilities according to state of Kansas guidelines, which are based on the IQ discrepancy model. The researchers verified that all fourteen students with LD had written language specified as part of their disability.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

12.0%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Students who participated in the treatment condition received instruction in the Fundamentals in the Sentence Writing Strategy program. This program involves the instruction of four types of complete simple sentences. Lessons 1 through 10 in the program were implemented with the students on consecutive school days.

Specify which condition is the control condition:

Students who participated in the comparison condition received writing instruction as it normally occurred in their classes. The teachers in this condition had been instructed by the school district to spend one class hour per day on writing instruction using the "Write for Power" program (Sparks, 1982) and the "Six Traits of Writing" (NREL, 2001) program.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

N/A

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

In the two participating schools, students had been randomly assigned to classes at the beginning of the school year. The five participating classes were randomly assigned to the experimental and comparison conditions. Two fifth-grade classes from School X and one class from School Y were randomly assigned to the experimental condition. Two fifth-grade classes from School Y were randomly assigned to the comparison condition.

What was the unit of assignment?

Students

If other, please specify:

First, the students were assigned randomly to the classes by the district, and then the classes were assigned to the experimental conditions.

Please describe the unit of assignment:

The unit of assignment was students.

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

Students' scores on writing assessments were used in the primary data analyses.

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

The instructor in the experimental classes was a graduate student who was working toward her Ph.D. in special education. She was not a certified teacher. She was supported by her Ph.D. advisors.

Describe when and how fidelity of treatment information was obtained.

To ensure instructional fidelity across the experimental classes, the instructor followed a written protocol (Schumaker & Sheldon, 1998) that specified step-by-step instructions for 10 lessons for teaching students to write simple sentences. The instructions include scripted words to say and details about how to use the supplied visual aids and how to present handouts and worksheets. The instructor checked off each instructional step as it was completed in each experimental class. Reference Schumaker, J. B., & Sheldon, J. B. (1998). Fundamentals in the Sentence Writing Strategy. Lawrence, KS: Edge Enterprises, Inc.

What were the results on the fidelity-of-treatment implementation measure?

The instructor's written records indicated that she completed 100% of the instructional steps for each lesson in each experimental classroom.

Was the fidelity measure also used in control classrooms?

No, the instruction in the comparison classrooms was "business as usual." The teachers presented their English Language Arts information as they normally would. They had been instructed to use the "Write for Power" program and the "Six Traits of Writing" program one hour per day. They were not monitored in any way by project staff. The fidelity measure used in the experimental classes would not have pertained to the instruction in the comparison classes because it listed the step-by-step procedures to be followed in teaching the Sentence Writing Strategy.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Statistical tests were conducted to determine whether the experimental students with LD were comparable to the comparison students with LD. A two-way contingency table analysis revealed no significant differences among the two groups of students with LD with regard to their gender, ethnicity, socioeconomic status, and residential status (e.g. single-parent household). One-way ANOVA tests revealed that there was a significant difference among the two groups of students with LD with regard to their average language arts scores on the previous year's Metropolitan Achievement Test. The students with LD in the experimental classes had a mean score of 28.4 and those in the comparison classes had a mean score of 38. However, there was no difference between the groups of students with LD with regard to age. When the student groups without LD were compared, a two-way contingency table analysis showed that there were no significant differences among the experimental and comparison students without LD with regard to gender, ethnicity, socioeconomic status, and residential status. The ANOVA tests showed there were also no differences with regard to age and average language arts scores. (See the statistical statements in the article.) Since the students with LD in the comparison group scored higher than the students with LD in the experimental group on the language arts test, no statistical controls were needed.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

The state writing assessment was an end-of-year test. No previous scores were available on this measure.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

This study involved the implementation of a package of instructional programs for teaching writing, including instruction in paragraph writing and essay writing. Measures that pertained to sentence writing have been included here. Other measures pertaining to other skills (e.g., paragraph writing and essay writing) were not.

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

From the research article: For the comparison group students without LD, a one-way repeated measures ANOVA was conducted for each of the dependent variables using the testing time (pretest and posttest) as the within subjects factor. Because there were only five students with LD in the comparison group, descriptive scores were calculated. For the experimental group, repeated measures ANOVA was conducted for each of the dependent variables using achievement group (students with and without LD) as the between-subjects factor and the testing time (pretest and posttest) as the within-subjects factor. No differences were found related to the achievement group factor. For the experimental students with LD and without LD, a significant difference was found between the pretest and posttest complete sentence scores in favor of the posttest, representing large effect sizes. The students with LD made a mean gain of 47 percent from pretest to posttest. The students without LD made a mean gain of 38 percent from pretest to posttest. Significant differences were also found for the students with and without LD from pretest to posttest on the complicated sentences score, representing mean gains of 19 and 23 percent, respectively. Both gains represented large effect sizes. (See the article for the statistical statements.) From the dissertation study: For the dissertation, data were analyzed for three groups: two classes of experimental students in School X (called Group A), one class of experimental students in School Y (called Group B), and two classes of comparison students in School Y (called Group C). One-way ANCOVAs were conducted to compare the posttest scores of the groups with the pretest scores serving as the covariate. Follow-up statistics (Holm's sequential Bonferroni procedure) were calculated to evaluate pairwise differences. The ANCOVAs and follow-up tests conducted on the percentage of complete sentences score earned by students with LD revealed that the Group A scores and the Group B scores were each significantly higher than the Group C scores. Similar significant differences were found for the percentage of complicated sentences for the whole groups: Group A scores were significantly higher than Group C scores, and Group B scores were significantly higher than Group C scores.