Class-Wide Function-Related Intervention Teams (CW-FIT )
Study: Wills et al. (draft paper)

Describe how students were selected to participate in the study:

Teachers nominated the students they were most concerned about for disruptive behavior using the ranking procedures outlined in the Systematic Screening for Behavior Disorders (Walker, Severson, & Feil, 1991).

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Students met the cut-points for "at-risk" on the Social Skills Rating System problem behavior subscale (Gresham & Elliott).

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

CW-FIT Program as previously described., see pages 19-21

Specify which condition is the control condition:

The baselines and comparison condition: Teachers generally had posted classroom rules, reminders about the rules, and reprimands for infractions. Many teachers used a response cost warning system with colored cards in pocket folders for each student. Repeated rule infractions resulted in the students moving their cards to a different color with consequences for each card change (5 minutes from recess, call to parent).

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

The randomization procedure included sorting participating teachers within a school. This was done by grade levels (K-2 and 3-6), then randomly assigning teachers (classrooms) evenly from each level into the intervention and comparison groups. This randomization procedure occurred for each school as the school entered the project.

What was the unit of assignment?

Teachers

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Teachers were the interventionists. Data were not collected on teachers' experience. Weekly feedback and monitoring of fidelity was conducted.

Describe when and how fidelity of treatment information was obtained.

A 13-item procedural fidelity checklist (see Figure 1) was used to determine the use of CW-FIT intervention components during sessions (e.g., skills are prominently displayed on posters, pre-corrects on skills occur at beginning of session, point goal is determined, points are awarded to individuals/teams for use of the skills at set intervals, etc.). A total of 532 probes (mean, 5.8 sessions, range of 2-13 sessions across teachers) were collected in baseline for the experimental group and 1,484 (mean, 16.7, range, 3-29) during CW-FIT conditions across all four years.

What were the results on the fidelity-of-treatment implementation measure?

Teachers in CW-FIT classrooms implemented the intervention with high fidelity, across all 4 years; averaging 92.4% (see Table 3). Baseline levels (use of intervention) was low 1-2%

Was the fidelity measure also used in control classrooms?

Frequency of probes collected for the comparison group was 408 (mean, 5.3, range, 1-11) and 737 (mean, 9.9, range 1-23) across conditions, respectively. The use of procedures was low for comparison teachers ranging from 1%-2%.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

1) General Linear Mixed Model (GLMM) analyses were used because of the nested structure of the data consisting of students nested within classrooms. A three-level model was used to analyze the data Level 1--Time (pre and post) was nested within Level 2—Students, which were nested in Level 3—Classrooms; however, no variables were entered at the classroom level (level three) of the model. Gender was a level 2 covariate. Separate models were estimated for each dependent variable. First, an unconditional model was examined determining the amount of variability in the dependent variable due to classroom level influences. Then the predictor variables were added to the model examining their influence on engagement and disruptions. Coefficients were estimated using the PROC MIXED routine in SAS with maximum likelihood estimation. Fully unconditional HLM models were estimated in preliminary analyses. These random-effect models enable estimation of the between-classroom and within-classroom variances. The analyses determine if there are between classroom differences on the outcome variables of engagement and disruptions. Although student’s classrooms were nested within schools, examination of unconditional models revealed that there was not a significant amount of variation between schools while there was a significant amount of variation between classrooms; therefore the school level was not included in final analyses. There was significant between classroom variation for both engagement and disruptions. The adjusted intraclass correlations for the repeated measure models for each outcome were 18% and 22% respectively. This indicates that approximately 82% of the variation in on-task and 78% of the variation in disruptions can be contributed to individual student differences. 2) Fully unconditional HLM models were estimated in preliminary analyses. These random-effect models enable estimation of the between-classroom and within-classroom variances. The analyses determine if there are between classroom differences on the outcome variables of engagement and disruptions. Although student’s classrooms were nested within schools, examination of unconditional models revealed that there was not a significant amount of variation between schools while there was a significant amount of variation between classrooms; therefore the school level was not included in final analyses. 3) Multilevel regression analyses were conducted to evaluate child and teacher behavior, one for each dependent variable. Gender, time and group were the independent variables in all models.