Fusion (Whole Number Foundations Level 1)
Study: Clarke et al. (2013)

Describe how students were selected to participate in the study:

Eligible students were from all first grade classrooms within schools participating in the project.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

All first-grade students completed the Group Curriculum Based Measure (Group CBM) screening assessment. This was a modified version of the Early Numeracy Curriculum Based Measures (Author, 2004). Eligible students were the 10 lowest students on the screener per school who could count and identify numbers from 1 to 10. We excluded students who had received special education services or had severely limited English proficiency.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Fusion

Specify which condition is the control condition:

Only one control condition

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

NA

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

The study design was a randomized control trial, with students randomly assigned to condition blocking on school. The study took place in 9 schools with approximately 10 eligible students per school, based on screening scores and teacher recommendation, selected to be in the study. The research team randomly assigned these10 students to intervention (Fusion instruction) or control (standard district practice). The final sample included 89 students, 44 intervention and 45 control, within nine schools.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Nine district-employed instructional aides (i.e., “interventionists”), all female, each taught one small FUSION group. Aides were included in the study based on time and schedule availability. Six had masters degrees, two had bachelors degrees, and one was a high school graduate. On average, they had 8.7 years teaching experience (range 3-25 years), 7.4 years experience teaching math (range 4-25 years), and 7.7 years teaching first grade (range 4-20 years). Interventionists participated in two, 3-hour professional development workshops led by the authoring and research team. Workshops were intended to deepen interventionists’ content knowledge for teaching mathematics, pedagogical knowledge, and comfort teaching Fusion lessons. Workshops provided time for interventionists to practice teaching Fusion lessons and receive feedback from their peers and the curriculum’s authors. The first workshop occurred in October, prior to Fusion instruction. Content included an overview of the study design and their role, an overview of the Fusion intervention and its underlying principles and content, lesson demonstrations, group management tips, and practice opportunities. The second training occurred in February, after interventionists had implemented a portion of Fusion lessons. During this training, interventionists had the opportunity to ask questions about the first half of the curriculum and were introduced to concepts in the second half of the curriculum.

Describe when and how fidelity of treatment information was obtained.

Trained project staff observed each group’s Fusion instruction three times (i.e., once during the beginning, middle, and end of the curriculum). Observers rated implementation fidelity for each primary activity in a FUSION lesson (Activities 1 through 3), using a 0-1 scale (0 = not taught, 0.5 = partial implementation, and 1 = full implementation). A fidelity score for each observation was calculated by averaging ratings across Activities 1 through 3. Each interventionist’s fidelity scores were averaged across the three observation occasions. Observers also provided a holistic rating of overall level of implementation on a 7-point scale with a score of “1” representing low implementation and “7” representing high implementation. Interobserver reliability was conducted on 20% of all observations and was 95% and 86% for the activity-based and the holistic rating, respectively.

What were the results on the fidelity-of-treatment implementation measure?

Results on specific components of fidelity ranged from 0.90-0.93 with a standard deviation range of 0.07 to 0.17. Overall fidelity was rated as 5.2 with a standard deviation of 1.1.

Was the fidelity measure also used in control classrooms?

No

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

A series of random effects models were estimated using SPSS MIXED procedure to compare gains in ProFusion, EN-CBM, and SAT-10 outcomes between the treatment and control conditions. The random effects models nested pretest and posttest assessments within students and students within instructional groups. The models included the effects of time (coded 0 = pretest, 1 = posttest), condition of instructional group (coded 0 = control and 1 = treatment), and the condition by time interaction. The condition by time interaction represents the difference in gains in outcomes between the two groups. Hedges’ g was reported as a metric of intervention effect size for each outcome (What Works Clearinghouse, 2008; .2, .5, and .8 are considered small, medium, and large effects). As recommended by Feingold (2009), Hedges’ g was computed as the condition by time interaction effect divided by the posttest pooled standard deviation of the outcome. In accordance with an intent-to-treat approach, maximum likelihood estimation was used to obtain model parameters and standard errors using all cases available, which results in less bias in parameter estimates and standard errors than other methods of handling missing data (e.g., listwise deletion; Schafer & Graham, 2002).