ROOTS (Whole Number Foundations Level K)
Study: Clarke et al. (2015)

Describe how students were selected to participate in the study:

A total of 37 kindergarten classrooms participated in the first year of the efficacy trial. Within the 37 participating classrooms, all kindergarten students with parental consent were screened in late fall of their kindergarten school year.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

The research team applied a three-step process to identify students who were at risk for mathematics difficulties and, in turn, might qualify for the ROOTS intervention. First, within the 37 participating classrooms, all kindergarten students with parental consent were screened in late fall of their kindergarten school year. Screening measures included two standardized assessments of early mathematics: Assessing Student Proficiency in Early Number Sense (ASPENS; Author et al., 2011) and the Number Sense Brief (NSB; Jordan, Glutting, & Ramineni, 2008). Students qualified for the intervention if they scored 20 or less on the NSB (Jordan et al., 2008) and had a composite score on the ASPENS that placed in the strategic or intensive range (Author et al., 2011). These thresholds were selected because prior research suggests that students who score in these ranges at the start of kindergarten are at risk for developing long-term mathematics difficulties (Authors et al., 2011a; Jordan et al., 2008). Second, prior to random assignment, students’ ASPENS and NSB scores were separately converted into standard scores and then combined to form an overall composite standard score. Third, students’ composite standard scores were rank ordered and the lowest ten students that met our established a priori criteria were considered eligible for random assignment.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

ROOTS

Specify which condition is the control condition:

BAU

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

NA

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Second, prior to random assignment, students’ ASPENS and NSB scores were separately converted into standard scores and then combined to form an overall composite standard score. Third, students’ composite standard scores were rank ordered and the lowest ten students that met our established a priori criteria were considered eligible for random assignment. Students were then randomly assigned to one of three conditions: (1) a ROOTS two-student group, (2) a

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

District-employed instructional assistants and interventionists hired specifically for the study facilitated the 58 ROOTS intervention groups. Observation data collected in a previous investigation of ROOTS showed that interventionists with these types of backgrounds were able to deliver the ROOTS curriculum with a high degree of implementation fidelity and quality (Authors et al., 2014a). Most interventionists in this study were female (98%) and 61% held a bachelor’s degree or higher. Interventionists had an average of eight years of experience in working in schools and approximately 20% held current teaching licenses. Among the interventionists, 94% identified themselves as White, 3% as Hispanic, and 3% as representing another ethnic group. Participating interventionists attended two five-hour professional development workshops focused on the ROOTS intervention. The initial workshop targeted the instructional objectives of Lessons 1-25, whole number concepts and skills identified in the CCSS-M (2010), small-group management techniques, and the instructional practices that have been empirically validated to increase student mathematics achievement, such as provision of student mathematics verbalizations. The second workshop had a similar agenda but focused on the mathematical content prioritized in the intervention’s final 25 lessons. Interventionists were given opportunities throughout both workshops to practice with sample lessons and receive feedback on their instructional delivery from key research staff. To bolster implementation fidelity and enhance the quality of Tier 2 mathematics instruction, each ROOTS group received between three and four in-class coaching visits. Five former educators, who were knowledgeable in the science of early mathematics development and instruction, served as coaches during the study. Coaching visits facilitated an observation-feedback loop that began with direct observation of intervention implementation followed by a debriefing on the quality of instructional delivery and implementation fidelity.

Describe when and how fidelity of treatment information was obtained.

Trained research staff observed each ROOTS group three times across the intervention time period. Observations were used to gauge implementation fidelity or the extent to which interventionists implemented the intervention as intended. The fidelity measure focused on four features of implementation adherence, including the extent to which ROOTS interventionists: (a) delivered the prescribed number of activities in the observed lesson, (b) met the observed lesson’s instructional objectives, (c) followed the teacher scripting, and (d) used the prescribed mathematics models. To measure the first implementation feature, the number of activities taught during an observation occasion, research staff used a copy of the ROOTS program to follow along with the small-group instruction. The final three features were measured using a 4-point scale (4 = all, 3 = most, 2 = some, 1 = none).

What were the results on the fidelity-of-treatment implementation measure?

On average, interventionists taught the majority of the prescribed activities (M = 4.2 out of 5, SD = 0.5). Results also indicated that interventionists met the instructional objectives (M = 3.5, SD = 0.5), adhered to the prescribed teacher scripting (M = 3.3, SD = 0.6), and used the ROOTS mathematical models (M = 3.6, SD = 0.5). Intervention dosage served as another metric of implementation fidelity. In this study, dosage was operationally defined as the amount of the intervention received by the participating students. Of the 58 intervention groups, 57 completed 97% or more of the prescribed lessons, while 1 group completed 88% of the lessons. Dosage also represented the degree to which interventionists delivered ROOTS at the prescribed intervention frequency (i.e., sessions per week) and duration (i.e., session length). As intended, students in all treatment groups received ROOTS at a frequency of 5 sessions per week.

Was the fidelity measure also used in control classrooms?

No. The ROOTS intervention occurred outside of and in addition to core mathematics instruction. To document the instructional practices and mathematics content employed in the control condition, research staff administered two surveys and conducted one direct observation of core mathematics instruction during the intervention time period. No evidence of treatment diffusion or contamination of ROOTS instruction was found in the core mathematics instructional settings.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

SAT 10 and SESAT were not administered pre and post and were used to investigate secondary research questions (e.g. closing the achievement gap with not-at risk peers).

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

We assessed intervention effects on each of the primary outcomes with a mixed model (multilevel) time × condition analysis (Murray, 1998) designed to account for students partially nested within small groups (Baldwin, Bauer, Stice, & Rohde, 2011; Bauer, Sterba, & Hallfors, 2008).