ROOTS (Whole Number Foundations Level K)

Study: Clarke, Doabler, Smolkowski, Kurtz-Nelson, Fien, & Baker (2015)

Clarke, B., Doabler, C.T., Smolkowski, K., Kurtz-Nelson, E., Fien, H., & Baker, S. (2015). Testing the immediate and long-term efficacy of a tier 2 kindergarten mathematics intervention (Technical Report 1502). Eugene, OR: University of Oregon.

Descriptive Information

Usage

Acquisition and Cost

Program Specifications and Requirements

Training

ROOTS is a 50-lesson kindergarten intervention program designed to develop procedural fluency with and conceptual understanding of whole number concepts. Each ROOTS lessons include approximately 4 to 5 brief math activities that center on whole number concepts and skills.

ROOTS provides in-depth instruction in whole number concepts by linking the informal mathematics developed prior to kindergarten to the formal mathematics of kindergarten.
Specifically, ROOTS focuses on three key areas of whole number understanding: (a) Counting and Cardinality, (b) Number Operations, and (c) Base 10/Place Value.

ROOTS is a program intended specifically for kindergarteners. It is intended for use with any student at risk of academic failure. The academic area of focus is math (including computation and concepts).

The ROOTS program was used in 14 classrooms in 3 school districts in Oregon. It has also been used in 32 classrooms in 3 school districts in Texas.

Where to obtain: UO CTL Marketplace website

Available as: Whole Number Foundations Level K

Website: https://dibels.uoregon.edu/market/movingup/kfoundation

Email: support@dibels.uoregon.edu

Phone: 1-888-497-4290

Cost: See website for details.

ROOTS is designed for use with small groups of 5 students.

ROOTS is delivered by instructional assistants (IAs) to small groups consisting of 4-5 students, 2 to 3 times per week, for 20 weeks during the second half of the school year.

The program includes highly specified teacher manuals. The program does not require technology.

The minimum qualifications for ROOTS instructors are that they be instructional assistants and/or paraprofessionals. The program does not assume that the instructor has expertise in a given area.

In the research study, instructional assistants (IAs) attended two half-day trainings, and regular on-going coaching support was provided to facilitate high levels of implementation fidelity.

The initial workshop focused on the instructional objectives related to Lessons 1-25, the critical content of kindergarten mathematics, small-group management techniques, and the instructional practices that have been empirically validated to increase student math achievement. In the second workshop the same format was followed as in workshop 1, but with a focus on the second half of the curriculum, Lessons 26-50. Workshops were organized around three principles: (a) active participation, (b) content focused, and (c) coherence.

Practitioners may obtain ongoing professional/technical support by contacting:

Email: support@dibels.uoregon.edu

Phone: 1-888-497-4290 

 

Participants: Convincing Evidence

Sample size: 261 (179 program, 82 control)

Risk Status: The research team applied a three-step process to identify students who were at risk for mathematics difficulties and, in turn, might qualify for the ROOTS intervention. First, within the 37 participating classrooms, all kindergarten students with parental consent were screened in late fall of their kindergarten school year. Screening measures included two standardized assessments of early mathematics: Assessing Student Proficiency in Early Number Sense (ASPENS; Author et al., 2011) and the Number Sense Brief (NSB; Jordan, Glutting, & Ramineni, 2008). Students qualified for the intervention if they scored 20 or less on the NSB (Jordan et al., 2008) and had a composite score on the ASPENS that placed in the strategic or intensive range (Author et al., 2011). These thresholds were selected because prior research suggests that students who score in these ranges at the start of kindergarten are at risk for developing long-term mathematics difficulties (Authors et al., 2011a; Jordan et al., 2008). Second, prior to random assignment, students’ ASPENS and NSB scores were separately converted into standard scores and then combined to form an overall composite standard score. Third, students’ composite standard scores were rank ordered and the lowest ten students that met our established a priori criteria were considered eligible for random assignment.

Demographics:

 

Program

Control

Cox Index

Number

Percentage

Number

Percentage

Grade level

Kindergarten

203

70%

87

30%

1.03

  Grade 1

 

 

 

 

 

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity

  African-American

 

 

 

 

 

  American Indian

8

4%

3

3%

0.18

  Asian/Pacific Islander

4

2%

4

5%

0.57

  Hispanic

67

33%

28

32%

0.03

  White

114

56%

52

60%

0.10

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

 

 

 

 

 

  No subsidized lunch

 

 

 

 

 

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

       

  Not identified with a disability

 

 

 

 

 

ELL status

  English language learner

65

32%

27

31%

0.03

  Not English language learner

138

68%

60

69%

0.03

Gender

Female

112

55%

51

59%

0.10

Male

91

45%

36

41%

0.10

Training of Instructors: District-employed instructional assistants and interventionists hired specifically for the study facilitated the 58 ROOTS intervention groups. Observation data collected in a previous investigation of ROOTS showed that interventionists with these types of backgrounds were able to deliver the ROOTS curriculum with a high degree of implementation fidelity and quality (Authors et al., 2014a). Most interventionists in this study were female (98%) and 61% held a bachelor’s degree or higher. Interventionists had an average of eight years of experience in working in schools and approximately 20% held current teaching licenses. Among the interventionists, 94% identified themselves as White, 3% as Hispanic, and 3% as representing another ethnic group.

Participating interventionists attended two, five-hour professional development workshops focused on the ROOTS intervention. The initial workshop targeted the instructional objectives of Lessons 1-25, whole number concepts and skills identified in the CCSS-M (2010), small-group management techniques, and the instructional practices that have been empirically validated to increase student mathematics achievement, such as provision of student mathematics verbalizations. The second workshop had a similar agenda, but focused on the mathematical content prioritized in the intervention’s final 25 lessons. Interventionists were given opportunities throughout both workshops to practice with sample lessons and receive feedback on their instructional delivery from key research staff.

To bolster implementation fidelity and enhance the quality of Tier 2 mathematics instruction, each ROOTS group received between three and four in-class coaching visits. Five former educators, who were knowledgeable in the science of early mathematics development and instruction, served as coaches during the study. Coaching visits facilitated an observation-feedback loop that began with direct observation of intervention implementation followed by a debriefing on the quality of instructional delivery and implementation fidelity.

Design: Convincing Evidence

Did the study use random assignment?: Yes

If not, was it a tenable quasi-experiment?: N/A

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Yes.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? N/A

Were the program and control groups demographically comparable at pretreatment?: Yes.

Was there attrition bias1? No.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Convincing Evidence

Describe when and how fidelity of treatment information was obtained: Trained research staff observed each ROOTS group three times across the intervention time period. Observations were used to gauge implementation fidelity or the extent to which interventionists implemented the intervention as intended. The fidelity measure focused on four features of implementation adherence, including the extent to which ROOTS interventionists: (a) delivered the prescribed number of activities in the observed lesson, (b) met the observed lesson’s instructional objectives, (c) followed the teacher scripting, and (d) used the prescribed mathematics models. To measure the first implementation feature, the number of activities taught during an observation occasion, research staff used a copy of the ROOTS program to follow along with the small-group instruction. The final three features were measured using a 4-point scale (4 = all, 3 = most, 2 = some, 1 = none). 

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: On average, interventionists taught the majority of the prescribed activities (M = 4.2 out of 5, SD = 0.5). Results also indicated that interventionists met the instructional objectives (M = 3.5, SD = 0.5), adhered to the prescribed teacher scripting (M = 3.3, SD = 0.6), and used the ROOTS mathematical models (M = 3.6, SD = 0.5).

Intervention dosage served as another metric of implementation fidelity. In this study, dosage was operationally defined as the amount of the intervention received by the participating students. Of the 58 intervention groups, 57 completed 97% or more of the prescribed lessons, while 1 group completed 88% of the lessons. Dosage also represented the degree to which interventionists delivered ROOTS at the prescribed intervention frequency (i.e., sessions per week) and duration (i.e., session length). As intended, students in all treatment groups received ROOTS at a frequency of 5 sessions per week.

Measures Targeted: Convincing Evidence

Measures Broader: Convincing Evidence

Targeted  Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

RAENS

Inter-rater 100%

Proximal

Through core math instruction

 

Broader  Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

ASPENS

Test-retest 0.74 to 0.85

Distal

Through core math instruction

TEMA

Alternate form 0.97

Test-retest 0.82-0.93

Distal

Through core math instruction

Oral Counting

Interrater 0.99; alternate form 0.83; test restest 0.78-0.80

Distal

Through core math instruction

NSB

Coefficient alpha 0.84

Distal

Through core math instruction

 

Number of Outcome Measures: 5 Math

Mean ES - Targeted: 0.75*

Mean ES - Broader: 0.38*

Effect Size:

Targeted Measures

Construct

Measure

Effect Size

Math

RAENS

0.75***

Broader Measures

Construct

Measure

Effect Size

Math

NSB

0.18

Math

ASPENS

0.58***

Math

Oral Counting

0.28*

Math

TEMA

0.32*

 

Key

*        p ≤ 0.05

**      p ≤ 0.01

***    p ≤ 0.001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

u      Effect size is based on unadjusted means

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: Yes

Administration Group Size: Small Group, (n=5)

Duration of Intervention: 20 minutes, 4-5 times a week, 17-18 weeks

Minimum Interventionist Requirements: Paraprofessional, 1-4 hours of training

Reviewed by WWC or E-ESSA: E-ESSA

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

Program Outcomes: ROOTS was evaluated in one qualifying study in Boston. On SESAT, TEMA-3, and NSB measures, students in ROOTS gained more than controls with an average effect size of +0.32. This qualifies ROOTS for the ESSA “Strong” category. However, there were no differences on a follow-up measure given in the middle of first grade.

Number of Studies: 1

Average Effect Size: 0.32

Full Report

Other Research: Potentially Eligible for NCII Review: 0 studies