ROOTS (Whole Number Foundations Level K)

Study: Clarke, Doabler, Smolkowski, Baker, Fien, et al. (2011)

Clarke, B., Doabler, C., Smolkowski, K., Baker, S.K., Fien, H., & Strand Cary, M. (2011). Examining the Efficacy of a Tier 2 Kindergarten Intervention Report (Technical Report 1104). Eugene, OR: University of Oregon.

Descriptive Information

Usage

Acquisition and Cost

Program Specifications and Requirements

Training

ROOTS is a 50-lesson kindergarten intervention program designed to develop procedural fluency with and conceptual understanding of whole number concepts. Each ROOTS lessons include approximately 4 to 5 brief math activities that center on whole number concepts and skills.

ROOTS provides in-depth instruction in whole number concepts by linking the informal mathematics developed prior to kindergarten to the formal mathematics of kindergarten.
Specifically, ROOTS focuses on three key areas of whole number understanding: (a) Counting and Cardinality, (b) Number Operations, and (c) Base 10/Place Value.

ROOTS is a program intended specifically for kindergarteners. It is intended for use with any student at risk of academic failure. The academic area of focus is math (including computation and concepts).

The ROOTS program was used in 14 classrooms in 3 school districts in Oregon. It has also been used in 32 classrooms in 3 school districts in Texas.

Where to obtain: UO CTL Marketplace website

Available as: Whole Number Foundations Level K

Website: https://dibels.uoregon.edu/market/movingup/kfoundation

Email: support@dibels.uoregon.edu

Phone: 1-888-497-4290

Cost: See website for details.

ROOTS is designed for use with small groups of 5 students.

ROOTS is delivered by instructional assistants (IAs) to small groups consisting of 4-5 students, 2 to 3 times per week, for 20 weeks during the second half of the school year.

The program includes highly specified teacher manuals. The program does not require technology.

The minimum qualifications for ROOTS instructors are that they be instructional assistants and/or paraprofessionals. The program does not assume that the instructor has expertise in a given area.

In the research study, instructional assistants (IAs) attended two half-day trainings, and regular on-going coaching support was provided to facilitate high levels of implementation fidelity.

The initial workshop focused on the instructional objectives related to Lessons 1-25, the critical content of kindergarten mathematics, small-group management techniques, and the instructional practices that have been empirically validated to increase student math achievement. In the second workshop the same format was followed as in workshop 1, but with a focus on the second half of the curriculum, Lessons 26-50. Workshops were organized around three principles: (a) active participation, (b) content focused, and (c) coherence.

Practitioners may obtain ongoing professional/technical support by contacting:

Email: support@dibels.uoregon.edu

Phone: 1-888-497-4290 

 

Participants: Convincing Evidence

Sample size: 143 nominated students (69 program, 74 control)

Risk Status: Participating teachers (n=29) were asked to, “please select 5 students whom you think would benefit from a small group math intervention program.”

Demographics:

  Program Control p of chi square
Number Percentage Number Percentage
Grade level
  Kindergarten 54 48.65 57 51.35  
  Grade 1          
  Grade 2          
  Grade 3          
  Grade 4          
  Grade 5          
  Grade 6          
  Grade 7          
  Grade 8          
  Grade 9          
  Grade 10          
  Grade 11          
  Grade 12          
Mean Age          
Race-ethnicity
  African-American          
  American Indian          
  Asian/Pacific Islander          
  Hispanic          
  White          
  Other          
Socioeconomic status
  Subsidized lunch          
  No subsidized lunch          
Disability status
  Speech-language impairments 10 9.01 11 9.91 0.6131
  Learning disabilities          
  Behavior disorders          
  Intellectual disabilities          
  Other     1 0.90  
  Not identified with a disability 44 39.64 45 40.54  
ELL status
  English language learner 41 36.94 35 31.53 0.0998
  Not English language learner 13 11.71 22 19.82  
Gender
  Female 25 22.52 27 24.32 0.9099
  Male 29 26.13 30 27.03  

Training of Instructors: Teachers had diverse backgrounds and varying levels of teaching experience, but all teachers using the treatment program received an initial hands-on 4-hour professional development course, which covered the pedagogy of the program, and basic functionality of the program and teacher management system. The initial professional development also allowed for teachers and administrators to design implementation and motivation plans specific for their school. A follow up 3-hour professional development course was provided approximately 4 weeks later. This time period allowed the teachers to gain experience and allowed the students to accumulate usage data.  The follow up professional development was then able to answer specific teacher usage questions and delve deeper into the progress monitoring and reporting features of the program. All teachers and administrators using the program received on-going support via site visits, email, and phone communications to address any questions or concerns that arose during the implementation period.

Design: Partially Convincing Evidence

Did the study use random assignment?: Yes.

If not, was it a tenable quasi-experiment?: Not applicable.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: No.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Not applicable.

Were the program and control groups demographically comparable at pretreatment?: No.

Was there attrition bias1? No.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Convincing Evidence

Describe when and how fidelity of treatment information was obtained: Online logs completed by the 14 instructional assistants (IAs) who delivered the ROOTS intervention. Research staff observed ROOTS instruction 1-3 times over the course of the study and rated fidelity of implementation using a 3-point rating scale where 3 represented “full implementation,” 2 represented “partial implementation,” and 1 was “not taught”.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Online logs revealed that groups generally completed all 50 ROOTS lessons during the year. IAs demonstrated high fidelity scores (M = 2.92, SD = 0.06) for the activities prescribed. Seventy five percent implementation would correspond to a 2.5 on our scales thus the average score of 2.92 represents high levels of fidelity above the 75% criteria. Intraclass correlation coefficients (ICC) representing inter-rater reliability showed substantial agreement between observers (ICCs = 0.67).  

Measures Targeted: Data Unavailable

Measures Broader: Convincing Evidence

Targeted Measure Score type and range of measure Reliability statistics Relevance to program instructional content
Not applicable      


 

Broader Measure Score type and range of measure Reliability statistics Relevance to program instructional content
EN-CBM Raw Scores 0-320 0.76-0.99 test-retest General indicator of math knowledge.
TEMA Raw Scores 0-61 0.97 test-retest, 0.83-0.92 alternate form Critical early math content skills and concepts.

 

 

Number of Outcome Measures: 4 Math

Mean ES - Targeted: Data Unavailable

Mean ES - Broader: 0.21

Effect Size:

Targeted Measures

Construct Measure Effect Size
  Not Applicable  

 Broader Measures

Construct Measure Effect Size
Math EN-CBM 0.24
Math TEMA Scaled Score 0.25
Math TEMA Raw Score 0.23
Math TEMA Percentile Score 0.11

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: Yes

Administration Group Size: Small Group, (n=4-5)

Duration of Intervention: 20 minutes, 3 times a week, 17-18 weeks

Minimum Interventionist Requirements: Paraprofessional, 1-4 hours of training

Reviewed by WWC or E-ESSA: E-ESSA

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

Program Outcomes: ROOTS was evaluated in one qualifying study in Boston. On SESAT, TEMA-3, and NSB measures, students in ROOTS gained more than controls with an average effect size of +0.32. This qualifies ROOTS for the ESSA “Strong” category. However, there were no differences on a follow-up measure given in the middle of first grade.

Number of Studies: 1

Average Effect Size: 0.32

Full Report

Other Research: Potentially Eligible for NCII Review: 0 studies