Math Recovery

Study: Smith, Cobb, Earran, Cordray, Munter, et al. (2007)

Smith, T., Cobb, P. Earran, D., Cordray, D., Munter, C. & Dunn, A. (2007). Evaluating Math Recovery: Evaluating the Causal Impact of Math Recovery on Student Achievement. Vanderbilt University.

Descriptive Information

Usage

Acquisition and Cost

Program Specifications and Requirements

Training

Math Recovery (MR) is an early intervention program designed to change the long-term mathematical development of initially low-performing children. The program identifies children who are falling behind in mathematics and provides intensive one-to-one tutoring designed to help them “catch up” to their higher performing peers. 

The MR program comprises both a body of theory and research-based strategies for supporting the development of children’s numerical reasoning, and a set of techniques for supporting instructors’ acquisition and use of this knowledge to work effectively with children. The developers of MR have codified this knowledge in a Learning Framework and an Instructional Framework that instructors use to assess children’s arithmetical knowledge and tailor instruction to their current levels of arithmetical reasoning.

MR tutoring is diagnostic rather than scripted in nature and is consistent with current theory and evidence on effective mathematics teaching. Instructors are expected to select instructional tasks and to adjust probes and questions as informed by their initial and ongoing assessment of students’ numerical competence. The MR program can be implemented with virtually any curricula as the 12-month teacher development program is grounded in gaining a deep understanding of the teaching and learning of elementary mathematics. 

MR is intended for use in kindergarten through fifth grade. Although targeted for first grade, the diagnostic assessments can be applied K-5. It is designed for use with any student at risk of academic failure. The academic area of focus is math (including computation, concepts, word problems, and algebra).

MR is used in school systems in more than 20 states in the U.S., Bahamas, and Canada. It is also used extensively in Great Britain and Australia.

Where to Obtain:
US Math Recovery
205 Powell Place
Brentwood, TN 37027
Phone #: 615-369-0700
Web Site: www.mathrecovery.org

Cost: There are no per-student costs.

The cost of the program is the professional development for the Intervention Specialist. The MR Intervention Specialist professional development program includes 2 weeks of course work, 3 additional individual on-site coaching sessions and/or distance video-coaching, and 3 facilitated collegial team meetings. The program includes a detailed understanding of the Learning Framework in Number and the Instructional Framework in Early Number. It incorporates a staff development model that builds communities of practice for teachers, leader/trainers and coaching teams. 

The cost of the MR Intervention Specialist 12-month course is $5,500 per person, with discounts for additional teachers from the same district. The fees include: access to Math Recovery tools and copyrighted material; assessment and teaching kits, textbooks and program handbooks; all travel and consulting costs for on-site coaching; and membership to the US Math Recovery Council for two years.

Each coaching session is individualized to meet the needs of the instructor, giving insight to specific issues that arise as the instructor begins to work with students.

The MR tutoring program is personalized for each child and highly interactive. The instructor is trained to meet the child at his or her current level of math understanding and provides specific activities to raise the child’s ability.

MR is designed for use with individual students or small groups of 3-4 students.

MR sessions are delivered in 30-minutes with a recommended 4-5 sessions per week for 12-15 weeks.

The program includes highly specified teacher manuals.

MR does require technology. The program requires a video camera to record the tutoring sessions for subsequent analysis. A computer is required for playback, to compile data for lesson plans, and to create an assessment and teaching case study incorporating video clips. A computer camera is required for video conferences. After the training period, video cameras are used to record and analyze student performance.

The instructor training requires a 12 month professional development course with on-site mentoring. The Math Recovery Intervention Specialist training is 10 days of face to face instruction with a trained Math Recovery Leader. The specialist receives five coaching sessions over the course of the year training.

Instructors must be at least professionals. Teaching experience is preferred, but not required. The program does not assume the instructor has expertise in a given area.

In addition, training manuals and materials are available. The math recovery assessments and teaching exemplars were created as part of a 3 year study (1992-1995). The study resulted in 2 books currently used as a part of the MR Intervention Specialist Training, MR assessments, teaching procedures and models for analyzing student performance on the assessments.

Ongoing support is provided through Regional Leaders as well as through web resources.

 

Participants: Unconvincing Evidence

Sample size: 1,027 students (343 program, 684 control)

Risk Status: Those students that scored in the twenty-fifth percentile or below in both the MR and the WJIII protocols were identified as at-risk for academic failure.

Demographics:

  Program Control p of chi square
Number Percentage Number Percentage
Grade level
  Kindergarten          
  Grade 1 343 33% 684 66%  
  Grade 2          
  Grade 3          
  Grade 4          
  Grade 5          
  Grade 6          
  Grade 7          
  Grade 8          
  Grade 9          
  Grade 10          
  Grade 11          
  Grade 12          
Mean Age          
Race-ethnicity
  African-American          
  American Indian          
  Asian/Pacific Islander          
  Hispanic          
  White 194 54% 347 51%  
  Other 148 46% 337 49%  
Socioeconomic status
  Subsidized lunch 220 64% 445 65%  
  No subsidized lunch 123 36% 236 35%  
Disability status
  Speech-language impairments          
  Learning disabilities          
  Behavior disorders          
  Intellectual disabilities          
  Other (SPED Status)          
  Not identified with a disability          
ELL status
  English language learner 43 13% 109 16%  
  Not English language learner 300 87% 575 84%  
Gender
  Female 188 55% 366 53%  
  Male 155 45% 318 47%  

Training of Instructors: The tutors were trained for one year in the Math Recovery. The training consisted of two weeks of coursework with three additional on site coaching sessions or distance video coaching. The training program includes a detailed explanation of the Learning Framework in Number and the Instructional Framework in Early number. Each tutor was released from half of their classroom duties to administer the math tutoring program.

Design: Partially Convincing Evidence

Did the study use random assignment?: Yes.

If not, was it a tenable quasi-experiment?: Not applicable.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: No.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Not applicable.

Were the program and control groups demographically comparable at pretreatment?: No.

Was there attrition bias1? Yes.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Convincing Evidence

Describe when and how fidelity of treatment information was obtained: Charles Munter conducted a separate Master’s Thesis to test the fidelity of the MR study against other instruments. The videos of 107 program subjects were studied by five experienced MR instructors.  Each video was independently coded with an instrument designed for the study. The MR scores were then compared to scores on Woodcock Johnson III testing instruments.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: The results in the Thesis study showed a 0.84 correlation between the MR instructor’s scores and the Woodcock Johnson III instruments where a 1.0 would be perfect fidelity.

Measures Targeted: Unconvincing Evidence

Measures Broader: Convincing Evidence

Targeted Measure

Score type and range of measure

Reliability statistics

Relevance to program instructional content

MR Proximal

 

 

 


 

Broader Measure

Score type and range of measure

Reliability statistics

Relevance to program instructional content

WJ-III Math Reasoning

Raw scores are converted to percentile scores

 0.95

Measures the student’s math reasoning skills.

 

 

Number of Outcome Measures: 6 Math

Mean ES - Targeted: Data Unavailable

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct Measure Effect Size
Math Math Recovery Diagnostic -
Math Math Recovery Proximal -

Broader Measures

Construct Measure Effect Size
Math WJ III Math Fluency -
Math WJ III Applied Problems -
Math WJ III Quantitative Concepts -
Math WJ III Math Concepts -

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.0001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual

Duration of Intervention: 30 minutes, 4-5 times per week, 11 weeks

Minimum Interventionist Requirements: Professional/Teaching, experience preferred, but not required. , 12 month PD course with onsite mentoring, and 4-8 hours of training

Reviewed by WWC or E-ESSA: E-ESSA

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

Program Outcomes: A large evaluation of Math Recovery found positive outcomes on Woodcock Johnson III Math Reasoning measures averaging an effect size of +0.30, in comparison to non-tutored students. The study was set up to be randomized, which would have qualified Math Recovery for the ESSA “Strong” category, but some students from the control group were moved into the experimental group during the study, making this a matched study and therefore qualifying for the ESSA “Moderate” category.

Number of Studies: 1

Average Effect Size: 0.31

Full Report

Other Research: Potentially Eligible for NCII Review: 0 studies