focusMATH Intensive Intervention

Study: Styers & Baird-Wilkerson (2011)

Styers, M., & Baird-Wilkerson, S. (2011). A Final Report for the Evaluation of Pearson's focusMath Program

Descriptive Information

Usage

Acquisition and Cost

Program Specifications and Requirements

Training

focusMATH Intensive Intervention is an intensive math intervention program designed to fit Response to Intervention (RtI) frameworks and work alongside all core math programs. focusMATH identifies at-risk students early and attempts to help accelerate their learning with instruction that is intensive, balanced, and individualized. Grade levels consist of three units, each built around a specific NCTM Curriculum Focal Point. All three units (or only one unit in kindergarten) are covered in a single Teacher Edition at that grade level. Lessons are organized by topic and can be covered in a single session, making the program ideal for pull-out, summer, or after-school district models.

focusMATH provides concise lessons built on the focal points to provide explicit and systematic instruction on foundational skills.

This intensive intervention program utilizes stepped-out instructional models along with strategic questions that let students verbalize their understanding.

focusMATH offers targeted placement and ongoing progress monitoring to help teachers individualize lessons.

focusMATH Intensive Intervention is intended for use in kindergarten through sixth grade. It is designed for any student at risk of academic failure. The academic area of focus is math (including computation, concepts, word problems, and fractions).

Since the program first became available in the fall of 2009, there are approximately 1,000 schools/districts using the program, with almost 2.4 million students in all 50 states.

Where to Obtain:
Pearson
K12 Customer Service
P. O. Box 2500
Lebanon, IN 46052
Phone #: 800-848-9500
Web Site: www.pearsonschool.com/focusmath

Cost: The initial cost per student for implementing the program depends on the grade level as well as which unit(s) the Placement Test has shown the student to need. Additionally some classrooms may already own manipulatives.

  • Initial cost per student may be anywhere from $4.60 (for a single Unit workbook) to $33.94 (for all three workbooks plus the optional Student Manipulative Kit).
  • Replacement cost per student for subsequent use: $4.60 per workbook ($6.97 in kindergarten).

Additional Cost Information: focusMATH Intensive Intervention consists of three units at the student level (one unit in kindergarten). The Teacher’s Edition contains support for all three student books, Teaching Tool Masters, Assessment Masters, and Answer Keys. There are three consumable student books, one for each Focal Point. Students may be recommended for one or more of the books depending on the results of their Program Placement Test. Each student book can be completed in 4–6 weeks. There are manipulative kits for each grade level. No other materials are required for implementation of the focusMATH Intensive Intervention program.

Pricing:
Grade K

  • Teacher's Edition , $84.97
  • Student Workbook, $6.97
  • Student Workbook (6-Pack), $34.47
  • Grade K Student Manipulative Kit, $8.97

Grades 1-6

  • Teacher's Editions, $127.97
  • Student Workbooks, $4.60
  • Student Workbook, (6 pack), $23.47
  • Grade Level Student Workbook Pack (3 units), $11.97
  • Grades 1-6 Student Manipulative Kits, $21.97

 

focusMATH Intensive Intervention is designed for use with individual students or small groups of 6-8 students.

Administration of focusMATH Intensive Intervention takes 45-60 minutes per session with a recommended 3-5 sessions per week for 4-6 weeks per Unit workbook.

The program includes a highly specified teacher’s manual. No technology is required.

Training is not required for the administration of focusMATH Intensive Intervention. The instructor can teach straight from the focusMATH Intensive Intervention Teacher Edition because the program provides a sufficient support and scripting to teach the program without prior training on response intervention.

Instructors must be at least paraprofessionals. The program was designed to facilitate the teaching of these intensive lessons by anyone no matter their response to intervention background. Due to the teacher support supplied in the focusMATH Teacher Edition, even parent aids, theoretically, can be confident in teaching the lessons.

The program does not assume that the instructor has expertise in a given area.

Online tutorials concerning the implementation of the program are available online at: www.mypearsontraining.com/elementary.

Pearson offers a variety of professional development options, both on-line and on-site where practitioners can obtain ongoing professional and technical support. Options are available here:
Phone: 1-877-637-1604
Website: www.pearsonpd.com

 

 

Participants: Convincing Evidence

Sample size: 357 students (174 program, 183 control)

Risk Status: The norm-referenced KeyMath3 assessment was administered at the beginning of the year to students at each study site.  Magnolia Consulting selected only those students who scored below the 30th percentile on the KeyMath3 to participate in the study. 

Demographics:

  Program Control p of chi square
Number Percentage Number Percentage
Grade level
  Kindergarten n/a        
  Grade 1 n/a        
  Grade 2 n/a        
  Grade 3 85 48.9 89 48.6 1.00
  Grade 4 n/a        
  Grade 5 89 51.1 94 51.4 1.00
  Grade 6 n/a        
  Grade 7 n/a        
  Grade 8 n/a        
  Grade 9 n/a        
  Grade 10 n/a        
  Grade 11 n/a        
  Grade 12 n/a        
Mean Age          
Race-ethnicity
  African-American 39 22.4 41 22.4 0.99
  American Indian Not reported        
  Asian/Pacific Islander 4 2.3 3 1.6 0.99
  Hispanic 57 32.8 61 33.3 0.99
  White 57 32.8 59 32.2 0.99
  Other 17 9.8 19 10.4 0.99
Socioeconomic status
  Subsidized lunch 122 70.5 131 71.6 0.92
  No subsidized lunch 51 29.5 52 28.4 0.92
Disability status
  Speech-language impairments          
  Learning disabilities          
  Behavior disorders          
  Intellectual disabilities          
  Other (SPED Status) 24 13.9 20 10.9 0.50
  Not identified with a disability 149 86.1 163 89.1 0.50
ELL status
  English language learner 45 26.0 41 22.4 0.50
  Not English language learner 128 74.0 142 77.6 0.50
Gender
  Female 89 51.1 90 49.2 0.79
  Male 85 48.9 93 50.8 0.79

Training of Instructors: The majority of facilitators held a master’s degree (59%), a bachelor’s degree (18%), or a high school degree (18%).  Facilitators had been teaching for an average of 12 years and had taught at their current school for an average of seven years. 

Each facilitator received 1.5 days of training. The first day occurred prior to the start of implementation, and was 7 hours.  The second day occurred after 4 – 6 weeks of implementation, and was 3 – 4 hours.  During the seven-hour program training, curriculum specialists reviewed the theory behind focusMATH, completed a program walk-through, discussed successful environments for implementing the program with fidelity, completed a model lesson, reviewed focusMATH lesson planning, and had a Questions and Answer session. The follow-up half-day training focused on specific needs and requests by each school. Trainings allowed curriculum specialists to ascertain the level of treatment fidelity and to work with teachers in reaching higher levels of program implementation. These procedures, with the exception of the research implementation guidelines, reflect Pearson’s standard training protocol for the focusMATH program.  Curriculum specialists shared their contact information with facilitators, and facilitators were encouraged to contact them if they had questions or required assistance after the training was complete.

Design: Convincing Evidence

Did the study use random assignment?: Yes.

If not, was it a tenable quasi-experiment?: Not applicable.

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: Yes.

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Not applicable.

Were the program and control groups demographically comparable at pretreatment?: Yes.

Was there attrition bias1? No.

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes.

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

 

Fidelity of Implementation: Convincing Evidence

Describe when and how fidelity of treatment information was obtained

Evaluators assessed program fidelity through two observations over the course of the school year and weekly self-report logs. Implementation fidelity scores were equally weighted to include ratings from observations and weekly log data. Throughout the school year, focusMATH facilitators completed 10 to 15-minute weekly implementation logs gauging the breadth and depth of their use of the focusMATH program. On the logs, facilitators indicated:

  1. the frequency and extent to which they implement specific focusMath components and materials,
  2. how often they use the program’s additional resources, including assessments, and
  3. their perceptions about the focusMath program.

Facilitators also reported interruptions in their intervention instruction periods (e.g., fire drill, testing, field trips, etc.), as well as student attrition. The logs allowed evaluators to assess implementation fidelity through adherence to required program components and weekly exposure to the program. Evaluators aggregated weekly log data at the end of the study period to aid in determining each teacher’s level of program implementation. Evaluators used this score to compare teachers’ implementation relative to the implementation guidelines established at the beginning of the study. The benchmarks reflect student exposure to the focusMATH program and teacher adherence to the required daily program components (e.g., Concept Development, Guided Practice, Independent Practice). Evaluators averaged the teachers’ overall log implementation ratings with overall observation fidelity ratings in order to create a fidelity measure that included teacher and observer ratings. Evaluators used the overall implementation fidelity score in analyses examining treatment student performance on outcomes and to describe teachers’ fidelity. 

To measure lesson implementation, evaluators observed student and teacher actions during 40- minute intervention periods in the fall and spring. During the observation, evaluators completed a checklist of materials facilitators and students used during the observation (e.g., student workbooks, practice pages, manipulatives, etc.) and rated quality, adherence, and exposure across 22 indicators (e.g., teacher–student interactions, use of instructional strategies, lesson implementation, and student engagement). The protocol allowed evaluators to indicate the extent of observed occurrences and to record notes for validity and reliability purposes. Evaluators qualitatively and quantitatively used the data to triangulate other data sources and to provide observer ratings of implementation fidelity. 

Evaluators established inter-rater reliability on the observation protocol through a facilitator video of a focusMATH lesson with students. Following observation of the video and individual determination of ratings on the 22 indicators, evaluators conducted an in-depth debriefing of the video and ratings, coming to a consensus on the meaning of high versus low ratings on each of the indicators. The five evaluators and researchers involved in observations and ratings established a high degree of inter-rater reliability (average measures intraclass correlation = 0.91, Cronbach’s Alpha = 0.91).

Evaluators observed all small groups instructed by treatment teachers and control teachers (if applicable) twice at all sites and twice in 10 out of 11 schools. One exception was at School A in Site 2. In the spring of 2012, the school had a last-minute weather-related closing and evaluators were not able to observe focusMATH and control groups (if applicable). Also in School A, one control teacher was not available for observation in the fall. Another exception was in Site 6, due to time limitations and scheduling constraints, evaluators observed 8 out of 11 groups instructed by the same study treatment teacher across two schools (School I and J) in the fall and spring (total of 16 observations).

In the fall, evaluators observed a total of 21 out of 22 study teachers, 143 treatment and 12 control students. In the spring, evaluators observed a total of 18 out of 22 study teachers, 152 treatment and 11 control students. The number of control students observed was less than the number of observed treatment students because the majority of control students in the study (92%) did not receive pull-out math intervention.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: The implementation fidelity grand mean for 14 third grade and 16 fifth grade focusMATH teachers was 87%, indicating that the facilitators implemented the program with high fidelity. Overall, 93% (n = 28) of facilitators implemented the program with high fidelity. One facilitator with a third and fifth grade classroom implemented the program with moderate fidelity.

Measures Targeted: Convincing Evidence

Measures Broader: Data Unavailable

Targeted Measure Score type and range of measure Reliability statistics Relevance to program instructional content
KeyMath 3 Total Test 0 – 258 (Raw Score) 3rd Grade Split-Half Reliability for Fall:  0.95 (Form A) and 0.95 (Form B)
3rd Grade Split-Half Reliability for Spring:  0.97 (Form A)
5th Grade Split-Half Reliability for Fall:  0.97 (Form A) and 0.97 (Form B)
5th Grade Split-Half Reliability for Spring:  0.98 (Form A)
Alternate Form Reliability Coefficients:  0.95
focusMath directly addresses Basic Concepts (including Numeration, Geometry, and Measurement), Operations (including Mental Computation and Estimation, Written Computation:  Addition and Subtraction, and Written Computation:  Multiplication and Division), and Applications (Foundations of Problem Solving). These focus areas include appropriate skills within them.
Basic Concepts: Numeration, Geometry, and Measurement 0 – 125 (Raw Score) 3rd Grade Split-Half Reliability for Fall:  0.91 (Form A) and 0.93 (Form B)
3rd Grade Split-Half Reliability for Spring:  0.95 (Form A)
5th Grade Split-Half Reliability for Fall:  0.95 (Form A) and 0.94 (Form B)
5th Grade Split-Half Reliability for Spring:  0.95 (Form A)
Alternate Form Reliability Coefficients:  0.91
focusMath directly addresses Basic Concepts (including Numeration, Geometry, and Measurement), Operations (including Mental Computation and Estimation, Written Computation:  Addition and Subtraction, and Written Computation:  Multiplication and Division), and Applications (Foundations of Problem Solving). These focus areas include appropriate skills within them.
Operations: Mental Computation and Estimation, Written Computation: Addition and Subtraction, Written Computation:  Multiplication and Division 0 – 106 (Raw Score) 3rd Grade Split-Half Reliability for Fall:  0.89 (Form A) and 0.90 (Form B)
3rd Grade Split-Half Reliability for Spring:  0.92 (Form A)
5th Grade Split-Half Reliability for Fall:  0.92 (Form A) and 0.92 (Form B)
5th Grade Split-Half Reliability for Spring:  0.90 (Form A)
Alternate Form Reliability Coefficients: 0.90
focusMath directly addresses Basic Concepts (including Numeration, Geometry, and Measurement), Operations (including Mental Computation and Estimation, Written Computation:  Addition and Subtraction, and Written Computation:  Multiplication and Division), and Applications (Foundations of Problem Solving). These focus areas include appropriate skills within them.
Applications: Foundations of Problem-Solving 0 – 27 (Raw Score) 3rd Grade Split-Half Reliability for Fall:  0.92 (Form A) and 0.88 (Form B)
3rd Grade Split-Half Reliability for Spring:  0.90 (Form A)
5th Grade Split-Half Reliability for Fall:  0.90 (Form A) and 0.90 (Form B)
5th Grade Split-Half Reliability for Spring:  0.90 (Form A)
Alternate Form Reliability Coefficients:  0.85
focusMath directly addresses Basic Concepts (including Numeration, Geometry, and Measurement), Operations (including Mental Computation and Estimation, Written Computation:  Addition and Subtraction, and Written Computation:  Multiplication and Division), and Applications (Foundations of Problem Solving). These focus areas include appropriate skills within them.


 

Broader Measure Score type and range of measure Reliability statistics Relevance to program instructional content
Not Applicable      

 

If you have excluded a variable on data that are reported in the attached document, explain rationale for exclusion:
The KeyMath3 assessment also includes subtests of Algebra and Data Analysis & Probability (under Basic Concepts) and Applied Problem Solving (under Applications). These 3 subtests were not included in the study because focusMath does not address them. The test publisher provided an algorithm to derive an adjusted Total GV score with the exclusion of these three subtests. 

Number of Outcome Measures: 15 Math

Mean ES - Targeted: 0.23*

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct Measure Effect Size
Math KeyMath3: Total GSV 0.28*
Math 3rd Grade: Numeration Scale Score 0.04
Math 3rd Grade: Geometry Scale 0.21
Math 3rd Grade: Measurement Scale Score -0.05
Math 3rd Grade: Mental Computation and Estimation Scale Score 0.22
Math 3rd Grade: Addition and Subtraction Scale Score 0.07
Math 3rd Grade: Multiplication and Division Scale Score 0.45**
Math 3rd Grade: Foundation of Problem Solving Scale Score 0.20
Math 5th Grade: Numeration Scale Score 0.32*
Math 5th Grade: Geometry Scale Score 0.27
Math 5th Grade: Measurement Scale Score 0.27
Math 5th Grade: Mental Computation and Estimation Scale Score 0.41**
Math 5th Grade: Addition and Subtraction Scale Score 0.48**
Math 5th Grade: Multiplication and Division Scale Score 0.23
Math 5th Grade: Foundation of Problem Solving Scale Score 0.09

Broader Measures

Construct Measure Effect Size
  Not Applicable  

 

Key
*      p ≤ 0.05
**    p ≤ 0.01
***  p ≤ 0.001
–      Developer was unable to provide necessary data for NCII to calculate effect sizes
u      Effect size is based on unadjusted means
†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Small Group, (n=3-8)

Duration of Intervention: 30-40 minutes, 2 times a week, 21-29 weeks

Minimum Interventionist Requirements: Paraprofessional, Training is not required

Reviewed by WWC or E-ESSA: E-ESSA

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

Program Outcomes: Students in focusMATH were compared to similar students who did not receive any structured remedial assistance. On Key Math 3, the average effect size was +0.24. This qualified focusMATH for the ESSA “Strong” category.

Number of Studies: 1

Average Effect Size: 0.24

Full Report

Other Research: Potentially Eligible for NCII Review: 0 studies