Spring Math

Study: Codding, VanDerHeyden, Martin, & Perrault (2016)

Codding, R., VanDerHeyden, A. M., Martin, R. J., & Perrault, L. (2016). Manipulating treatment dose: Evaluating the Frequency of a Small Group Intervention Targeting Whole Number Operations. Learning Disabilities Research & Practice, 31, 208-220.

Descriptive Information

Usage

Acquisition and Cost

Program Specifications and Requirements

Training

Spring Math is a web-based RTI system for mathematics.

 

Please note: As an RTI system, Spring Math include screening, progress monitoring, and intervention however, NCII has only reviewed the intervention component for the purposes of the Academic Intervention Tools Chart.

Spring Math is intended for use in Kindergarten through eighth grade. The program is intended for use with any student at risk of academic failure. The academic area of focus is math (including computation, concepts, word problems, algebra, problem solving, decimals, and fractions).

Where to Obtain: 
1667 Snelling Ave. N.
St. Paul, MN 55108
Phone: 651-999-6000
Web Site: www.springmath.com

Cost: The initial cost for implementing Spring Math is $7.00 per student, and the replacement cost per student for subsequent use is $7.00.

There are no additional costs or requirements for implementation. The annual per student cost includes initial setup, universal screening and progress monitoring assessments, interventions, and related reporting.

Support documentation is also included.

It is recommended that Spring Math is used with individual students or small groups 15-20 minutes per session, five days a week, for 15-32 weeks.

The program includes a highly specified teacher’s manual

Teachers and administrators interact with Spring Math online. When the teacher logs in to his or her account, the teacher dashboard provides a screening packet for the teacher to print. The packet contains assessments with standardized directions printed on each page in the form of a script for the teacher. The teacher administers and scores the screening measures using answer keys that are provided by the system and then enters the scores into the dashboard. Spring Math provides graphs of students’ performance and if the criterion is met (median score in the risk range on the first two screening measures), recommends a classwide intervention. The teacher dashboard now directs classwide intervention, providing a printable intervention packet that contains the intervention protocol and all materials needed to conduct the classwide intervention for one week. Each week, the teacher enters scores from the 5th session, Spring Math graphs the performance of students, and advances the class to the next target skill or stays on the current skill level. Student progress monitoring data are tracked and used to recommend individual students for more intensive (Tier 3) intervention where needed and the system provides all necessary materials to conduct Tier 3 intervention.

Training is required for the instructor and will take less than one hour to complete.

Training is provided by video tutorials and an online user manual. Topics covered in the tutorial and manual include how to conduct the screening and how to conduct classwide and individual intervention. The research basis for the assessments and interventions are detailed in the online manual. An FAQ section is included.  Implementation support includes: system interpretation of data, provision of all needed assessment and intervention materials, and a coach dashboard that tracks intervention use schoolwide and student learning gains and then directs the coach to support intervention where needed for better results.

 

 

Participants: Convincing Evidence

Sample size: All students in grades 2, 3, and 4 in the participating school were eligible (N = 236). Eligible students were screened for mathematics difficulties using standard CBM screening procedures for mathematics and 141 of the screened students met at-risk criteria (less than 40 digits correct per two minutes for grade 2 and 3 students and less than 80 digits correct per two minutes for students in grade 4) for inclusion in study procedures. Because the study used a fluency-building intervention to examine dosage effects, follow-up assessments were conducted with all 141 students to verify that a fluency-building intervention would be a good match with their learning needs. Specifically, students were excluded if (a) they scored in the frustrational range (less than 20 digits correct per two minutes for grades 2 and 3 and less than 40 digits correct for grade 4) on all grade-level probes (this sequence is provided in Table 1 on p. 5 of the LDRP paper and described below); (b) they could not represent a computation problem using drawings; or (c) they were receiving special education services. The final sample was 101 students in 10 classrooms from grade 2 (n = 39), grade 3 (n = 46), and grade 4 (n = 16).

Risk Status: All students in grades 2, 3, and 4 were screened using standard CBM procedures. The grade 2 screening was addition and subtraction fact families 0-20; Grade 3 screening was mixed multi-digit addition and subtraction with and without regrouping; Grade 4 was multiplication and division fact families 0-12. Any student who scored below the mastery criterion for these measures at their grade level, participated in a follow-up survey-level assessment that was administered individually by researchers. In Grade 2, this involved administering subtraction with minuends to 20, subtraction with minuends to 9, subtraction with minuends to 5, addition with sums to 20, addition with sums to 12, and addition with sums to 6 in that order. Grade 3 skills assessed were subtraction of 2-digit by 2-digit with and without regrouping, subtraction with minuends to 20, subtraction with minuends to 9, subtraction with minuends to 5, addition of 2-digit by 2-digit with and without regrouping, sums to 20, sums to 12, and sums to 6. Grade 4 skills assessed were division 0-12, division 0-5, multiplication 0-12, multiplication 0-5, addition & subtraction fact families 0-20, subtraction 0-20, and sums to 20. Assessment progressed until the child scored in the instructional range (grades 2-3- 20-39 digits correct in two minutes and grade 4- 40-79 digits correct per two minutes). For the instructional level skill, the student was asked to draw a solution to the problem to assess the child’s mathematical thinking, reasoning, and to verify conceptual understanding of the task (i.e., understanding multiplication as repeated addition can be demonstrated by the child drawing factor sets such as 5 x 3 drawn as three sets of five or five sets of three).

Demographics:

 

Program

Control

Cox Index

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

 

 

 

 

 

  Grade 2

11

42

10

38

0.10

  Grade 3

12

46

11

42

0.10

  Grade 4

3

12

5

19

0.33

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity

  African-American

 

 

 

 

 

  American Indian

 

 

 

 

 

  Asian/Pacific Islander

 

 

 

 

 

  Hispanic

 

 

 

 

 

  White

 

 

 

 

 

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

 

 

 

 

 

  No subsidized lunch

 

 

 

 

 

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Intellectual disabilities

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

26

100

26

100

0

ELL status

  English language learner

 

 

 

 

 

  Not English language learner

 

 

 

 

 

Gender

Female

 

 

 

 

 

Male

 

 

 

 

 

Training of Instructors: Eight school psychology graduate students collected screening and progress monitoring data (3 doctoral, 5 specialist-level). Six of these students (3 doctoral, 3 specialist-level) also led the intervention groups for each condition. Each graduate student was trained to use curriculum-based assessment and implement academic interventions as part of their program coursework. Graduate students participated in two training sessions that were specific to the procedures employed in the current study. First, graduate students viewed a 20-min video, in which the first author modeled the treatment protocol with three graduate students (not otherwise serving in the study) acting as participants. Graduate students observed the video in pairs and practiced implementing the treatment protocol with one another. During the second session, the first author reviewed all screening materials and observed test administration to correct errors and provide feedback. Scripted protocols guided all assessment and intervention activities.

Design: Partially Convincing Evidence

Did the study use random assignment?: Yes

If not, was it a tenable quasi-experiment?: N/A

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes?: No

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences?: N/A

Were the program and control groups demographically comparable at pretreatment?: Yes

Was there attrition bias1 ?: No

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: No

 

 

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

Fidelity of Implementation: Convincing Evidence

Describe when and how fidelity of treatment information was obtained: Procedural fidelity of intervention steps was evaluated using a checklist that accounted for 18 distinct interventionist behaviors representing one 12-min treatment session. All intervention sessions for each group were recorded using handheld audiotape recorders. Two specialist-level school psychology graduate students were trained in the intervention procedures and acted as independent reviewers. These graduate students were not otherwise involved with the study. The observers listened to 54 audio recordings (i.e., 12-min sessions) representing between 2 and 6 days of treatment for each of the six interventionists (57% of treatment sessions administered). We also observed and quantified child engagement during 12-18% of intervention sessions to ensure equivalence across experimental conditions.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Mean treatment adherence was 93% (60% - 100%) as measured across all six interventionists. There were no commission or omission errors. Errors reflected wording alterations when intervention activities were explained. Mean treatment adherence was (a) 100% and 86% (66%-100%) for the four-times-weekly interventionists, (b) 97% (89% - 100%) and 87% (60% - 100%) for the twice-weekly interventionists, and (c) 100%, respectively, for both once-weekly interventionists. Mean observed student engagement was 98% (94-100%) on the Behavioral Observation of Students in Schools (BOSS; Shapiro, 2004).

Measures Targeted: Convincing Evidence

Measures Broader: Convincing Evidence

Targeted  Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

Curriculum-Based Measurement Grade-Level Probe

Grade 2: Fact Families for Addition & Subtraction 0-20

Grade 3: Multi-digit Addition & Subtraction With & Without Regrouping

 

For single-skill computation CBMs, Christ et al. (2008) reported reliability indices of r  = 0.83 to 0.90, interrater agreement of 83% to 100%,

and validity coefficients of r = 0.74 to 0.83.

Delayed (2-week) alternate form reliability for these probes has been reported as r = 0.85 in previously published studies. Validity coefficients between probe scores and Stanford Achievement Test have been reported to range from r = 0.27 to r = 0.40 in previously published studies.

Correlations across subsequent screening occasions for the multi-digit addition & subtraction measure was r = 0.77 to 0.78 in the sample for the AERJ and SPR articles; Correlations across subsequent screening occasions for the Fact Families Multiplication & Division measure was r = 0.61 to 0.83 in the sample used for the AERJ and SPR articles.

 

Interscorer agreement on approximately 20% of scores in this study was 97% (60%-100%)

These were the screening measures used to determine risk and then to reflect risk reduction during weekly progress monitoring assessment during the intervention. Intervention was conducted on the screening skill or a related prerequisite (subskill).

These measures were administered weekly to all students in the intervention and control groups.

Grade 4: Fact Families for Multiplication & Division 0-12

 

 

 

Broader  Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

Monitoring Basic Skills Progress - Basic Math Computation  (MBSP-COMP)

Alternate-form reliability coefficients for students without disabilities in grades 2-4 range from r = 0.73 to r = 0.81. Temporal reliability coefficients range from r = 0.81 to r = 0.86. Validity coefficients are r = 0.82 with Math Computation Test scores, r = 0.67 with the numbers and concepts subscore of the Stanford Achievement Test.

 

Interscorer agreement on approximately 20% of scores in this study was 96% (57%-100%)

 

This measure includes items reflecting targeted intervention skills, but is broader and may include skills not specifically targeted during intervention.

These measures were administered weekly to all students in the intervention and control groups.

Monitoring Basic Skills Progress - Concepts and Applications (MBSP-APP)

Internal consistency coefficients are greater than 0.94 for grades 2, 3, & 4. Validity coefficients range from r = 0.74 to r = 0.81 for grades 2 to 4 and with CBT/McGraw Hill and r = 0.63 to r = 0.81 for Monitoring Basic Skills Progress- Basic Math Computation.

 

Interscorer agreement on approximately 20% of scores in this study was 90% (62% - 100%)

This measure includes items reflecting targeted intervention skills, but is broader and may include skills not specifically targeted during intervention.

These measures were administered weekly to all students in the intervention and control groups.

 

Number of Outcome Measures: 2 Math

Mean ES - Targeted: 0.52

Mean ES - Broader: 0.14

Effect Size:

Targeted Measures

Construct

Measure

Effect Size

Math

CBM Grade-Level Probe

0.52

Broader Measures

Construct

Measure

Effect Size

Math

Monitoring Basic Skills Progress - Basic Math Computation  (MBSP-COMP)

-0.12

Math

Monitoring Basic Skills Progress - Concepts and Applications (MBSP-APP)

0.40

Key

*        p ≤ .05

**      p ≤ .01

***    p ≤ .001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual, Small Group, (n=4-30)

Duration of Intervention: 15-20 minutes, 5 times a week, 15-32 weeks

Minimum Interventionist Requirements: Paraprofessional, Less than 1 hour of training

Reviewed by WWC or E-ESSA: No

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

This program was not reviewed by Evidence for ESSA.

Other Research: Potentially Eligible for NCII Review: 0 studies