Strategic Math Series: Standard Algorithm
Study: Flores, Hinton, & Strozier (2014)

Describe how students were selected to participate in the study:

The students met the criteria for participation which were: (a) parent permission, (b) mastery of addition, subtraction, and multiplication facts as defined as writing 30 correct digits per minute on a curriculum-based measure, (c) demonstration of knowledge of place value as demonstrated by correct identification of ones’, tens’, and hundreds’ places, and lack of skill associated with subtraction and multiplication with regrouping as defined by writing less than 10 correct digits on a timed curriculum-based measure.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional/behavioral difficulties (BI):

All were students receiving tier three mathematics intervention within a MTSS model. None of the students had identified disabilities; however, all of the students failed to respond to tier one and tier two mathematics interventions according to benchmark assessments

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

100.0%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Please describe the study design:

multiple probe across behaviors

Clarify and provide a detailed description of the treatment in the submitted program/intervention:

Materials The materials consisted of assessment probes and instructional items. There were four sets of assessments probes, one for each behavior: subtraction of three-digit numbers with regrouping in the ones place, b) subtraction of three-digit numbers with regrouping in the ones and tens places, c) multiplication of two-digit and one-digit numbers with regrouping, and d) multiplication of two two-digit numbers with regrouping. Each assessment probe was an eight-inch by eleven-inch sheet of paper with twenty-four problems printed in twelve-point font. Measures to ensure content validity for the probes were conducted for each behavior. Pools of the computation questions for each skill area were distributed to three teachers to review. All three teachers had at least a master’s degree from an accredited university and at least four years of experience in teaching mathematics to elementary or middle school students. The reviewers were asked to score each problem for each skill area according to its relevance to the content using a 4-point scale to avoid having neutral and ambivalent midpoint. The item relevance continuum used was 1= not relevant, 2= somewhat relevant, 3= quite relevant, and 4= highly relevant. Then for each item, the item-level content validity index (I-CVI) was computed as the number of experts giving a rating of 3 or 4 (thus dichotomizing the ordinal scale into relevant and not relevant), divided by the total number of experts (Lynn, 1976). The I-CVI for all items for subtraction with regrouping of the ones column and for subtraction with regrouping of the ones and tens column was calculated as 1.00 and 1.00 respectively. The I-CVI for 48 of the 50 two digit by one digit multiplication with regrouping problems was 1.00 and .66 for the remaining two problems. The I-CVI for 63 of the 65 two digit by two digit multiplication with regrouping problems was 1.00 and .66 for two remaining problems. To further examine the items in each probe, the three teachers were also asked to rate the same problems as the following: easy, average or difficult for the typical elementary school student who was learning the computation problem represented by all four behaviors. The difficulty level of the questions required consistency of two of the three raters on each computation problem. Based on the survey, all 56 problems for subtraction with regrouping of the ones column, all 40 problems for subtraction with regrouping of the ones and tens column, and all 50 problems for two digit by one digit multiplication with regrouping were rated as average. The 64 of the 65 problems involving two digit by two digit multiplication with regrouping were rated as average and one problem was rated as easy. Reliability of the probes was also examined for each skill area. The researchers created an assessment for each skill area and each assessment contained 75% or more of the problems from the probes created for the participants. The researchers administered the four assessments to 12 college level students from a major four-year university. The assessments were untimed and all 12 of the assessments were completed. Results from the internal consistency test revealed Cronbach’s Alpha Coefficient of r = .69 for subtraction with regrouping of the ones column, r = .72 for subtraction with regrouping of the ones and tens column, r = .78 for two digit by one digit multiplication with regrouping and r = .73 for two digit by one digit multiplication with regrouping. There was a Cronbach’s Alpha Coefficient of r = .83 for all test items. The instructional materials varied based on skill and level of instruction. Materials used to teach subtraction with regrouping in the ones place included the following: a) lessons one three through included student learning sheets with subtraction with regrouping problems divided into three sections labeled demonstration, guided practice, and independent practice and foam base-ten mathematics manipulative blocks; b) lessons four through six included student learning sheets with subtraction with regrouping problems divided into three sections labeled demonstration, guided practice, and independent practice; c) lesson seven consisted of a learning sheet with the mnemonic RENAME printed with the steps for the RENAME strategy printed in twenty-point font; and d) lessons eight through ten included student learning sheets with subtraction with regrouping problems divided into three sections labeled demonstration, guided practice, and independent practice. Materials used to teach subtraction with regrouping in the ones and tens places, multiplication with one-digit multipliers with regrouping and multiplication with two-digit multipliers with regrouping were similar. These materials included learning sheets and base-ten manipulatives for lessons one through three, learning sheets for lessons four through six, a RENAME strategy sheet for lesson seven, and learning sheets for lessons eight through ten. Procedures The first author, a certified special education teacher, served as the instructor. Prior to instruction, probes were administered until the students demonstrated a stable baseline. A stable baseline was defined as the last three data points in the path varying no more than twenty-percent from the mean. After stable baselines were established, probes were administered prior to daily instruction in order to measure students’ progress. Probes were administered prior to instruction in order to measure learning that had occurred during the previous day’s instruction. The researcher administered the probes by providing the students with a sheet of problems with instructions to complete as many problems as he/she could until told to stop. The students were given two minutes to complete the problems. Instruction began with subtraction with regrouping in the ones place. Prior to lesson one, the instructor showed each student one of his/her baseline probes, discussed his/her difficulty with computation and obtained a written commitment to participate in learning. The instructor and students agreed to work hard to learn a new way of completing math problems. The instructor continued to involve the students in progress monitoring by showing each student his/her progress on the probes in the form of a graph. The instructor showed the students their graphs individually at the beginning of each instructional session. Each graph included an aim line representing the criterion for each phase so that the students could see their progress toward the goal of thirty correct digits. During lessons one through three, the instructor began the lesson with an advance organizer, providing an overview of the lesson activities. Next, the instructor demonstrated how to solve the problems within the Demonstration section of the learning sheet using base-ten manipulatives. This demonstration included physical demonstration as well as thinking aloud. The students participated by answering simple questions in order to maintain engagement, but did not solve problems. The third part of the lesson was guided practice in which the instructor and the students solved problems in the Guided Practice section of the learning sheet together. The instructor and the students alternated turns completing steps of the problem solving process using base-ten manipulatives to solve the problems. Next, the students completed the Independent Practice section of the learning sheet using the base-ten manipulatives without the instructor’s assistance. Finally, the lesson ended with a post organizer in which the instructor reviewed the lesson activities. During lessons four through six, the instructor began the lesson with an advance organizer, providing an overview of the lesson activities. Next, the instructor demonstrated how to solve the problems within the Demonstration section of the learning sheet by drawing pictures. This demonstration included physical demonstration as well as thinking aloud. The students participated by answering simple questions in order to maintain engagement, but did not solve problems. The third part of the lesson was guided practice in which the instructor and the students solved problems in the Guided Practice section of the learning sheet together. The instructor and the students alternated steps of the problem solving process drawing pictures to solve the problems. Next, the students completed the Independent Practice section of the learning sheet drawing pictures without the instructor’s assistance. Finally, the lesson ended with a post organizer in which the instructor reviewed the lesson activities. During lesson seven, the instructor provided an advance organizer, explaining that the students would learn a memory device to help them remember the steps for solving regrouping problems. The instructor recited the strategy and guided the students in reciting the steps. The students practiced the RENAME steps until they could recite the steps without assistance. After the students had memorized the RENAME mnemonic, lessons eight through ten involved instruction solving problems using numbers only and the RENAME mnemonic device. Lessons eight through ten included an advance organizer, demonstration, guided practice, independent practice, and a post organizer. There were ten lessons that followed this same framework for each of the other behaviors: subtraction with regrouping in the ones and tens places, multiplication with a one-digit multiplier and regrouping, and multiplication with a two-digit multiplier and regrouping.

Clarify what procedures occurred during the control/baseline condition (third, competing conditions are not considered; if you have a third, competing condition [e.g., multi-element single subject design with a third comparison condition], in addition to your control condition, identify what the competing condition is [data from this competing condition will not be used]):

The baseline condition involved administration of probes with no instruction. The researcher presented student with probe, started a timer for 2 minutes. After 2 minutes, took the probe away and provide not feedback, but thanked the student for working hard.

Please describe how replication of treatment effect was demonstrated (e.g., reversal or withdrawal of intervention, across participants, across settings)

replicated across behaviors : subtraction with regrouping in ones, subtraction with regrouping in ones and tens, multiplication with regrouping 1-digit multiplier, multiplication with regrouping with 2-digit multiplier

Please indicate whether (and how) the design contains at least three demonstrations of experimental control (e.g., ABAB design, multiple baseline across three or more participants).

A multiple-probe across behaviors design, effects shown across 3 behaviors (Lena & May) and 4 behaviors (Al) at different points in time

If the study is a multiple baseline, is it concurrent or non-concurrent?

Non-concurrent

How was the program delivered?

Individually
Small Group
Classroom

Study measures are classified as targeted, broader, or administrative data according to the following definitions:

• Targeted measures
  Assess outcomes, such as competencies or skills, that the program was directly targeted to improve.
  • In the academic domain, targeted measures typically are not the very items taught but rather novel items structured similarly to the content addressed in the program. For example, if a program taught word-attack skills, a targeted measure would be decoding of pseudo words. If a program taught comprehension of cause-effect passages, a targeted measure would be answering questions about cause-effect passages structured similarly to those used during intervention, but not including the very passages used for intervention.
  • In the behavioral domain, targeted measures evaluate aspects of external or internal behavior the program was directly targeted to improve and are operationally defined.
• Broader measures
  Assess outcomes that are related to the competencies or skills targeted by the program but not directly taught in the program.
  • In the academic domain, if a program taught word-level reading skill, a broader measure would be answering questions about passages the student reads. If a program taught calculation skill, a broader measure would be solving word problems that require the same kinds of calculation skill taught in the program.
  • In the behavioral domain, if a program taught a specific skill like on-task behavior in one classroom, a broader measure would be on-task behavior in another setting.
• Administrative data measures apply only to behavioral intervention tools and are measures such as office discipline referrals (ODRs) and graduation rates, which do not have psychometric properties as do other, more traditional targeted or broader measures.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion: