Reflex

Study: Sarrell (2014)

Sarrell, D. A., (2014). The Effects of Reflex Math as a Response to Intervention Strategy to Improve Math Automaticity Among Male and Female At-risk Middle School Students. Dissertation rectrieved from https://digitalcommons.liberty.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1948&context=doctoral

Descriptive Information

Usage

Acquisition and Cost

Program Specifications and Requirements

Training

Reflex is a game-based adaptive system for math fact acquisition. Individual students login using separate accounts and receive assessment, instruction, and practice tailored to their particular needs based on an internal student model. There is an in-product indicator confirming recommended daily use. Model usage is 3 days a week until students are 100% fluent. Time required to complete individual recommended usage varies by age. For students in grade 2, the median time is approximately 16 minutes a day. The product is intended for use by students in grades 2+. It uses a Fact Family approach that leverages the relationships between addition/subtraction and multiplication/division.

Reflex is intended for students in grades 2-8, particularly students with learning disabilities, intellectual disabilities, English language learners, and any student with basic understanding of numeracy.

Where to obtain:

110 Avon Street

Charlottesville, VA 22902

Phone Number: 866-882-4141

Website:  www.reflexmath.com

Cost: $3,295 per school per year

Reflex is available in annual licenses that range from per-student teacher licenses, site licenses, or district licenses. All licenses come bundled with professional development. Multi-year licenses are also available. Reflex also has a grant program that teachers can apply to to get Reflex free for 1 year.

Recommended usage for a second or third grade student would be 3 times a week for 15-20 minutes a day for the bulk of the school year. Students in 4th or 5th grade typically require less time.

Anticipated usage is for students to use Reflex until they reach 100% fluency, at which point the intervention is complete.

To use Reflex students must have access to the internet and a computer.

Instructors may be teachers or support staff. The program does not assume that the instructor has expertise in a given area.

A free 90-minute training is offered but not required to use Reflex since it is delivered individually straight to the students. the training does equip teachers or other personnel to know how to get the best results and understand the reporting and admin features of their teacher accounts.

Users can undertake annual professional development, and there is also a live customer service division that teachers can call, email, or chat with.

 

Participants: Convincing Evidence

Sample size: 89 students (45 program students and 44 control students).

Risk Status: Students were identified based on being in pre-existing math RTI groups or math support classes. Those designations were based on failing the CRCT (Criterion Referenced Competency Test) along with a computerized math assessment. The CRCT is a statewide assessment used in Georgia to evaluate students' abilities relative to the Georgia Performance Standards).

Demographics:

 

Program Number

Program Percentage

Control Number

Control Percentage

Cox Index

Grade level

  Kindergarten

 

 

 

 

 

  Grade 1

 

 

 

 

 

  Grade 2

 

 

 

 

 

  Grade 3

 

 

 

 

 

  Grade 4

 

 

 

 

 

  Grade 5

 

 

 

 

 

  Grade 6

 

 

 

 

 

  Grade 7

 

 

 

 

 

  Grade 8

 

 

 

 

 

  Grade 9

 

 

 

 

 

  Grade 10

 

 

 

 

 

  Grade 11

 

 

 

 

 

  Grade 12

 

 

 

 

 

Race-ethnicity

  African-American

 

 

 

 

 

  American Indian

 

 

 

 

 

  Asian/Pacific Islander

 

 

 

 

 

  Hispanic

 

 

 

 

 

  White

 

 

 

 

 

  Other

 

 

 

 

 

Socioeconomic status

  Subsidized lunch

 

 

 

 

 

  No subsidized lunch

 

 

 

 

 

Disability status

  Speech-language impairments

 

 

 

 

 

  Learning disabilities

 

 

 

 

 

  Behavior disorders

 

 

 

 

 

  Mental retardation

 

 

 

 

 

  Other

 

 

 

 

 

  Not identified with a disability

 

 

 

 

 

ELL status

  English language learner

 

 

 

 

 

  Not English language learner

 

 

 

 

 

Gender

Female

23

51

19

43

0.20

Male

22

49

25

57

0.20

Training of Instructors: All teachers for both groups were highly qualified teachers.

Design: Unconvincing Evidence

Did the study use random assignment? No

If not, was it a tenable quasi-experiment? Yes

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes? NA

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Yes

Were the program and control groups demographically comparable at pretreatment? No

Was there attrition bias1? No

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)? No

 

1 NCII follows guidance from the What Works Clearinghouse (WWC) in determining attrition bias. The WWC model for determining bias based on a combination of differential and overall attrition rates can be found on pages 13-14 of this document: http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf

Fidelity of Implementation: Unconvincing Evidence

Describe when and how fidelity of treatment information was obtained: All scheduled Reflex usage occurred during classroom time with monitoring teachers whose only requirement for the study was to ensure that the students were using the software.

Provide documentation (i.e., in terms of numbers) of fidelity of treatment implementation: Not provided

Measures Targeted: Convincing Evidence

Measures Broader: Data Unavailable

Targeted Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

Basic Math Operations Task [Foegen, A., & Deno, S. L. (2001). Identifying growth indicators for low-achieving students in middle school mathematics. The Journal of Special Education, 35(1), 4-16.]

For this study two forms were used. Cronbach alpha was measured for both, with scores of 0.91 and 0.92. Test retest correlation had a Pearson's r of 0.80 and 0.84 for the two forms. Parallel forms had Pearson's r of 0.79 and 0.80. For criterion validity, see citation in title. [Foegen, A., & Deno, S. L. (2001).]

The BMOT measures students' ability to answer math facts among the four main mathematical operations. This is the main purpose of Reflex, which provides game-based, adaptive instruction and practice on such facts.

No explicit exposure to math fact automaticity, but control group otherwise had business-as-usual math intervention instruction.

 

Broader Measure

Reliability Statistics

Relevance to Program Instructional Content

Exposure to Related Content Among Control Group

None

 

 

 

 

Number of Outcome Measures: 1 Math

Mean ES - Targeted: 0.58*

Mean ES - Broader: Data Unavailable

Effect Size:

Targeted Measures

Construct

Measure

Effect Size

Math

Basic Math Operations Task

0.58**

 

Broader Measures

Construct

Measure

Effect Size

 

None

 

 

Key

*        p ≤ .05

**      p ≤ .01

***    p ≤ .001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

u      Effect size is based on unadjusted means

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single Subject Design): N/A

Disaggregated Data for Demographic Subgroups: Yes

Targeted Measures

Construct

Measure

Effect Size

Math

Basic Math Operations Task – Females

0.86*

Math

Basic Math Operations Task – Males

0.41

 

Broader Measures

Construct

Measure

Effect Size

 

None

 

 

Key

*        p ≤ .05

**      p ≤ .01

***    p ≤ .001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

u      Effect size is based on unadjusted means

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual

Duration of Intervention: 10 minutes, 2 times a week, Until effective

Minimum Interventionist Requirements: Professional, Training is not required

Reviewed by WWC or E-ESSA: No

What Works Clearinghouse Review

This program was not reviewed by What Works Clearinghouse.

 

Evidence for ESSA

This program was not reviewed by Evidence for ESSA.

Other Research: Potentially Eligible for NCII Review: 1 study

Rudel, David (2018) Evaluating Effect of Reflex on Math Fact Fluency in Grades 2 & 3. Retrieved from https://elgizmos.s3.amazonaws.com/docs/Florida_RCT_2016.pdf

Rudel, David (2018) Additional Subpopulation Analysis. Retrieved from https://elgizmos.s3.amazonaws.com/docs/Florida_RCT_2016_Addendum.pdf