Star
Math

Descriptive Information

Please provide a description of your tool:

Star Math is a computer-adaptive assessment of general mathematics achievement for students in grades 1 to 12. Star Math provides information on student performance on hundreds of skills within 32 domains. Mathematics computation, mathematic application, and mathematics concepts can be assessed. The difficulty of items is adjusted automatically to reflect the skill level of all students.

Is your tool designed to measure progress towards an end-of-year goal (e.g., oral reading fluency) or progress towards a short-term skill (e.g., letter naming fluency)?

End-year goal
Short-term skill

The tool is intended for use with the following grade(s).

Preschool / Pre - kindergarten
Kindergarten
First grade
Second grade
Third grade
Fourth grade
Fifth grade
Sixth grade
Seventh grade
Eighth grade
Ninth grade
Tenth grade
Eleventh grade
Twelfth grade

The tool is intended for use with the following age(s).

0-4 years old
5 years old
6 years old
7 years old
8 years old
9 years old
10 years old
11 years old
12 years old
13 years old
14 years old
15 years old
16 years old
17 years old
18 years old

The tool is intended for use with the following student populations.

Students in general education
Students with disabilities
English language learners

ACADEMIC ONLY: What dimensions does the tool assess?

Reading

Global Indicator of Reading Competence
Listening Comprehension
Vocabulary
Phonemic Awareness
Decoding
Passage Reading
Word Identification
Comprehension

Spelling & Written Expression

Global Indicator of Spelling Competence
Global Indicator of Writting Expression Competence

Mathematics

Global Indicator of Mathematics Comprehension
Early Numeracy
Mathematics Concepts
Mathematics Computation
Mathematics Application
Fractions
Algebra

Other

Please describe specific domain, skills or subtests:

BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.

BEHAVIOR ONLY: Which category of behaviors does your tool target?

Acquisition and Cost Information

Administration

BEHAVIOR ONLY: What type of administrator is your tool designed for?

General education teacher
Special education teacher
Parent
Child
External observer
Other

If other, please specify:

BEHAVIOR ONLY: What is the administration format?

Direct observation
Rating scale
Checklist
Performance measure
Other

If other, please specify:

BEHAVIOR ONLY: What is the administration setting?

General education classroom
Special education classroom
School office
Recess
Lunchroom
Home
Other

If other, please specify:

Does the program require technology?

Yes

If yes, what technology is required to implement your program? (Select all that apply)

Computer or tablet
Internet connection
Other technology (please specify)

What is the administration context?

Individual

Small group   If small group, n=

Large group   If large group, n=

Computer-administered
Other

If other, please specify:

ACADEMIC ONLY: What are the discontinue rules?

No discontinue rules provided
Basals
Ceilings
Other

If other, please specify:

Training & Scoring

Training

Is training for the administrator required?

Yes

Describe the time required for administrator training, if applicable:

Less than one hour of training.

Please describe the minimum qualifications an administrator must possess.

No minimum qualifications

Are training manuals and materials available?

Yes

Are training manuals/materials field-tested?

Yes

Are training manuals/materials included in cost of tools?

Yes

If No, please describe training costs:

Can users obtain ongoing professional and technical support?

Yes

If Yes, please describe how users can obtain support:

Renaissance Technical Support Staff, reachable by phone, email, or chat.

Scoring

BEHAVIOR ONLY: What types of scores result from the administration of the assessment?

Score

Observation	Behavior Rating
Frequency Duration Interval Latency	Raw score

Conversion

Observation	Behavior Rating
Rate Percent	Standard score Subscale/ Subtest Composite Stanine Percentile ranks Normal curve equivalents IRT based scores

Interpretation

Observation	Behavior Rating
Error analysis Peer comparison Rate of change	Dev. benchmarks Age-Grade equivalent

How are scores calculated?

Manually (by hand)
Automatically (computer-scored)
Other

If other, please specify:

What is the basis for calculating performance level and percentile scores?

Age norms
Grade norms
Classwide norms
Schoolwide norms
Stanines
Normal curve equivalents

What types of performance level scores are available?

Raw score
Standard score
Percentile score
Grade equivalents
IRT-based score
Age equivalents
Stanines
Normal curve equivalents
Developmental benchmarks
Developmental cut points
Equated
Probability
Lexile score
Error analysis
Composite scores
Subscale/subtest scores
Other

If other, please specify:

Scaled Score

Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.

All scores are calculated automatically by the software. The software calculates a maximum likelihood Rasch ability estimate based on the calibrated difficulty of the items that were administered to the student, and the pattern of the student’s right and wrong responses to those items. Star Math uses a proprietary, Rasch-based, 1-parameter logistic response model to calculate scores. The scaled score is a non-linear, monotonic transformation of the Rasch ability estimate resulting from the adaptive test. From the scaled scores and the student’s current grade placement are derived grade equivalent, percentile, and normal curve equivalent scores. No clusters, composite, or raw scores are reported.

Do you provide basis for calculating slope (e.g., amount of improvement per unit in time)?

Yes

ACADEMIC ONLY: Do you provide benchmarks for the slopes?

Yes

ACADEMIC ONLY: Do you provide percentile ranks for the slopes?

Yes

What is the basis for calculating slope and percentile scores?

Age norms
Grade norms
Classwide norms
Schoolwide norms
Stanines
Normal curve equivalents

Describe the tool’s approach to progress monitoring, behavior samples, test format, and/or scoring practices, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.

Each Star Math test contains 34 items in a multiple-choice format. Aligned to state-specific standards and the Common Core State Standards, the test blueprint specifies operational item counts from each of the four domains: Numbers and Operations, Algebra, Geometry and Measurements and Data Analysis, Statistics and Probability. The items are administered to ensure that a balance of content is administered at each grade, appropriate to the typical curriculum for that grade. The assessments’ computer-adaptive structure matches students to items of appropriate difficulty, which in turn may help to reduce student frustration during testing. The assessments can also provide accommodations for students with hearing and visual impairments. The item bank is demographically and contextually balanced. Test blueprint goals are established and tracked to ensure appropriate balance in items addressing use of fiction and nonfiction text, subject and topic areas, geographic region, gender, ethnicity, occupation, age, and disability. Items are free of stereotyping, representing different groups of people in non-stereotypical settings. Items do not refer to inappropriate content that includes, but is not limited to content that presents stereotypes based on ethnicity, gender, culture, economic class, or religion. Items do not present any ethnicity, gender, culture, economic class, or religion unfavorably. Items do not introduce inappropriate information, settings, or situations. All items are subjected to field testing, rigorous psychometric review and related calibration procedures before being included in operational item banks. This skill measurement provides a crucial component in progress monitoring. As students learn new skills, Star Math can assess the level of achievement as often as weekly, although less frequent testing is recommended to avoid over-testing. All items have been calibrated, so that the difficulty of each item is expressed on a Rasch difficulty scale that spans the range of mathematics proficiency from grade 1 through grade 12. Rasch scores are converted to a wide range of scores including Scaled Score, Percentile Rank, Normal Curve Equivalent, etc. Student Growth Percentiles are also reported after a least two tests, to help the teacher understand the student’s velocity relative to academic peers (those in the same grade with a similar score history). For students receiving intensive intervention, the software provides a goal setting tool that presents teachers with several goal options including alignment with state assessment proficiency levels, growth norms (Student Growth Percentile), and score norms (Percentile Rank). Once a goal has been set for a student, the student’s progress on Star Math is graphed against the goal line on the progress monitoring report.

Rates of Improvement and End of Year Benchmarks

Is minimum acceptable growth (slope of improvement or average weekly increase in score by grade level) specified in your manual or published materials?

Yes

If yes, specify the growth standards:

Star Math reports Student Growth Percentile for students with two or more tests. SGP describes a student’s velocity (slope) relative to a national sample of academic peers – those students in the same grade with a similar score history. SGPs work like Percentile Ranks (1-99) but once an SGP goal has been set, it is converted to a Scaled Score goal at the end date specified by the teacher, thus it is converted into an average weekly increase in a Scaled Score metric. We recommend SGP 50 (which indicates typical or expected growth) as minimum acceptable growth, though teachers are of course free to establish their own goals, as they best understand the student and the intensity of services to be provided.

Are benchmarks for minimum acceptable end-of-year performance specified in your manual or published materials?

Yes

If yes, specify the end-of-year performance standards:

Renaissance provides default risk benchmarks at the 10th (urgent intervention), 25th (intervention) and 40th (low risk, lower-bound threshold for working at grade level) Percentile Ranks. Those default cutpoints can be adjusted by educators but are based on expert guidance. For educators wanting to base decisions on likely performance on end-of-year state summative assessments, Renaissance also provides links to those state proficiency categories. Educators select from the default PR benchmarks, the state proficiency projections, or their own custom benchmarks.

What is the basis for specifying minimum acceptable growth and end of year benchmarks?

Norm-referenced
Criterion-referenced
Other

If other, please specify:

False

National representation (check all that apply):

Northeast:

New England

Middle Atlantic

Midwest:

East North Central

West North Central

South:

South Atlantic

East South Central

West South Central

West:

Mountain

Pacific

Local representation (please describe, including number of states)

Data from all 50 states plus the District of Columbia are included in the sample used to develop both growth norms (SGP) and score norms.

Date

Both types of norms updated just prior to the 2017-18 school year. Criterion-referenced benchmarks are based on scale linkages between state summative math tests and Star Math. Completion dates vary by state.

Size

SGP: 5.4 million students. Score norms: 1.9 million students. Criterion-referenced benchmarks (state proficiency indicators): Varies by state.

Gender (Percent)

Male

50

Female

50

Unknown

SES indicators (Percent)

Eligible for free or reduced-price lunch

(FRL data collected at a school level)

Other SES Indicators

Race/Ethnicity (Percent)

White, Non-Hispanic

55.2

Black, Non-Hispanic

19.6

Hispanic

19.0

American Indian/Alaska Native

1.9

Asian/Pacific Islander

4.3

Other

Unknown

Disability classification (Please describe)

Not available.

First language (Please describe)

Not available.

Language proficiency status (Please describe)

Not available.

Do you provide, in your user’s manual, norms which are disaggregated by race or ethnicity? If so, for which race/ethnicity?

White, Non-Hispanic
Black, Non-Hispanic
Hispanic
American Indian/Alaska Native
Asian/Pacific Islander
Other
Unknown

If criterion-referenced, describe procedure for specifying criterion for adequate growth and benchmarks for end-of-year performance levels.

For many districts, end-of-year performance on the state summative test is of extreme importance, and they may choose to define intervention cateogires based on state proficiency levels (e.g., students likely to place in the lowest proficiency category may be identified for intenstive intervention). Star Math predicts performance on end-of-year summative mathematics assessments, allowing educators to estimate where students are likely to place as they are being progress monitored throughout the year. This is accomplished by (1) a concurrent scale linking between Star Math and the state’s summative mathematics test scale (and accompanying proficiency cut points), and (2) growth norms that project students forward in time, to the time of the state assessment. Technical reports are available for each state linking study. The process used to project growth is Student Growth Percentile, and information on that approach can be found here. If educators choose to use their state math proficiency categories as their benchmarks, a number of Star Math dashboards and reports specify those benchmarks (equated to the Star Math scale) and the growth necessary to achieve those benchmarks.

Describe any other procedures for specifying adequate growth and minimum acceptable end of year performance.

Reliability

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

*Offer a justification for each type of reliability reported, given the type and purpose of the tool.

The internal consistency reliability coefficient estimates the proportion of variability within a single administration of a test that is due to inconsistency among the items that comprise the test.

*Describe the sample(s), including size and characteristics, for each reliability analysis conducted.

For each grade, a large sample (n = 131,103) of students completed Star Math assessments throughout the 2012–2013 and 2013–2014 school year.

*Describe the analysis procedures for each reported type of reliability.

Reliability was defined as the proportion of test score variance that is attributable to true variation in the trait the test measures, expressed analytically as:1-(σ^2 error)/(σ^2 total). where σ2 error is the variance of the errors of measurement, and σ2 total is the variance of test scores. The variance of the test scores was calculated from Scaled Score data. The variance of the errors of measurement was estimated from the conditional standard error of measurement (CSEM) statistics that accompany each of the IRT-based test scores, including the Scaled Scores. The conditional standard error of measurement (CSEM) was calculated along with the IRT ability estimate and Scaled Score. Squaring and summing the CSEM values yielded an estimate of total squared error; dividing by the number of observations yielded an estimate of error variance. Generic reliability was calculated by subtracting the ratio of error variance to Scaled Score variance from 1.

*In the table(s) below, report the results of the reliability analyses described above (e.g., model-based evidence, internal consistency or inter-rater reliability coefficients). Include detail about the type of reliability data, statistic generated, and sample size and demographic information.

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

Yes

Provide citations for additional published studies.

Renaissance Learning (2016). Star Assessments™ for Math Abridged Technical Manual. Wisconsin Rapids, WI: Author. Available by request to research@renaissance.com.

Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?

No

If yes, fill in data for each subgroup with disaggregated reliability data.

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

Provide citations for additional published studies.

Validity

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

*Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.

All criterion measures were external to the screening tool system and represent widely used assessments of general math ability. • ACT. The American College Testing college readiness assessment is a national standardized test for high school achievement and college admissions. • ACT Aspire: A nationally normed summative test of general mathematics achievement is designed to measure students' progress toward college and career readiness. • CAT-5. The California Achievement Test, is a nationally normed standardized test that measures achievement in mathematics. • IA. Iowa Assessments provide standardized mathematics tests as a service to schools by the College of Education of the University of Iowa. • M-CAP. Pearson’s aimsweb Mathematics Concepts and Applications is a brief, standardized test of problem solving skills and elements included in typical math curriculum. • NWEA MAP. Measures of Academic Progress offers an adaptive computerized test for Mathematics. • PARCC. The Partnership for Assessment of Readiness for College and Careers end-of-year assessment covers mathematics and is intended to be used as an indicator of student needs and progress. • SBA. Smarter Balanced assessments are summative tests designed to measure student achievement and growth in math to support teaching and learning. • SAT. The College Board’s SAT is widely used for college admissions in the United States that covers a range of math practices, including problem solving, modeling, using tools strategically, and using algebraic structure.

*Describe the sample(s), including size and characteristics, for each validity analysis conducted.

Samples included students who had taken both Star Math and the criterion measure. The sample sizes varied across criterion and grades, ranging from 17 to 10,800 students.

*Describe the analysis procedures for each reported type of validity.

Concurrent and predictive correlations were calculated. A criterion assessment was considered concurrent if it was taken during the same school year as the Star Math assessment. The correlation was considered predictive if the criterion assessment was one school year or more after the Star Math assessment.

*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Results from other forms of validity analysis not compatible with above table format:

Manual cites other published reliability studies:

Yes

Provide citations for additional published studies.

Provide citations for additional published studies. Renaissance Learning (2016). Star Assessments™ for Math Abridged Technical Manual. Wisconsin Rapids, WI: Author. Available by request to research@renaissance.com.

Describe the degree to which the provided data support the validity of the tool.

The data indicate that Star Math results strongly correspond to other various respected measures of general mathematics ability.

Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?

No

If yes, fill in data for each subgroup with disaggregated validity data.

Results from other forms of validity analysis not compatible with above table format:

Manual cites other published reliability studies:

No

Provide citations for additional published studies.

Bias Analysis

Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.

Yes

If yes,

a. Describe the method used to determine the presence or absence of bias:

Logistic regression analyses conditional on ability, group membership, and ability by group interaction were conducted to assess the presence of both uniform and non-uniform DIF simultaneously. Additionally, an effect size measure – Nagelkerke R-squared – was computed to quantify the magnitude of DIF where present.

b. Describe the subgroups for which bias analyses were conducted:

DIF analyses were conducted for gender (males and females) and race/ethnicity (Caucasian, African American, American Indian, Asian, and Hispanic subpopulations). Due to insufficient samples sizes on English Language Learner (ELLs) and students with disabilities (SWD), DIF analyses for these two subgroups were not possible at the time of the analyses.

c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.

Using a blended criterion that flagged items for uniform/non-uniform DIF if they had a p-value less than 0.01 and Nagelkerke R2 greater than or equal to 0.035, the results indicated that Star Math is sufficiently bias-free. A total of 391 items (4% of the Star Math items) were flagged for DIF. Those flagged items were removed from the item banks for review and recalibration.

Sensitivity: Reliability of Slope

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

Describe the sample, including size and characteristics. Please provide documentation showing that the sample was composed of students in need of intensive intervention. A sample of students with intensive needs should satisfy one of the following criteria: (1) all students scored below the 30th percentile on a local or national norm, or the sample mean on a local or national test fell below the 25th percentile; (2) students had an IEP with goals consistent with the construct measured by the tool; or (3) students were non-responsive to Tier 2 instruction. Evidence based on an unknown sample, or a sample that does not meet these specifications, may not be considered.

The sample consisted of students who took Star Math tests during the 2016-2017 school year. Sample sizes ranged from 397 to 18,460 students depending on grade.

Describe the frequency of measurement (for each student in the sample, report how often data were collected and over what span of time).

A student was included in the sample if they had performance below PR 30; tested at least ten times during the school year; and had 140 days or more between their first and last test.

Describe the analysis procedures.

For each student, tests were divided into two groups – odd and even numbered test depending on the chronological order in which they were taken. For each test group for each student, a slope was estimated by computing an OLS regression coefficient. The table below summarizes Pearson correlation coefficients as a measure of the strength of association between even and odd numbered test slopes.

In the table below, report reliability of the slope (e.g., ratio of true slope variance to total slope variance) by grade level (if relevant).

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

No

Provide citations for additional published studies.

Do you have reliability of the slope data that is disaggregated by subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)?

No

If yes, fill in data for each subgroup with disaggregated reliability of the slope data.

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

Provide citations for additional published studies.

Sensitivity: Validity of Slope

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.

Describe the sample(s), including size and characteristics. Please provide documentation showing that the sample was composed of students in need of intensive intervention. A sample of students with intensive needs should satisfy one of the following criteria: (1) all students scored below the 30th percentile on a local or national norm, or the sample mean on a local or national test fell below the 25th percentile; (2) students had an IEP with goals consistent with the construct measured by the tool; or (3) students were non-responsive to Tier 2 instruction. Evidence based on an unknown sample, or a sample that does not meet these specifications, may not be considered.

Describe the frequency of measurement (for each student in the sample, report how often data were collected and over what span of time).

Describe the analysis procedures for each reported type of validity.

In the table below, report predictive validity of the slope (correlation between the slope and achievement outcome) by grade level (if relevant).
NOTE: The TRC suggests controlling for initial level when the correlation for slope without such control is not adequate.

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published validity studies:

Provide citations for additional published studies.

Describe the degree to which the provided data support the validity of the tool.

Do you have validity of the slope data that is disaggregated by subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)?

If yes, fill in data for each subgroup with disaggregated validity of the slope data.

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published validity studies:

Provide citations for additional published studies.

Alternate Forms

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

Describe the sample for these analyses, including size and characteristics:

The analysis sample was comprised of students struggling in mathematics whose initial score was at or below the 25th percentile in Star Math. The data were drawn from students’ first and second tests of the year; the testing occasions were close in time to one another.

What is the number of alternate forms of equal and controlled difficulty?

Because Star Math is computer adaptive and comprises several thousand items, there are at a minimum several hundred alternate forms for a student at a given ability level.

If IRT based, provide evidence of item or ability invariance

Item invariance. See attachment.

If computer administered, how many items are in the item bank for each grade level?

Item counts by grade are included in the attachment.

If your tool is computer administered, please note how the test forms are derived instead of providing alternate forms:

Star Math is an instance of computer-administered adaptive testing, based on item response theory. It administers math test items selected one at a time to tailor item difficulty to the student’s own performance as it progresses during each test. The items have all been calibrated previously, using the 1-parameter logistic “Rasch” item response model. After each item is administered, the student’s score is updated, and a target Rasch difficulty level is calculated. The adaptive item selection algorithm seeks an item close to the target difficulty level, consistent with a content blueprint that specifies the number of items to be administered from each of five math content domains and 16 general skills. The adaptive item bank contains more than 6,000 items. The content constraints ensure that each test is parallel to others at the same grade level, in terms of content. The adaptive item selection ensures comparable measurement precision from one test to another. If a student’s performance differs little from one testing occasion to another, the successive tests can be expected to be closely parallel to one another in difficulty and score level.

Decision Rules: Setting & Revising Goals

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

In your manual or published materials, do you specify validated decision rules for how to set and revise goals?

Yes

If yes, specify the decision rules:

The Star Math software provides educators with guidance regarding the establishment of goals for each individual student to inform instructional decisions. There is not a single recommended goal for each grade level; instead, the goal setting approach is unique to each student and takes into consideration his/her prior achievement trajectory. The default goal for each individual student is based on national growth norms, as discussed elsewhere in the submission. Growth norms are used to understand typical and above typical rates of growth for a student based on comparisons with academic peers – those students in the same grade with a similar score history. Educators are encouraged to select either a typical rate of growth (one in which the student has about a 50% chance of attaining) or a more aggressive rate of growth (one in which the student has about a 35% chance of attaining. Educators may opt to select a different goal, but are not encouraged to select a goal below the 50% threshold, as students in intensive intervention are typically trying to catch up to grade level peers. Throughout the course of the intervention and progress monitoring, educators are encouraged to check student progress. Empirical research on Star decision rules (Cormier & Bulut, 2017, manuscript submitted for publication) has determined that decisions can safely be made after a minimum of five administrations. So, after at least five administrations, educators review the Star Math Progress Monitoring report. If actual student progress is at or above the student’s growth norm-based goal line, educators are encouraged to continue the intervention or revise the goal upward. If actual student progress is below the student’s growth norm-based goal line, educators are encouraged to alter the intervention and supports the student is receiving. A sample Star Progress Monitoring report is available from the Center upon request.

What is the evidentiary basis for these decision rules?
NOTE: The TRC expects evidence for this standard to include an empirical study that compares a treatment group to a control and evaluates whether student outcomes increase when decision rules are in place.

A quasi-experimental study was conducted involving more than 3.5 million students who began the year struggling in mathematics (defined as having a Star Math National Percentile Rank score below 30). Students were classified as treatment if they used the Star Math goal setting recommendations, and control if they did not. All students in the study (both treatment and comparison) were intensively progress monitored during the 2014-15 school year on an approximately weekly basis. (The duration of progress monitoring varied, with decisions up to local educators and unique to each student.) All students’ beginning (fall) and end of year (spring) Star Math Normal Curve Equivalent (NCE) scores were compared (posttest minus pretest), and are summarized by grade and condition in the table below. In every grade, 1-12, treatment students (using the Star Math goal recommendations) experienced significantly greater NCE change than their control counterparts who did not use the goal recommendations, providing support for the claim that using Star Math’s evidence-based goal recommendations and decision rules is superior to other locally determined approaches. Within each grade, the differences in NCE gains favoring treatment students were significant at p < .001. The results of this study suggest that students experience greater growth when educators follow the individualized goal and decision rule guidance provided by Star Math. The guidance is based on a combination of expert guidance and growth norms drawn from a large, diverse sample of students. A table detailing the results of this study is available from the Center upon request.

Decision Rules: Changing Instruction

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grade 9	Grade 10	Grade 11
Rating

Legend

Convincing evidence

Partially convincing evidence

Unconvincing evidence

Data unavailable

^dDisaggregated data available

In your manual or published materials, do you specify validated decision rules for when changes to instruction need to be made?

If yes, specify the decision rules:

What is the evidentiary basis for these decision rules?
NOTE: The TRC expects evidence for this standard to include an empirical study that compares a treatment group to a control and evaluates whether student outcomes increase when decision rules are in place.