Star CBM
Passage Oral Reading (formerly CES Standard Reading Passages)

Descriptive Information

Please provide a description of your tool:

The Star CBM Passage Oral Reading measures (formerly Children’s Educational Services’ Standard Reading Passages) include grade-level passages for teachers and other school personnel to use in screening to establish student achievement in reading, to set goals for those students, to monitor their progress, and to evaluate the effectiveness of instruction for the purpose of modifying instruction when indicated. The essential procedures were developed by researchers at the University of Minnesota beginning in 1977 and the years following. The reading passages were initially written in 1988 and disseminated as the Standard Reading Passages by Children’s Educational Services. In 2019, those passages were acquired by Renaissance Learning and incorporated into Star Curriculum Based Measurement as the Passage Oral Reading measures.

The tool is intended for use with the following grade(s).

Preschool / Pre - kindergarten
Kindergarten
First grade
Second grade
Third grade
Fourth grade
Fifth grade
Sixth grade
Seventh grade
Eighth grade
Ninth grade
Tenth grade
Eleventh grade
Twelfth grade

The tool is intended for use with the following age(s).

0-4 years old
5 years old
6 years old
7 years old
8 years old
9 years old
10 years old
11 years old
12 years old
13 years old
14 years old
15 years old
16 years old
17 years old
18 years old

The tool is intended for use with the following student populations.

Students in general education
Students with disabilities
English language learners

ACADEMIC ONLY: What skills does the tool screen?

Reading

Phonological processing:

RAN
Memory
Awareness
Letter sound correspondence
Phonics
Structural analysis

Word ID

Accuracy
Speed

Nonword

Accuracy
Speed

Spelling

Accuracy
Speed

Passage

Accuracy
Speed

Reading comprehension:

Multiple choice questions
Cloze
Constructed Response
Retell
Maze
Sentence verification
Other (please describe):

Listening comprehension:

Multiple choice questions
Cloze
Constructed Response
Retell
Maze
Sentence verification
Vocabulary
Expressive
Receptive

Mathematics

Global Indicator of Math Competence

Accuracy
Speed
Multiple Choice
Constructed Response

Early Numeracy

Accuracy
Speed
Multiple Choice
Constructed Response

Mathematics Concepts

Accuracy
Speed
Multiple Choice
Constructed Response

Mathematics Computation

Accuracy
Speed
Multiple Choice
Constructed Response

Mathematic Application

Accuracy
Speed
Multiple Choice
Constructed Response

Fractions/Decimals

Accuracy
Speed
Multiple Choice
Constructed Response

Algebra

Accuracy
Speed
Multiple Choice
Constructed Response

Geometry

Accuracy
Speed
Multiple Choice
Constructed Response

Other (please describe):

Please describe specific domain, skills or subtests:

BEHAVIOR ONLY: Which category of behaviors does your tool target?

Internalizing
Externalizing
Internalizing and Externalizing

BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.

Acquisition and Cost Information

Where to obtain:

Email Address

answers@renaissance.com

Address

Renaissance Learning, PO Box 8036, Wisconsin Rapids, WI 54495

Phone Number

(800) 338-4204

Website

http://www.renaissance.com

Initial cost for implementing program:

Cost

Unit of cost

Replacement cost per unit for subsequent use:

Cost

Unit of cost

Duration of license

Additional cost information:

Describe basic pricing plan and structure of the tool. Provide information on what is included in the published tool, as well as what is not included but required for implementation.

Total cost will depend on the number of schools and students; annual subscription required. Please contact: answers@renaissance.com or (800) 338-4204 for specific details on pricing. Star CBM is cloud-based and purchase includes the application, technical manual, administration instructions, and quick-start guidance for professional learning.

Provide information about special accommodations for students with disabilities.

Administration

BEHAVIOR ONLY: What type of administrator is your tool designed for?

General education teacher
Special education teacher
Parent
Child
External observer
Other

If other, please specify:

What is the administration setting?

Direct observation
Rating scale
Checklist
Performance measure
Questionnaire
Direct: Computerized
One-to-one
Other

If other, please specify:

Does the tool require technology?

Yes

If yes, what technology is required to implement your tool? (Select all that apply)

Computer or tablet
Internet connection
Other technology (please specify)

If your program requires additional technology not listed above, please describe the required technology and the extent to which it is combined with teacher small-group instruction/intervention:

What is the administration context?

Individual
Small group   If small group, n=
Large group   If large group, n=
Computer-administered
Other

If other, please specify:

What is the administration time?

Time in minutes

1

per (student/group/other unit)

student

Additional scoring time:

Time in minutes

1

per (student/group/other unit)

student

ACADEMIC ONLY: What are the discontinue rules?

No discontinue rules provided
Basals
Ceilings
Other

If other, please specify:

Are norms available?

Yes

Are benchmarks available?

Yes

If yes, how many benchmarks per year?

Three: Fall, Winter, Spring

If yes, for which months are benchmarks available?

September, January, May

BEHAVIOR ONLY: Can students be rated concurrently by one administrator?

If yes, how many students can be rated concurrently?

Training & Scoring

Training

Is training for the administrator required?

Yes

Describe the time required for administrator training, if applicable:

Less than one hour of training

Please describe the minimum qualifications an administrator must possess.

No minimum qualifications

Are training manuals and materials available?

Yes

Are training manuals/materials field-tested?

Yes

Are training manuals/materials included in cost of tools?

Yes

If No, please describe training costs:

Can users obtain ongoing professional and technical support?

Yes

If Yes, please describe how users can obtain support:

Renaissance Technical Support Staff

Scoring

How are scores calculated?

Manually (by hand)
Automatically (computer-scored)
Other

If other, please specify:

Do you provide basis for calculating performance level scores?

Yes

What is the basis for calculating performance level and percentile scores?

Age norms
Grade norms
Classwide norms
Schoolwide norms
Stanines
Normal curve equivalents

What types of performance level scores are available?

Raw score
Standard score
Percentile score
Grade equivalents
IRT-based score
Age equivalents
Stanines
Normal curve equivalents
Developmental benchmarks
Developmental cut points
Equated
Probability
Lexile score
Error analysis
Composite scores
Subscale/subtest scores
Other

If other, please specify:

Words read correctly per minute

Does your tool include decision rules?

No

If yes, please describe.

Can you provide evidence in support of multiple decision rules?

No

If yes, please describe.

Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.

Count of number of words read correctly in one minute of reading aloud. Note that the score may be adjusted as a result of equating forms to ensure difficulty is consistent within grade.

Describe the tool’s approach to screening, samples (if applicable), and/or test format, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.

Three of each set of passages are used for screening to establish students’ current level of proficiency in reading aloud from text and are used to establish cut scores for identifying students who are academically at risk as readers.

Classification Accuracy & Cross-Validation Summary

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Classification Accuracy Fall
Classification Accuracy Winter
Classification Accuracy Spring

Measures of Academic Progress (MAP)

Classification Accuracy

Select time of year

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

The Measures of Academic Progress (MAP) was used as the outcome measure. Published by the NWEA the MAP is regarded as a highly valid and reliable measure of broad reading ability. The NWEA website states, “Our tools are trusted by educators in 140 countries and more than half the schools in the US” which indicates it can be considered an excellent outcome measure for classification studies on the Oral Reading measure studied here.

Do the classification accuracy analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Both Star CBM and MAP assessments were administered during the 2019-2020 school year. Within 3 months of taking the STAR CBM POR assessment, students also completed a MAP assessment.

Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Using the SPSS program ROC analyses were conducted on the sample. The Oral Reading data (Words Read Correctly) was administered in the Winter of grade 1. MAP was used to determine students “at-risk” and in “intensive need” and was administered three months later in the Spring (as specified in the NCII criteria). The 20th percentile on the MAP was used to determine the cut-points on the MAP. The results of the ROC analysis provided the Area Under the Curve (AUC) with 95% confidence intervals for determining the lower and upper bounds. In addition, the ROC analysis provides the Coordinates of the Curve provide Sensitivity and Specificity estimates. In addition, the coordinates are used to determine “risk” cut-points on Oral Reading that are associated with different levels of sensitivity and specificity. The cut-points on the Oral Reading were then used for further classification analysis in determining overall correct classification rates, false positives, false negative, Positive Predictive Power and Negative Predictive Power.

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

No

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

Cross-Validation

Has a cross-validation study been conducted?

No

If yes,

Select time of year.

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Do the cross-validation analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

---

Minnesota Comprehensive Assessment (MCA) - Reading

Classification Accuracy

Select time of year

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

The 2nd criterion used in our classification analysis is the Minnesota Comprehensive Assessment (MCA) in reading. The MCA is the state accountability test for Minnesota and has been established by the Minnesota Department of Education to be a highly reliable and valid measure or reading. See http://education.state.mn.us/MDE/dse/test/mn/Tech/ for more technical information. The MCA is an external measure.

Do the classification accuracy analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Using the SPSS program ROC analyses were conducted on the sample. The Oral Reading data (Words Read Correctly) was administered in the Winter of grade 3. MCA was used to determine students “at-risk” and in “intensive need” and was administered three months later in the Spring (as specified in the NCII criteria). The 20th percentile on the MCA was used to determine the cut-points on the MCA. The results of the ROC analysis provided the Area Under the Curve (AUC) with 95% confidence intervals for determining the lower and upper bounds. In addition, the ROC analysis provides the Coordinates of the Curve provide Sensitivity and Specificity estimates. In addition, the coordinates are used to determine “risk” cut-points on Oral Reading that are associated with different levels of sensitivity and specificity. The cut-points on the Oral Reading were then used for further classification analysis in determining overall correct classification rates, false positives, false negatives, Positive Predictive Power and Negative Predictive Power.

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

No

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

Cross-Validation

Has a cross-validation study been conducted?

No

If yes,

Select time of year.

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Do the cross-validation analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

---

Dynamic Indicators of Basic Early Literacy Skills (DIBELS)

Classification Accuracy

Select time of year

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

The Dynamic Indicators of Basic Early Literacy Skills (DIBELS) was developed by University of Oregon's Center on Teaching and Learning (CTL) and assesses the acquisition of early literacy skills. The DIBELS Oral Reading Fluency (ORF) is a standardized test of accuracy and fluency with connected text.

Do the classification accuracy analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Both Star CBM and DIBELS ORF assessments were administered during the 2019-2020 school year. Within 3 months of taking the STAR CBM POR assessment, students also completed a DIBELS ORF assessment.

Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

A ROC analysis was used to compare performance on Star CBM to DIBELS ORF performance. Selection of Star CBM cut scores was based on striking an optimal balance between specificity and sensitivity when classifying students as “at risk”. For these analyses, students scoring below the Star cut score are classified as “high risk” and students scoring at or above the Star cut score are classified as “low risk”. Students scoring below the 20th percentile on the criterion measure are considered to be “actually at risk” and students scoring at or above the 20th percentile on the criterion measure are considered to be “actually not at risk”. As a result, students classified as “at risk” on Star and are considered to be “actually at risk” represent a true positive. Students classified as “at risk” on Star, but are “actually not at risk” represent a false positive. Students classified as “not at risk” on Star, but are “actually at risk” represent a false negative. Students classified as “not at risk” on Star and are “actually not at risk” represent a true negative.

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

Cross-Validation

Has a cross-validation study been conducted?

No

If yes,

Select time of year.

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Do the cross-validation analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

---

Star Reading

Classification Accuracy

Select time of year

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Star Reading is a online computer adaptive assessment of general reading ability developed by Renaissance Learning. Although they now share a common name, Star Reading is entirely independent to Star CBM POR. The Star CBM POR measures were recently purchased by Renaissance Learning, the publisher of Star Assessments, in 2019. The POR measures were originally named Standard Reading Passages, and were developed by Dr. Stanley Deno and Dr. Douglas Martson in 1985 under the auspices of Children’s Educational Services, Inc. Therefore the POR and Star Reading measures were developed entirely independently of one another, at different times (the POR passages predate Star Reading by more than a decade), by different individuals and organizations.

Do the classification accuracy analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Both Star CBM and Star Reading assessments were administered during the 2019-2020 school year. Within 3 months of taking the STAR CBM POR assessment, students also completed a Star Reading assessment.

Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

A ROC analysis was used to compare performance on Star CBM to Star Reading performance. Selection of Star CBM cut scores was based on striking an optimal balance between specificity and sensitivity when classifying students as “at risk”. For these analyses, students scoring below the Star cut score are classified as “high risk” and students scoring at or above the Star cut score are classified as “low risk”. Students scoring below the 20th percentile on the criterion measure are considered to be “actually at risk” and students scoring at or above the 20th percentile on the criterion measure are considered to be “actually not at risk”. As a result, students classified as “at risk” on Star and are considered to be “actually at risk” represent a true positive. Students classified as “at risk” on Star, but are “actually not at risk” represent a false positive. Students classified as “not at risk” on Star, but are “actually at risk” represent a false negative. Students classified as “not at risk” on Star and are “actually not at risk” represent a true negative.

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

Cross-Validation

Has a cross-validation study been conducted?

No

If yes,

Select time of year.

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Do the cross-validation analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

---

Classification Accuracy - Fall

Evidence	Grade 1	Grade 2	Grade 4	Grade 5	Grade 6
Criterion measure	Dynamic Indicators of Basic Early Literacy Skills (DIBELS)	Dynamic Indicators of Basic Early Literacy Skills (DIBELS)	Measures of Academic Progress (MAP)	Measures of Academic Progress (MAP)	Measures of Academic Progress (MAP)
Cut Points - Percentile rank on criterion measure	20	20	20	20	20
Cut Points - Performance score on criterion measure
Cut Points - Corresponding performance score (numeric) on screener measure	19	57	81	96	115
Classification Data - True Positive (a)	14	13	22	21	25
Classification Data - False Positive (b)	4	4	15	18	21
Classification Data - False Negative (c)	3	3	4	5	5
Classification Data - True Negative (d)	45	68	144	191	190
Area Under the Curve (AUC)	0.93	0.96	0.94	0.94	0.91
AUC Estimate’s 95% Confidence Interval: Lower Bound	0.87	0.93	0.90	0.91	0.85
AUC Estimate’s 95% Confidence Interval: Upper Bound	0.99	1.00	0.98	0.98	0.97

Statistics	Grade 1	Grade 2	Grade 4	Grade 5	Grade 6
Base Rate	0.26	0.18	0.14	0.11	0.12
Overall Classification Rate	0.89	0.92	0.90	0.90	0.89
Sensitivity	0.82	0.81	0.85	0.81	0.83
Specificity	0.92	0.94	0.91	0.91	0.90
False Positive Rate	0.08	0.06	0.09	0.09	0.10
False Negative Rate	0.18	0.19	0.15	0.19	0.17
Positive Predictive Power	0.78	0.76	0.59	0.54	0.54
Negative Predictive Power	0.94	0.96	0.97	0.97	0.97

Sample	Grade 1	Grade 2	Grade 4	Grade 5	Grade 6
Date	Fall 2019	Fall 2019	Fall 2019	Fall 2019	Fall 2019
Sample Size	66	88	185	235	241
Geographic Representation	East South Central (AL)	East South Central (AL)	West North Central (MN)	West North Central (MN)	West North Central (MN)
Male	51.5%	45.5%	49.2%	47.2%	47.7%
Female	48.5%	54.5%	50.8%	52.8%	52.3%
Other
Gender Unknown
White, Non-Hispanic	80.3%	63.6%	82.2%	87.2%	43.6%
Black, Non-Hispanic	16.7%	34.1%	3.8%	3.0%	19.1%
Hispanic			4.3%	4.3%	26.1%
Asian/Pacific Islander
American Indian/Alaska Native			1.6%		4.1%
Other	3.0%	2.3%	8.1%	5.5%	7.1%
Race / Ethnicity Unknown
Low SES
IEP or diagnosed disability
English Language Learner

Classification Accuracy - Winter

Evidence	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Criterion measure	Dynamic Indicators of Basic Early Literacy Skills (DIBELS)	Dynamic Indicators of Basic Early Literacy Skills (DIBELS)	Measures of Academic Progress (MAP)	Minnesota Comprehensive Assessment (MCA) - Reading	Measures of Academic Progress (MAP)	Measures of Academic Progress (MAP)
Cut Points - Percentile rank on criterion measure	20	20	20	20	20	20
Cut Points - Performance score on criterion measure
Cut Points - Corresponding performance score (numeric) on screener measure	19	69		94		128
Classification Data - True Positive (a)	7	12	181	46	119	25
Classification Data - False Positive (b)	2	2	176	14	316	35
Classification Data - False Negative (c)	0	1	46	9	28	6
Classification Data - True Negative (d)	22	57	1540	185	1231	179
Area Under the Curve (AUC)	0.98	0.96	0.94	0.95	0.89	0.90
AUC Estimate’s 95% Confidence Interval: Lower Bound	0.93	0.90	0.92	0.92	0.86	0.83
AUC Estimate’s 95% Confidence Interval: Upper Bound	1.00	1.00	0.95	0.98	0.92	0.96

Statistics	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Base Rate	0.23	0.18	0.12	0.22	0.09	0.13
Overall Classification Rate	0.94	0.96	0.89	0.91	0.80	0.83
Sensitivity	1.00	0.92	0.80	0.84	0.81	0.81
Specificity	0.92	0.97	0.90	0.93	0.80	0.84
False Positive Rate	0.08	0.03	0.10	0.07	0.20	0.16
False Negative Rate	0.00	0.08	0.20	0.16	0.19	0.19
Positive Predictive Power	0.78	0.86	0.51	0.77	0.27	0.42
Negative Predictive Power	1.00	0.98	0.97	0.95	0.98	0.97

Sample	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Date	Winter 2020	Winter 2020	Winter 2017 and Spring 2017	Winter 2020	Winter 2017 and Spring 2017	Winter 2020
Sample Size	31	72	1943	254	1694	245
Geographic Representation	East South Central (AL)	East South Central (AL)	West North Central (MN)	West North Central (MN)	West North Central (MN)	West North Central (MN)
Male	54.8%	44.4%	50.8%	48.4%	51.2%	48.2%
Female	45.2%	55.6%	49.2%	51.6%	48.8%	51.8%
Other
Gender Unknown
White, Non-Hispanic	90.3%	63.9%	78.1%	50.8%	77.6%	44.1%
Black, Non-Hispanic	6.5%	33.3%	6.5%	20.9%	6.7%	19.2%
Hispanic			8.9%	19.3%	10.0%	26.9%
Asian/Pacific Islander
American Indian/Alaska Native			1.1%	3.1%	1.3%	3.3%
Other	3.2%	2.8%	5.5%	5.9%	4.4%	6.5%
Race / Ethnicity Unknown
Low SES			24.6%		26.7%
IEP or diagnosed disability			11.7%		12.3%
English Language Learner			5.0%		2.2%

Classification Accuracy - Spring

Evidence	Grade 4	Grade 5
Criterion measure	Star Reading	Star Reading
Cut Points - Percentile rank on criterion measure	20	20
Cut Points - Performance score on criterion measure
Cut Points - Corresponding performance score (numeric) on screener measure	89	89
Classification Data - True Positive (a)	85	46
Classification Data - False Positive (b)	48	32
Classification Data - False Negative (c)	15	4
Classification Data - True Negative (d)	410	238
Area Under the Curve (AUC)	0.94	0.95
AUC Estimate’s 95% Confidence Interval: Lower Bound	0.92	0.91
AUC Estimate’s 95% Confidence Interval: Upper Bound	0.97	0.99

Statistics	Grade 4	Grade 5
Base Rate	0.18	0.16
Overall Classification Rate	0.89	0.89
Sensitivity	0.85	0.92
Specificity	0.90	0.88
False Positive Rate	0.10	0.12
False Negative Rate	0.15	0.08
Positive Predictive Power	0.64	0.59
Negative Predictive Power	0.96	0.98

Sample	Grade 4	Grade 5
Date	Spring 2020	Spring 2020
Sample Size	558	320
Geographic Representation	West North Central (MN)	West North Central (MN)
Male	54.3%	49.7%
Female	45.7%	50.3%
Other
Gender Unknown
White, Non-Hispanic	40.7%	31.3%
Black, Non-Hispanic	44.6%	52.5%
Hispanic	7.0%	8.4%
Asian/Pacific Islander
American Indian/Alaska Native	2.7%	3.8%
Other	5.0%	4.1%
Race / Ethnicity Unknown
Low SES
IEP or diagnosed disability
English Language Learner

Reliability

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Rating		d	d	d	d	d

*Offer a justification for each type of reliability reported, given the type and purpose of the tool.

Reliability information and data provided in the last submission (March 2011) remains the same for Internal Consistency and Alternate Form reliabity analyses. However, since the most recent reliability standards published by NCII require confidence intervals there is new evidence provided. This information is presented in #4 below.

*Describe the sample(s), including size and characteristics, for each reliability analysis conducted.

*Describe the analysis procedures for each reported type of reliability.

*In the table(s) below, report the results of the reliability analyses described above (e.g., internal consistency or inter-rater reliability coefficients).

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

No

Provide citations for additional published studies.

Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?

Yes

If yes, fill in data for each subgroup with disaggregated reliability data.

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

No

Provide citations for additional published studies.

Validity

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Rating	d	d	d	d	d	d

*Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.

The Measures of Academic Progress (MAP) was used as the outcome measure. Published by the NWEA the MAP is regarded as a highly valid and reliable measure of broad reading ability. The NWEA website states, “Our tools are trusted by educators in 140 countries and more than half the schools in the US” which indicates it can be considered an excellent outcome measure for classification studies on the Oral Reading measure studied here. See https://www.nwea.org/normative-data-rit-scores/ for more information. The MAP is an external measure. A second criterion used in our analysis is the Minnesota Comprehensive Assessment (MCA) and reading. The MCA is the state accountability test for Minnesota and has been established by the Minnesota Department of Education to be a highly reliable and valid measure or reading. See http://education.state.mn.us/MDE/dse/test/mn/Tech/ for more technical information. The MCA is an external measure.

*Describe the sample(s), including size and characteristics, for each validity analysis conducted.

For the Concurrent and Predictive validity analyses for Passage Reading and MAP there was a total sample of 9,410 students (grades 1-6). Demographic data indicated 48.4% of the sample was female, 12.4% was special education, 24.0% received Title services, 7.5% was served in a gifted program, 38.2% received free or reduced lunch, and 5.1% received ELL service. Ethnic percentages for this sample were 1.3% for American Indian, 4.9% for Asian American, 10.3% for Hispanic American, 7.7% for Black American, and 75.8% for White American. For the Concurrent and Predictive validity analyses for Passage Reading and MCA there was a total sample of 11,094 students (grades 3-6). Demographic data indicated 48.2% of the sample was female, 12.4% was special education, 27.2% received Title services, 9.3% was served in a gifted program, 27.6% received free or reduced lunch, and 7.7% received ELL service. Ethnic percentages for this sample were 2.6% for American Indian, 4.0% for Asian American, 9.4% for Hispanic American, 12.2% for Black American, and 71.8% for White American.

*Describe the analysis procedures for each reported type of validity.

Two types of validity analysis were conducted; Concurrent and Predictive. The Concurrent study used Pearson Product Moment Correlational analysis to examine the relationship between Passage Reading (Words Read Correctly) and the MAP and the MCA. All measures were administered in the Spring of 2017. The Predictive study used Pearson Product Moment Correlational analysis to examine the relationship between Passage Reading (Words Read Correctly) and the MAP and the MCA. Passage Reading was administered in the Winter of 2017 and the criterion measures of MAP and MCA were administered in the Spring of 2017, approximately three months later.

*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of validity analysis not compatible with above table format:

Manual cites other published reliability studies:

No

Provide citations for additional published studies.

Describe the degree to which the provided data support the validity of the tool.

Validity coefficients for both concurrent and predictive validity are typically high. The median correlation coefficient for concurrent validity is .73 and is .70 for predictive validity.

Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?

Yes

If yes, fill in data for each subgroup with disaggregated validity data.

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of validity analysis not compatible with above table format:

Manual cites other published reliability studies:

No

Provide citations for additional published studies.

Bias Analysis

Grade	Grade 1	Grade 2	Grade 3	Grade 4	Grade 5	Grade 6
Rating	Not Provided	Not Provided	Not Provided	Not Provided	Not Provided	Not Provided

Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.

Yes

If yes,

a. Describe the method used to determine the presence or absence of bias:

First, the authors conducted a measurement invariance analysis using invariance testing in multiple-group confirmatory factor models. Second, the authors fit several regression models in which the spring benchmark progress monitoring score was regressed on a score on an external standardized measure of broad reading ability, The Measures of Academic Progress (MAP). In addition to the spring MAP assessment score, the authors included various demographic variables in the model to estimate main effects and potential interaction effects between the demographic variables and the MAP score.

b. Describe the subgroups for which bias analyses were conducted:

The subgroups for the measurement invariance analysis were formed based on students' race/ethnicity and included White students, African American students, Asian students, Hispanic students, and American Indian students. The subgroups used for the regression analysis were formed based on demographic factors including ethnicity, free and reduced lunch eligibility, and English Language Learner (ELL) status.

c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.

Measurement Invariance Analysis Results: Overall, every ethnic group’s model-fit well. All relative fit indices (CFI, TLI) were above .98, which indicates very good model fit, given that a value over .90 is suggested to be a reasonably good fit for CFI and TLI (Hu & Bentler, 1998). In addition, the 90 percent confidence interval for RMSEA indicates very good model fit, given that a value below or at .05 is considered a very close approximate fit (Browne & Cudeck, 1993). These results show that all CBM-R assessments appeared to function equally well as one factor for all ethnic groups. For the first three comparisons (White vs. African American/Asian/Hispanic), the configural (equal structure) model had a good fit, and thus a series of model constraints were then applied in successive models to examine potential decrease in model fit. Two nested models were compared in model fit based on chi-square and degree of freedom differences at α= .05. The full metric invariance model fit well; but resulted in a marginally significant decrease in fit (chi-square difference) relative to the configural model. The partial metric invariance model (freeing just one loading), however, did not result in a significant decrease in fit, which indicates that CBM-R measures seem to be related to the factor equivalently between White and other groups. The full scalar model fit well; but also resulted in a marginally significant decrease in fit relative to the partial metric model. However, the partial scalar model (freeing just one intercept) did not result in a significant decrease in fit, which means that both ethnic groups (with the same level of reading) are expected to perform on CBM-R assessments in a very similar way. In case of the comparison between White and American Indian, both the full metric invariance model and the full scalar model did not result in a significant decrease in fit, which indicates that the same latent factor was being measured for the two groups and they have exactly the same expected performance on CBM-R if their levels are the same. These results suggest that the same structure (configural invariance) and the same latent factor (full/partial metric invariance) were being measured in each ethnic group. In addition, the fact that the partial or full scalar invariance held indicates that White and other ethnic groups (with the same level of reading) have almost the same expected performance on CBM-R. In conclusion, there was no evidence on structural/measurement invariance (i.e., test bias) between White and other ethnic groups, meaning that CBM-R appears to be an unbiased measure among different ethnic groups. Test bias (measurement invariance) analysis results using invariance testing in multiple-group confirmatory factor models. Tables with detailed results from these analyses are available from the Center upon request. Regression Analysis Results: The authors interpreted any main effects associated with the demographic variables, or interaction effects between the demographic variables and the MAP assessment score as evidence of bias in passages (analogous to differential item functioning). Most P-values exceed the .001 level of significance. Tables with detailed results from these analyses are available from the Center upon request.