Star Reading

Reading

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

There is a one-time setup fee along with a per student subscription fee. Total cost will depend on the number of schools and students. Please contact: answers@renaissance.com or (800) 338-4204 for specific details on pricing for your district.

 

Replacement Cost:

No information provided; contact vendor for details.

 

Included in Cost:

Star Reading is cloud-based and purchase includes the tool, software/technical manual, installation guide, testing instructions, and remote installation and setup.

Technology Requirements:

  • Computer or tablet
  • Internet connection

 

Training Requirements:

  • Less than 1 hour of training

 

Qualified Administrators:

  • No minimum qualifications

 

Accommodations:

Star Reading is a computer-adaptive assessment, and the difficulty of items presented is adjusted automatically to reflect the skill level of the student. Students may use the keyboard or the mouse, accommodating students with limited motor skills.  Star Reading offers several accommodations for students with disabilities through the accessibility options built into a computer's operating system. For students with limited vision, the introductory screens of Star Reading respond to the "high contrast" accessibility feature within Windows and the "switch to black and white" accessibility feature in Mac OS. The assessment screens within Star Reading already provide visual contrast through a yellow background and black writing. Furthermore, Star Reading is compatible with Mac’s "zoom in" accessibility feature, which allows users to magnify nearly all Star Reading screens.

Where to Obtain:

Website: http://www.renaissance.com

Address: Renaissance Learning, PO Box 8036, Wisconsin Rapids, WI 54495

Phone number: (800) 338-4204     

Email address: answers@renaissance.com


Access to Technical Support:

Renaissance Technical Support Staff

Star Reading is a computer-adaptive, interim assessment of reading comprehension and overall reading achievement for students in grades 1 through 12. The tool provides information on student performance in 46 reading skill areas in 11 domains.

Assessment Format:

  • Direct: Computerized
  • Group administered

 

Administration Time:

  • 18 minutes per student
  • 18 minutes per total group

 

Scoring Time:

  • Scoring is automatic

 

Scoring Method:

All scores are calculated automatically by the software. The software calculates a maximum likelihood Rasch ability estimate based on the calibrated difficulty of the items that were administered to the student, and the pattern of the student’s right and wrong responses to those items. Star Reading uses a proprietary, Rasch-based, 1-parameter logistic response model to calculate scores. The scaled score is a non-linear, monotonic transformation of the Rasch ability estimate resulting from the adaptive test. From the scaled scores and the student’s current grade placement are derived grade equivalent, percentile, and normal curve equivalent scores. No clusters, composite, or raw scores are reported.

 

Scores Generated:

  • Percentile score
  • Grade equivalents
  • IRT-based score            
  • Normal curve equivalents
  • Equated
  • Lexile score
  • Instructional Reading Level
  • Scaled score
  • Student Growth Percentile

 

Classification Accuracy

Grade1234567891011
Criterion 1 Falldashdashdashdashdashdashdashdashdashdashdash
Criterion 1 Winterdashdashdashdashdashdashdashdashdashdashdash
Criterion 1 SpringFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
Criterion 2 Falldashdashdashdashdashdashdashdashdashdashdash
Criterion 2 Winterdashdashdashdashdashdashdashdashdashdashdash
Criterion 2 SpringFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbledashHalf-filled bubbleHalf-filled bubble

Primary Sample

 

Criterion 1 Spring

 

Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

Grade 9

Grade 10

Grade 11

Criterion

NWEA MAP Reading

NWEA MAP Reading

PARCC Reading

PARCC Reading

PARCC Reading

NWEA MAP Reading

ACT Aspire

ACT Aspire

SAT Evidence Based Reading and Writing

SAT Evidence Based Reading and Writing

SAT Evidence Based Reading and Writing

Cut points

Star PR = 25, MAP PR =20

Star PR = 25, MAP PR =20

Star PR = 25, PARCC PR =20

Star PR = 25, PARCC PR =20

Star PR = 25, PARCC PR =20

Star PR = 25, MAP PR =20

Star PR = 25, Aspire PR =20

Star PR = 25, Aspire PR =20

Star PR = 25, SAT EBRW PR =20

Star PR = 25, SAT EBRW PR =20

Star PR = 25, SAT EBRW PR =20

Base rate in the sample for children requiring intensive intervention

0.11

0.18

0.25

0.30

0.29

0.29

0.08

0.07

0.14

0.15

0.16

Base rate in the sample for children considered at-risk, including those with the most intensive needs

0.26

0.22

0.32

0.34

0.37

0.35

0.16

0.18

0.27

0.24

0.27

False Positive Rate

0.20

0.10

0.16

0.15

0.19

0.20

0.11

0.13

0.18

0.15

0.18

False Negative Rate

0.24

0.27

0.22

0.20

0.19

0.28

0.27

0.20

0.16

0.27

0.24

Sensitivity

0.76

0.73

0.78

0.80

0.81

0.72

0.73

0.80

0.84

0.73

0.76

Specificity

0.80

0.90

0.84

0.85

0.81

0.80

0.89

0.87

0.82

0.85

0.82

Positive Predictive Power

0.32

0.61

0.62

0.70

0.63

0.59

0.37

0.32

0.43

0.46

0.46

Negative Predictive Power

0.96

0.94

0.92

0.91

0.91

0.87

0.97

0.98

0.97

0.95

0.95

Overall Classification Rate

0.80

0.86

0.82

0.84

0.81

0.77

0.88

0.86

0.83

0.83

0.81

Area Under the Curve (AUC)

0.86

0.91

0.90

0.91

0.89

0.85

0.91

0.92

0.90

0.88

0.88

AUC 95% Confidence Interval Lower

0.83

0.90

0.88

0.90

0.88

0.83

0.90

0.90

0.87

0.85

0.85

AUC 95% Confidence Interval Upper

0.88

0.93

0.91

0.92

0.90

0.87

0.93

0.94

0.93

0.91

0.94

At 90% Sensitivity, specificity equals

0.67

0.75

0.69

0.74

0.70

0.60

0.73

0.79

0.78

0.68

0.71

At 80% Sensitivity, specificity equals

0.78

0.87

0.83

0.85

0.83

0.71

0.86

0.87

0.84

0.81

0.81

At 70% Sensitivity, specificity equals

0.84

0.91

0.89

0.91

0.89

0.81

0.91

0.93

0.87

0.87

0.86

 

Criterion 2 Spring

 

Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

Grade 9

Grade 10

Grade 11

Criterion

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Pennsylvania System of School Assessment ELA

Not provided

Smarter Balanced ELA

Smarter Balanced ELA

Cut points

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Star PR = 25, PSSA PR =20

Not provided

Star PR = 25, SBA PR =20

Star PR = 25, SBA PR =20

Base rate in the sample for children requiring intensive intervention

0.14

0.19

0.18

0.17

0.19

0.15

0.17

0.14

Not provided

0.16

0.17

Base rate in the sample for children considered at-risk, including those with the most intensive needs

0.28

0.30

0.25

0.22

0.27

0.29

0.29

0.25

Not provided

0.36

0.32

False Positive Rate

0.20

0.18

0.13

0.11

0.14

0.19

0.17

0.17

Not provided

0.26

0.23

False Negative Rate

0.28

0.22

0.21

0.23

0.18

0.13

0.15

0.27

Not provided

0.10

0.22

Sensitivity

0.72

0.78

0.79

0.77

0.82

0.87

0.85

0.73

Not provided

0.90

0.78

Specificity

0.80

0.82

0.87

0.89

0.86

0.81

0.83

0.83

Not provided

0.74

0.77

Positive Predictive Power

0.37

0.51

0.57

0.57

0.58

0.45

0.50

0.42

Not provided

0.39

0.41

Negative Predictive Power

0.95

0.94

0.95

0.95

0.95

0.97

0.96

0.95

Not provided

0.98

0.95

Overall Classification Rate

0.78

0.81

0.86

0.87

0.85

0.82

0.83

0.82

Not provided

0.76

0.77

Area Under the Curve (AUC)

0.84

0.89

0.90

0.91

0.92

0.91

0.91

0.89

Not provided

0.90

0.85

AUC 95% Confidence Interval Lower

0.82

0.87

0.90

0.91

0.91

0.90

0.90

0.86

Not provided

0.87

0.84

AUC 95% Confidence Interval Upper

0.87

0.90

0.91

0.92

0.92

0.91

0.91

0.92

Not provided

0.93

0.87

At 90% Sensitivity, specificity equals

0.60

0.68

0.72

0.75

0.76

0.73

0.74

0.72

Not provided

0.74

0.57

At 80% Sensitivity, specificity equals

0.73

0.80

0.84

0.86

0.86

0.85

0.86

0.82

Not provided

0.82

0.75

At 70% Sensitivity, specificity equals

0.81

0.88

0.90

0.91

0.91

0.91

0.91

0.89

Not provided

0.88

0.84

 

 

Additional Classification Accuracy

The following are provided for context and did not factor into the Classification Accuracy ratings.

 

Cross-Validation Sample

 

Criterion 1 Spring

 

Grade 3

Grade 4

Grade 5

Grade 7

Grade 8

Grade 11

Criterion

PARCC Reading

PARCC Reading

PARCC Reading

ACT Aspire

ACT Aspire

SAT Evidence Based Reading and Writing

Cut points

Star PR = 25, PARCC PR =20

Star PR = 25, PARCC PR =20

Star PR = 25, PARCC PR =20

Star PR = 25, Aspire PR =20

Star PR = 25, Aspire PR =20

Star PR = 25, SAT EBRW PR =20

Base rate in the sample for children requiring intensive intervention

0.07

0.08

0.06

0.17

0.18

0.31

Base rate in the sample for children considered at-risk, including those with the most intensive needs

0.12

0.12

0.13

0.24

0.26

0.45

False Positive Rate

0.07

0.06

0.08

0.14

0.16

0.30

False Negative Rate

0.17

0.22

0.13

0.28

0.29

0.19

Sensitivity

0.83

0.78

0.88

0.72

0.71

0.81

Specificity

0.93

0.94

0.92

0.86

0.84

0.70

Positive Predictive Power

0.46

0.52

0.38

0.51

0.49

0.55

Negative Predictive Power

0.99

0.98

0.99

0.94

0.93

0.89

Overall Classification Rate

0.93

0.92

0.91

0.83

0.81

0.74

Area Under the Curve (AUC)

0.95

0.95

0.96

0.89

0.86

0.85

AUC 95% Confidence Interval Lower

0.92

0.92

0.94

0.86

0.84

0.82

AUC 95% Confidence Interval Upper

0.98

0.98

0.99

0.91

0.89

0.88

At 90% Sensitivity, specificity equals

0.89

0.88

0.90

0.74

0.60

0.59

At 80% Sensitivity, specificity equals

0.86

0.97

0.95

0.79

0.77

0.71

At 70% Sensitivity, specificity equals

0.98

0.97

0.98

0.87

0.85

0.83

 

 

Criterion 2 Spring

 

Grade 2

Criterion

Smarter Balanced ELA

Cut points

Star PR = 25, SBA PR =20

Base rate in the sample for children requiring intensive intervention

0.05

Base rate in the sample for children considered at-risk, including those with the most intensive needs

0.12

False Positive Rate

0.08

False Negative Rate

0.14

Sensitivity

0.86

Specificity

0.92

Positive Predictive Power

0.33

Negative Predictive Power

0.99

Overall Classification Rate

0.91

Area Under the Curve (AUC)

0.96

AUC 95% Confidence Interval Lower

0.94

AUC 95% Confidence Interval Upper

0.97

At 90% Sensitivity, specificity equals

0.91

At 80% Sensitivity, specificity equals

0.92

At 70% Sensitivity, specificity equals

0.96

 

Reliability

Grade1234567891011
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
  1. Justification for each type of reliability reported, given the type and purpose of the tool

The internal consistency reliability coefficient estimates the proportion of variability within a single administration of a test that is due to inconsistency among the items that comprise the test.

 

  1. Description of the sample(s), including size and characteristics, for each reliability analysis conducted

For each grade, a large sample (n = 100,000) of students completed Star Reading assessments throughout the 2012–2013 school year.

 

  1. Description of the analysis procedures for each reported type of reliability:

Reliability was defined as the proportion of test score variance that is attributable to true variation in the trait the test measures.  

The variance of the test scores was calculated from Scaled Score data. The variance of the errors of measurement was estimated from the conditional standard error of measurement (CSEM) statistics that accompany each of the IRT-based test scores, including the Scaled Scores. The conditional standard error of measurement (CSEM) was calculated along with the IRT ability estimate and Scaled Score. Squaring and summing the CSEM values yielded an estimate of total squared error; dividing by the number of observations yielded an estimate of error variance.

Generic reliability was calculated by subtracting the ratio of error variance to Scaled Score variance from 1.

 

  1. Reliability of performance level score (e.g., model-based, internal consistency, inter-rater reliability).

Type of Reliability

Age or Grade

n

Coefficient

Confidence Interval

Generic

Grade 1

100,000

0.95

0.95-0.95

Generic

Grade 2

100,000

0.94

0.94-0.94

Generic

Grade 3

100,000

0.93

0.93-0.93

Generic

Grade 4

100,000

0.93

0.93-0.93

Generic

Grade 5

100,000

0.93

0.93-0.93

Generic

Grade 6

100,000

0.93

0.93-0.93

Generic

Grade 7

100,000

0.93

0.93-0.93

Generic

Grade 8

100,000

0.94

0.94-0.94

Generic

Grade 9

100,000

0.94

0.94-0.94

Generic

Grade 10

100,000

0.94

0.94-0.94

Generic

Grade 11

100,000

0.95

0.95-0.95

Generic

Grade 12

100,000

0.95

0.95-0.95

 

Disaggregated Reliability

The following disaggregated reliability data are provided for context and did not factor into the Reliability rating.

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

Confidence Interval

None

 

 

 

 

 

 

Validity

Grade1234567891011
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
  1. Description of each criterion measure used and explanation as to why each measure is appropriate, given the type and purpose of the tool

All criterion measures were external to the screening tool system and represent widely used assessments of general reading ability.

  • CAT-5. The California Achievement Test, is a nationally normed standardized test that measures achievement in reading.
  • ITBS. The Iowa Tests For Basic Skills are nationally normed standardized tests that offer educators a diagnostic look at how their students are progressing in reading comprehension.
  • PARCC. The Partnership for Assessment of Readiness for College and Careers end-of-year assessment covers reading and is intended to be used as an indicator of student needs and progress.
  • SBA. Smarter Balanced assessments are summative tests designed to measure student achievement and growth in reading to support teaching and learning.
  • ACT. The American College Testing college readiness assessment is a national standardized test for high school achievement and college admissions.
  • SAT EBRW. The SAT Evidence-based reading and writing assessment is a standardized test widely used for college admissions in the United States.

 

  1. Description of the sample(s), including size and characteristics, for each validity analysis conducted:

Samples included students who had taken both Star Reading and the criterion measure.   The sample sizes varied across criterion and grades, ranging from 105 to 10,800 students.

 

  1. Description of the analysis procedures for each reported type of validity:

Concurrent and predictive correlations were calculated.  A criterion assessment was considered concurrent if it was taken during the same school year as the Star Reading assessment.  The correlation was considered predictive if the criterion assessment was taken one or more school years after the Star Reading assessment.

 

  1. Validity for the performance level score (e.g., concurrent, predictive, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Concurrent

Grade 1

MAP

2,007

0.79

0.77-0.81

Concurrent

Grade 1

CAT-5

68

0.79

0.68-0.86

Concurrent

Grade 2

CAT-5

315

0.72

0.66-0.77

Concurrent

Grade 2

ITBS

58

0.74

0.60-0.84

Concurrent

Grade 3

PARCC

1,391

0.83

0.81-0.85

Concurrent

Grade 3

SBA

11,068

0.82

0.81-0.83

Concurrent

Grade 4

PARCC

4,606

0.82

0.81-0.83

Concurrent

Grade 4

SBA

12,200

0.82

0.81-0.82

Concurrent

Grade 5

PARCC

5,391

0.78

0.77-0.79

Concurrent

Grade 5

SBA

10,908

0.83

0.82-0.84

Concurrent

Grade 6

PARCC

5,466

0.79

0.78-0.8

Concurrent

Grade 6

SBA

8,072

0.81

0.80-0.82

Concurrent

Grade 7

PARCC

5,197

0.8

0.79-0.81

Concurrent

Grade 7

SBA

6,320

0.83

0.82-0.84

Concurrent

Grade 8

PARCC

4,733

0.77

0.76-0.78

Concurrent

Grade 8

SBA

6,209

0.83

0.82-0.84

Concurrent

Grade 9

PARCC

3,344

0.77

0.76-0.78

Predictive

Grade 9

SAT grade 11

509

0.74

0.70-0.78

Concurrent

Grade 10

PARCC

1,901

0.73

0.71-0.75

Concurrent

Grade 10

SBA

261

0.74

0.68-0.79

Concurrent

Grade 11

ACT

1,776

0.70

0.67-0.72

Concurrent

Grade 11

CAT-5

48

0.79

0.65-0.88

Predictive

Grade 1

ACT Aspire grade 3

4,201

0.69

0.68-0.71

Predictive

Grade 2

ACT Aspire grade 3

5,136

0.81

0.8-0.82

Predictive

Grade 3

ACT Aspire

6,052

0.84

0.83-0.85

Predictive

Grade 4

ACT Aspire

6,102

0.82

0.81-0.83

Predictive

Grade 5

ACT Aspire

6,037

0.82

0.81-0.83

Predictive

Grade 6

ACT Aspire

5,480

0.82

0.81-0.83

Predictive

Grade 7

ACT Aspire

5,269

0.81

0.8-0.82

Predictive

Grade 8

ACT Aspire

5,552

0.79

0.78-0.8

Predictive

Grade 10

ACT Aspire

4,965

0.76

0.75-0.77

Predictive

Grade 11

ACT

10,839

0.71

0.70-0.72

 

  1. Results for other forms of validity (e.g. factor analysis) not conducive to the table format:

None provided

 

  1. Describe the degree to which the provided data support the validity of the tool

The provided data indicate that Star Reading results correspond to other various respected measures of general reading ability.

 

 

Disaggregated Validity

The following disaggregated validity data are provided for context and did not factor into the Validity rating.

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

None

 

 

 

 

 

 

 

Results for other forms of disaggregated validity (e.g. factor analysis) not conducive to the table format:

None provided

 

Sample Representativeness

Grade1234567891011
RatingEmpty bubbleHalf-filled bubbleFull bubbleFull bubbleFull bubbleHalf-filled bubbleFull bubbleFull bubbleEmpty bubbleHalf-filled bubbleFull bubble

Primary Sample

 

Criterion 1 Spring

 

Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

Grade 9

Grade 10

Grade 11

Criterion

NWEA MAP Reading

NWEA MAP Reading

PARCC Reading

PARCC Reading

PARCC Reading

NWEA MAP Reading

ACT Aspire

ACT Aspire

SAT Evidence Based Reading and Writing

SAT Evidence Based Reading and Writing

SAT Evidence Based Reading and Writing

Representation

Mountain. New Mexico

Mountain. New Mexico

New England, Middle Atlantic, West South Central, Mountain. Colorado, Arkansas, Rhode Island, New Jersey, Ohio, Illinois, New Mexico

New England, Middle Atlantic, West South Central, Mountain. Colorado, Arkansas, Rhode Island, New Jersey, Ohio, Illinois, New Mexico

New England, Middle Atlantic, West South Central, Mountain. Colorado, Arkansas, Rhode Island, New Jersey, Ohio, Illinois, New Mexico

East North Central, Mountain. New Mexico and Wisconsin

South Atlantic, East South Central. AL, SC

South Atlantic, East South Central. AL, SC

East North Central, Mountain. ID, IL, MI

New England, East North Central, Mountain. CT, ID, IL, MI

New England, East North Central, Mountain. CT, ID, IL, ME, MI

Date

2007-08, 2008-09, 2009-10 and 2012-13 school years

2006-07, 2007-08, 2008-09, 2009-10, 2010-11, 2011-12 and 2012-13 school years

2014-15 school year

2014-15 school year

2014-15 school year

2008-09, 2009-10, 2010-11, 2011-12, 2012-13 and 2013-14 school years

2013-2014, 2014-2015, 2015-2016 school years

2013-2014, 2014-2015, 2015-2016 school years

2013-2014, 2014-2015 school years

2014-2015, 2015-2016 school years

2015-2016, 2016-17 school years

Size

1,365

2,150

1,756

3,932

3,813

1,224

2,948

3,029

509

689

880

Male

51.70%

48.80%

52.20%

50.50%

50.70%

47.20%

17.6%

17.1%

46.80%

46.40%

47.60%

Female

48.00%

50.90%

47.80%

49.50%

49.30%

44.90%

17.4%

17.0%

45.80%

46.60%

46.70%

Unknown

0.30%

0.20%

Not provided

Not provided

Not provided

7.90%

65.0%

65.9%

7.50%

7.00%

5.70%

Other SES Indicators

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Free or reduced-price lunch

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

White, Non-Hispanic

0.70%

1.00%

2.70%

9.30%

9.50%

2.90%

60.7%

59.2%

41.80%

43.30%

37.40%

Black, Non-Hispanic

0.00%

0.00%

4.20%

13.80%

13.30%

0.10%

20.0%

21.5%

1.20%

0.90%

2.00%

Hispanic

49.90%

49.80%

37.90%

45.40%

46.60%

40.50%

6.3%

5.4%

36.30%

32.40%

26.50%

American Indian/Alaska Native:

0.00%

0.00%

0.00%

0.60%

0.40%

0.00%

0.7%

0.8%

0.60%

0.70%

1.40%

Asian/Pacific Islander:

0.00%

0.00%

0.60%

2.50%

3.00%

0.00%

6.9%

7.4%

4.50%

3.30%

3.20%

Other

0.00%

0.00%

0.30%

3.20%

2.90%

0.00%

2.4%

2.4%

0.00%

0.00%

13.40%

Unknown

49.50%

49.20%

54.20%

25.20%

24.40%

56.50%

3.1%

3.3%

15.50%

19.40%

16.10%

Disability classification

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

First language

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Language proficiency status

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

 

Criterion 2 Spring

 

Grade 1

Grade 2

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

Grade 10

Grade 11

Criterion

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Pennsylvania System of School Assessment ELA

Smarter Balanced ELA

Smarter Balanced ELA

Representation

Pacific. CA, OR, WA

New England, Pacific. CA, CT, OR, WA

New England, Pacific. CA, CT, OR, WA

New England, Pacific. CA, CT, OR, WA

New England, Pacific. CA, CT, OR, WA

New England, Pacific. CA, CT, OR, WA

New England, Pacific. CA, CT, OR, WA

Middle Atlantic. PA

Pacific. WA

New England, Pacific. CA, CT, OR, WA

Date

2012-13, 2014-15 school years

2013-14, 2014-15 school years

2014-2015 school year

2014-2015 school year

2014-2015 school year

2014-2015 school year

2014-2015 school year

2014-2015 school year

2014-2105 school year

2014-2105 school year

Size

1,850

5,576

8,475

8,522

8,516

8,420

8,515

616

711

2,937

Male

47.8%

50.2%

50.50%

51.20%

51.40%

50.90%

50.10%

48.10%

48.70%

50.90%

Female

52.2%

49.6%

49.20%

48.50%

48.40%

48.90%

49.60%

51.60%

51.30%

49.10%

Unknown

0.0%

0.1%

0.30%

0.20%

0.20%

0.20%

0.20%

0.30%

0

0

Other SES Indicators

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Not Provided

Free or reduced-price lunch

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Not Provided

White, Non-Hispanic

26.3%

35.2%

37.80%

37.60%

38.90%

39.00%

39.10%

35.70%

56.10%

24.80%

Black, Non-Hispanic

4.7%

5.9%

5.40%

5.20%

5.00%

4.70%

5.40%

1.30%

2.40%

7.00%

Hispanic

52.6%

39.2%

32.90%

32.50%

30.40%

30.60%

30.30%

1.80%

20.80%

33.00%

American Indian/Alaska Native:

0.1%

0.7%

1.10%

1.10%

1.20%

1.00%

1.20%

0.00%

4.10%

0.60%

Asian/Pacific Islander:

7.1%

8.4%

12.30%

13.10%

12.20%

13.50%

12.50%

0.50%

0.09

0.044

Other

3.3%

2.0%

3.90%

3.60%

4.10%

3.60%

3.90%

0.00%

7.60%

2.80%

Unknown

5.8%

8.6%

6.70%

6.90%

8.10%

7.50%

7.60%

60.70%

0%

27.40%

Disability classification

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

First language

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Language proficiency status

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

 

 

Cross-Validation Sample

 

Criterion 1 Spring

 

Grade 3

Grade 4

Grade 5

Grade 7

Grade 8

Grade 11

Criterion

PARCC Reading

PARCC Reading

PARCC Reading

ACT Aspire

ACT Aspire

SAT Evidence Based Reading and Writing

Representation

Middle Atlantic (NJ)

Middle Atlantic (NJ)

Middle Atlantic (NJ)

East South Central (AL)

East South Central (AL)

East North Central (IL)

Date

2014-15, 2015-16, 2016-17 school years

2014-15, 2015-16, 2016-17 school years

2014-15, 2015-16, 2016-17 school years

2014-15, 2015-16, school years

2014-15, 2015-16, school years

2016-17 school year

Size

712

717

724

1,293

1,310

632

Male

50.3%

49.4%

52.2%

49.3%

47.7%

47.9%

Female

49.7%

50.6%

47.8%

50.7%

52.3%

52.1%

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Other SES Indicators

Not provided

Not provided

Not provided

Unknown

Unknown

Unknown

Free or reduced-price lunch

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

White, Non-Hispanic

11.2%

11.4%

13.0%

93.2%

93.8%

5.9%

Black, Non-Hispanic

6.9%

6.3%

6.4%

0.0%

0.0%

87.7%

Hispanic

50.1%

51.6%

50.7%

0.0%

0.0%

2.5%

American Indian/Alaska Native:

0.1%

0.3%

0.3%

1.5%

0.9%

0.3%

Asian/Pacific Islander:

29.5%

28.9%

28.0%

0.8%

1.0%

3.6%

Other

2.1%

1.5%

1.7%

0.0%

0.0%

0.0%

Unknown

Not provided

Not provided

Not provided

4.5%

4.3%

0.0%

Disability classification

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

First language

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

Language proficiency status

Unknown

Unknown

Unknown

Unknown

Unknown

Unknown

 

 

Criterion 2 Spring

 

Grade 2

Criterion

Smarter Balanced ELA

Representation

Pacific (WA)

Date

2013-14, 2014-15 school years

Size

1,099

Male

50.6

Female

49.4

Unknown

Unknown

Other SES Indicators

Unknown

Free or reduced-price lunch

Unknown

White, Non-Hispanic

41.9

Black, Non-Hispanic

1.8

Hispanic

9.9

American Indian/Alaska Native

0.2

Asian/Pacific Islander

36.9

Other

9.3

Unknown

0.0

Disability classification

Unknown

First language

Unknown

Language proficiency status

Unknown

 

Bias Analysis Conducted

Grade1234567891011
RatingYesYesYesYesYesYesYesYesYesYesYes
  1. Description of the method used to determine the presence or absence of bias:

Logistic regression analyses conditional on ability, group membership,

and ability by group interaction were conducted to assess the presence of both uniform and non-uniform DIF simultaneously. Additionally, an effect size measure – Nagelkerke R-squared – was computed to quantify the magnitude of DIF where present.

 

  1. Description of the subgroups for which bias analyses were conducted:

DIF analyses were conducted for gender (males and females) and race/ethnicity (Caucasian, African American, American Indian, Asian, and Hispanic subpopulations). Due to insufficient samples sizes on English Language Learner (ELLs) and students with disabilities (SWD), DIF analyses for these two subgroups were not possible at the time of the analyses.  

 

  1. Description of the results of the bias analyses conducted, including data and interpretative statements:

Using a blended criterion that flagged items for uniform/non-uniform DIF if they had a p-value less than 0.01 and Nagelkerke R2 greater than or equal to 0.035, the results indicated that Star Reading is sufficiently bias-free. A total of 153 items (2% of the Star Reading items) were flagged for DIF. Those flagged items were removed from the item banks for review and recalibration.

Administration Format

Grade1234567891011
Data
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Administration & Scoring Time

    Grade1234567891011
    Data
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • 18 minutes
  • Scoring Format

    Grade1234567891011
    Data
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Types of Decision Rules

    Grade1234567891011
    Data
  • None
  • None
  • None
  • None
  • None
  • None
  • None
  • None
  • None
  • None
  • None
  • Evidence Available for Multiple Decision Rules

    Grade1234567891011
    Data
  • No
  • No
  • No
  • No
  • No
  • No
  • No
  • No
  • No
  • No
  • No