Acadience Reading (aka DIBELS Next)

Composite Score

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

Free Version: No cost

Print Version: $3.64 – $3.72 per student

Mobile Version: $14.90 per student

 

Replacement Cost:

Free Version: No cost

Print Version: $3.62 – $3.74 per student per year

Mobile Version: $14.90 per student per year
Annual license renewal fees subject to change.

 

Included in Cost:

Users adopting the free version can download all materials and supporting documents from https://acadiencelearning.org/acadiencereading.html - however, users may incur some costs when copying/printing materials. Data management and reporting for Acadience Reading paper/pencil users are available from Acadience Data Management, a service provided by Dynamic Measurement Group. Acadience Data Management is optional and costs $1.00 per student per school year.

Users purchasing the mobile version will receive access to all materials as well as an online administration platform and data reporting system.

Users purchasing the print version will receive a classroom set, which includes all necessary materials for assessing 25 students.

 

Technology Requirements:

  • No technology is required*

*Users who choose Amplify’s mobile device version will need internet access and a tablet or computer.

 

Training Requirements:

  • 4-8 hours of training

 

Qualified Administrators:

  • No minimum qualifications specified*

*No specific expertise is required, but training in the specific administration and scoring procedures should be provided.

 

Accommodations:

Acadience Reading is appropriate for most students for whom an instructional goal is to learn to read in English. For English language learners who are learning to read in English, Acadience Reading is appropriate for assessing and monitoring progress in acquisition of early reading skills. For all Acadience Reading measures (including those that comprise the Reading Composite), students are never penalized for articulation or dialect differences that are part of their typical speech. In addition, Acadience Reading measures include discontinue rules to prevent student frustration, as well as a set of approved accommodations that assessors may use when appropriate (see Acadience Reading Assessment Manual).

There are a few groups of students for whom Acadience Reading is not appropriate: (a) students who are learning to read in a language other than English; (b) students who are deaf; (c) students who have fluency-based speech disabilities such as stuttering (if the stuttering affects the student's response fluency within a one-minute timed assessment) and oral apraxia; and (d) students with severe disabilities for whom learning to read connected text is not an IEP goal.

Assessment accommodations are used for those students for whom the standard administration conditions would not produce accurate results. Approved accommodations are those accommodations which are unlikely to change how the assessment functions. When approved accommodations are used, the scores can be reported and interpreted as official Acadience Reading scores (see Acadience Reading Assessment Manual for a list of approved accommodations). Approved accommodations should be used only for students for whom the accommodations are necessary to provide an accurate assessment of student skills.

Unapproved accommodations are accommodations that are likely to change how the assessment functions (such as modifying the timing rules). Scores from measures administered with unapproved accommodations should not be treated or reported as official Acadience Reading scores and cannot be compared to other Acadience Reading scores or benchmark goals but can be used to measure individual growth for a student. An unapproved accommodation may be used when (a) a student cannot be tested accurately using the standardized rules or approved accommodations, but the school would still like to measure progress for that student; or (b) a student’s Individualized Education Plan (IEP) requires testing with an unapproved accommodation.

Where to Obtain:

Free Version:

Website: https://acadiencelearning.org/acadiencereading.html

Address: Dynamic Measurement Group, 859 Willamette Street, Suite 320, Eugene, OR 97410

Phone number: 541-431-6931 or toll free 888-943-1240

Email: info@acadiencelearning.org

 

Print Version (published under the name DIBELS Next®):

Website: http://voyagersopris.com

Address: Voyager Sopris Learning, 17855 Dallas Parkway, Suite 400, Dallas, TX 75287-6816  

Telephone: (800) 547-6747

 

Mobile Version (published under the name mCLASS: DIBELS Next®):

Website: www.amplify.com

Address: Amplify, 55 Washington Street, Suite 800, Brooklyn, NY 11201

Telephone: (800) 823-1969


Access to Technical Support:

Dynamic Measurement Group provides customer support for all Acadience Reading assessments, as well as support for the data management and reporting system, Acadience Data Management. Staff are available by phone and email on weekdays from 7am to 5pm Pacific Time at no additional cost. The majority of customer support requests are resolved in less than one business day.

Acadience Reading (also published as DIBELS Next®)1 is a screening and progress monitoring assessment used to measure early literacy and reading skills for students in grades K-6. The Acadience Reading screening measures are efficient indicators of recognized research-based foundational, or core, early literacy skills. The foundational early literacy skills are those skills that all students must master in order to become proficient readers, including Phonemic Awareness, Phonics and Word Recognition, and Fluency, to support comprehension. For each grade and time of year, the Acadience Reading component measures that correlate highly with later outcomes are combined to form a Reading Composite Score. The component measures used in the Composite Score depend upon grade and time of year. The Composite Score predicts later outcomes and conveys that all of the aspects of reading proficiency are critical.

 

1 AcadienceTM is a trademark of Dynamic Measurement Group, Inc. (DMG). DMG sold the DIBELS® and DIBELS Next® trademarks to the University of Oregon; DMG's continued use of the DIBELS Next mark is by license from UO. The DIBELS Next copyrighted content and copyright continues to be owned by DMG.

Assessment Format:

  • Direct observation
  • Performance measure
  • One-to-one

 

Administration Time:

  • 3-8 minutes per student

 

Scoring Time:

  • .5-3 minutes per student*

*Scoring is automatic for users who purchase Amplify’s mobile device version or who purchase the Acadience Data Management system.

 

Scoring Method:

  • Calculated manually*

*Scoring is automatic for users who purchase Amplify’s mobile device version or who purchase the Acadience Data Management system.

 

Scores Generated:

  • Raw score
  • Percentile score
  • Developmental benchmarks
  • Lexile score*
  • Composite scores

*Available with the mobile device version or with Acadience Data Management.

 

Classification Accuracy

GradeK123456
Criterion 1 FallHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleFull bubbleFull bubbleFull bubble
Criterion 1 WinterHalf-filled bubbleFull bubbleFull bubbleHalf-filled bubbleFull bubbleFull bubbleHalf-filled bubble
Criterion 1 SpringHalf-filled bubbleFull bubbleFull bubbleHalf-filled bubbleFull bubbleFull bubbleFull bubble
Criterion 2 FalldashdashFull bubbleHalf-filled bubbleFull bubbleFull bubbleFull bubble
Criterion 2 WinterdashdashFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
Criterion 2 SpringdashdashFull bubbleFull bubbleFull bubbleHalf-filled bubbleFull bubble

Primary Sample

 

Criterion 1, Fall

Grade

K

1

2

3

4

5

6

Criterion

GRADE

GRADE

GRADE

GRADE

GRADE

GRADE

GRADE

Cut points: Percentile rank on criterion measure

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

Cut points: Performance score (numeric) on criterion measure

70

55

80

78

41

39

53

Cut points: Corresponding performance score (numeric) on screener measure

13

97

109

180

245

258

280

Base rate in the sample for children requiring intensive intervention

0.20

0.20

0.20

0.20

0.20

0.20

0.20

False Positive Rate

0.17

0.16

0.04

0.12

0.08

0.11

0.13

False Negative Rate

0.36

0.31

0.36

0.31

0.17

0.27

0.18

Sensitivity

0.64

0.69

0.64

0.69

0.83

0.73

0.82

Specificity

0.83

0.84

0.96

0.88

0.92

0.89

0.87

Positive Predictive Power

0.26

0.24

0.80

0.51

0.71

0.55

0.43

Negative Predictive Power

0.96

0.97

0.91

0.94

0.97

0.95

0.98

Overall Classification Rate

0.81

0.83

0.89

0.85

0.92

0.87

0.86

Area Under the Curve (AUC)

0.79

0.80

0.92

0.88

0.92

0.88

0.90

AUC 95% Confidence Interval Lower Bound

0.70

0.72

0.87

0.81

0.87

0.81

0.83

AUC 95% Confidence Interval Upper Bound

0.87

0.88

0.97

0.95

0.95

0.91

0.97

 

Criterion 1, Winter

Grade

K

1

2

3

4

5

6

Criterion

GRADE

GRADE

GRADE

GRADE

GRADE

GRADE

GRADE

Cut points: Percentile rank on criterion measure

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

Cut points: Performance score (numeric) on criterion measure

70

55

80

78

41

39

53

Cut points: Corresponding performance score (numeric) on screener measure

85

100

145

235

290

310

285

Base rate in the sample for children requiring intensive intervention

0.20

0.20

0.20

0.20

0.20

0.20

0.20

False Positive Rate

0.17

0.09

0.07

0.12

0.08

0.13

0.14

False Negative Rate

0.33

0.22

0.18

0.31

0.15

0.30

0.33

Sensitivity

0.67

0.78

0.82

0.69

0.85

0.70

0.67

Specificity

0.83

0.91

0.93

0.88

0.92

0.87

0.86

Positive Predictive Power

0.28

0.64

0.71

0.55

0.64

0.41

0.32

Negative Predictive Power

0.96

0.96

0.96

0.93

0.97

0.96

0.96

Overall Classification Rate

0.82

0.89

0.91

0.85

0.91

0.85

0.84

Area Under the Curve (AUC)

0.78

0.91

0.94

0.89

0.91

0.87

0.87

AUC 95% Confidence Interval Lower Bound

0.70

0.86

0.90

0.83

0.86

0.82

0.78

AUC 95% Confidence Interval Upper Bound

0.87

0.96

0.98

0.95

0.97

0.93

0.96

 

Criterion 1, Spring

Grade

K

1

2

3

4

5

6

Criterion

GRADE

GRADE

GRADE

GRADE

GRADE

GRADE

GRADE

Cut points: Percentile rank on criterion measure

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

20th Percentile

Cut points: Performance score (numeric) on criterion measure

70

55

80

78

41

39

53

Cut points: Corresponding performance score (numeric) on screener measure

89

111

180

280

330

340

324

Base rate in the sample for children requiring intensive intervention

0.20

0.20

0.20

0.20

0.20

0.20

0.20

False Positive Rate

0.20

0.09

0.09

0.12

0.08

0.13

0.10

False Negative Rate

0.40

0.21

0.18

0.22

0.11

0.30

0.20

Sensitivity

0.60

0.79

0.83

0.53

0.89

0.70

0.80

Specificity

0.80

0.91

0.91

0.88

0.92

0.87

0.90

Positive Predictive Power

0.10

0.61

0.61

0.53

0.68

0.41

0.57

Negative Predictive Power

0.99

0.96

0.97

0.96

0.98

0.96

0.96

Overall Classification Rate

0.80

0.89

0.89

0.87

0.92

0.85

0.88

Area Under the Curve (AUC)

0.71

0.89

0.94

0.86

0.94

0.87

0.92

AUC 95% Confidence Interval Lower Bound

0.62

0.83

0.90

0.79

0.90

0.82

0.84

AUC 95% Confidence Interval Upper Bound

0.80

0.95

0.97

0.93

 

0.98

0.93

0.97

 

Criterion 2, Fall

Grade

K

1

2

3

4

5

6

Criterion

Not Provided

Not Provided

CST Reading

CST Reading

CST Reading

CST Reading

CST Reading

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

≤ 350

≤ 315

≤ 350

≤ 350

≤ 350

Cut points: Corresponding performance score (numeric) on screener measure

Not Provided

Not Provided

109

180

245

258

280

Base rate in the sample for children requiring intensive intervention

Not Provided

Not Provided

0.20

0.20

0.25

0.27

0.18

False Positive Rate

Not Provided

Not Provided

0.13

0.12

0.13

0.12

0.12

False Negative Rate

Not Provided

Not Provided

0.25

0.32

0.24

0.23

0.24

Sensitivity

Not Provided

Not Provided

0.75

0.68

0.76

0.77

0.76

Specificity

Not Provided

Not Provided

0.87

0.88

0.87

0.88

0.88

Positive Predictive Power

Not Provided

Not Provided

0.63

0.43

0.58

0.67

0.52

Negative Predictive Power

Not Provided

Not Provided

0.92

0.95

0.94

0.93

0.95

Overall Classification Rate

Not Provided

Not Provided

0.85

0.86

0.85

0.86

0.86

Area Under the Curve (AUC)

Not Provided

Not Provided

0.90

0.87

0.89

0.90

0.88

AUC 95% Confidence Interval Lower Bound

Not Provided

Not Provided

0.88

0.85

0.87

0.88

0.85

AUC 95% Confidence Interval Upper Bound

Not Provided

Not Provided

0.92

0.87

0.91

0.92

0.92

 

Criterion 2, Winter

Grade

K

1

2

3

4

5

6

Criterion

Not Provided

Not Provided

CST Reading

CST Reading

CST Reading

CST Reading

CST Reading

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

350

315

350

350

350

Cut points: Corresponding performance score (numeric) on screener measure

Not Provided

Not Provided

145

180

290

310

285

Base rate in the sample for children requiring intensive intervention

Not Provided

Not Provided

0.20

0.20

0.25

0.27

0.18

False Positive Rate

Not Provided

Not Provided

0.11

0.12

0.13

0.13

0.12

False Negative Rate

Not Provided

Not Provided

0.20

0.30

0.25

0.27

0.27

Sensitivity

Not Provided

Not Provided

0.80

0.70

0.75

0.73

0.73

Specificity

Not Provided

Not Provided

0.89

0.89

0.87

0.87

0.88

Positive Predictive Power

Not Provided

Not Provided

0.67

0.44

0.57

0.63

0.55

Negative Predictive Power

Not Provided

Not Provided

0.94

0.96

0.94

0.91

0.94

Overall Classification Rate

Not Provided

Not Provided

0.87

0.87

0.84

0.83

0.85

Area Under the Curve (AUC)

Not Provided

Not Provided

0.92

0.88

0.89

0.89

0.89

AUC 95% Confidence Interval Lower Bound

Not Provided

Not Provided

0.90

0.86

0.87

0.87

0.86

AUC 95% Confidence Interval Upper Bound

Not Provided

Not Provided

0.94

0.91

0.91

0.91

0.92

 

Criterion 2, Spring

Grade

K

1

2

3

4

5

6

Criterion

Not Provided

Not Provided

CST Reading

CST Reading

CST Reading

CST Reading

CST Reading

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

350

315

350

350

350

Cut points: Corresponding performance score (numeric) on screener measure

Not Provided

Not Provided

180

280

330

340

324

Base rate in the sample for children requiring intensive intervention

Not Provided

Not Provided

0.20

0.20

0.25

0.27

0.18

False Positive Rate

Not Provided

Not Provided

0.12

0.12

0.13

0.14

0.13

False Negative Rate

Not Provided

Not Provided

0.18

0.30

0.23

0.31

0.30

Sensitivity

Not Provided

Not Provided

0.82

0.70

0.77

0.69

0.70      

Specificity

Not Provided

Not Provided

0.88

0.88

0.87

0.86

0.87

Positive Predictive Power

Not Provided

Not Provided

0.63

0.47

0.59

0.59

0.49

Negative Predictive Power

Not Provided

Not Provided

0.95

0.95

0.94

0.91

0.94

Overall Classification Rate

Not Provided

Not Provided

0.87

0.86

0.85

0.82

0.85

Area Under the Curve (AUC)

Not Provided

Not Provided

0.92

0.89

0.89

0.89

0.87

AUC 95% Confidence Interval Lower Bound

Not Provided

Not Provided

0.90

0.87

0.87

0.87

0.84

AUC 95% Confidence Interval Upper Bound

Not Provided

Not Provided

0.93

0.91

0.92

0.91

0.90

 

Additional Classification Accuracy

The following are provided for context and did not factor into the Classification Accuracy ratings.

 

Cross-Validation Sample

Fall

Grade

K

1

2

3

4

5

6

Criterion

Not Provided

Not Provided

Not Provided

SBAC ELA

SBAC ELA

SBAC ELA

Not Provided

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Not Provided

< 2432

< 2473

< 2502

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

Not Provided

Not Provided

Not Provided

180

245

258

Not Provided

Base rate in the sample for children requiring intensive intervention

Not Provided

Not Provided

Not Provided

0.22

0.38

0.34

Not Provided

False Positive Rate

Not Provided

Not Provided

Not Provided

0.39

0.21

0.24

Not Provided

False Negative Rate

Not Provided

Not Provided

Not Provided

0.19

0.13

0.36

Not Provided

Sensitivity

Not Provided

Not Provided

Not Provided

0.81

0.87

0.64

Not Provided

Specificity

Not Provided

Not Provided

Not Provided

0.61

0.79

0.76

Not Provided

Positive Predictive Power

Not Provided

Not Provided

Not Provided

0.28

0.56

0.45

Not Provided

Negative Predictive Power

Not Provided

Not Provided

Not Provided

0.95

0.95

0.88

Not Provided

Overall Classification Rate

Not Provided

Not Provided

Not Provided

0.79

0.81

0.74

Not Provided

Area Under the Curve (AUC)

Not Provided

Not Provided

Not Provided

0.79

0.86

0.82

Not Provided

AUC 95% Confidence Interval Lower Bound

Not Provided

Not Provided

Not Provided

0.71

0.78

0.74

Not Provided

AUC 95% Confidence Interval Upper Bound

Not Provided

Not Provided

Not Provided

0.87

0.94

0.90

Not Provided

 

Winter

Grade

K

1

2

3

4

5

6

Criterion

Not Provided

Not Provided

Not Provided

SBAC ELA

SBAC ELA

SBAC ELA

Not Provided

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Not Provided

< 2432

< 2473

< 2502

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

Not Provided

Not Provided

Not Provided

235

290

310

Not Provided

Base rate in the sample for children requiring intensive intervention

Not Provided

Not Provided

Not Provided

0.22

0.38

0.34

Not Provided

False Positive Rate

Not Provided

Not Provided

Not Provided

0.20

0.20

0.18

Not Provided

False Negative Rate

Not Provided

Not Provided

Not Provided

0.14

0.17

0.24

Not Provided

Sensitivity

Not Provided

Not Provided

Not Provided

0.86

0.83

0.77

Not Provided

Specificity

Not Provided

Not Provided

Not Provided

0.80

0.80

0.82

Not Provided

Positive Predictive Power

Not Provided

Not Provided

Not Provided

0.48

0.60

0.62

Not Provided

Negative Predictive Power

Not Provided

Not Provided

Not Provided

0.96

0.93

0.90

Not Provided

Overall Classification Rate

Not Provided

Not Provided

Not Provided

0.85

0.81

0.81

Not Provided

Area Under the Curve (AUC)

Not Provided

Not Provided

Not Provided

0.84

0.86

0.83

Not Provided

AUC 95% Confidence Interval Lower Bound

Not Provided

Not Provided

Not Provided

0.76

0.78

0.75

Not Provided

AUC 95% Confidence Interval Upper Bound

Not Provided

Not Provided

Not Provided

0.91

0.93

0.91

Not Provided

 

Spring

Grade

K

1

2

3

4

5

6

Criterion

Not Provided

Not Provided

Not Provided

SBAC ELA

SBAC ELA

SBAC ELA

Not Provided

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Not Provided

< 2432

< 2473

< 2502

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

Not Provided

Not Provided

Not Provided

280

330

340

Not Provided

Base rate in the sample for children requiring intensive intervention

Not Provided

Not Provided

Not Provided

0.22

0.38

0.34

Not Provided

False Positive Rate

Not Provided

Not Provided

Not Provided

0.17

0.22

0.23

Not Provided

False Negative Rate

Not Provided

Not Provided

Not Provided

0.14

0.27

0.36

Not Provided

Sensitivity

Not Provided

Not Provided

Not Provided

0.86

0.74

0.65

Not Provided

Specificity

Not Provided

Not Provided

Not Provided

0.84

0.78

0.77

Not Provided

Positive Predictive Power

Not Provided

Not Provided

Not Provided

0.31

0.60

0.48

Not Provided

Negative Predictive Power

Not Provided

Not Provided

Not Provided

0.99

0.87

0.87

Not Provided

Overall Classification Rate

Not Provided

Not Provided

Not Provided

0.84

0.76

0.74

Not Provided

Area Under the Curve (AUC)

Not Provided

Not Provided

Not Provided

0.83

0.88

0.80

Not Provided

AUC 95% Confidence Interval Lower Bound

Not Provided

Not Provided

Not Provided

0.75

0.81

0.71

Not Provided

AUC 95% Confidence Interval Upper Bound

Not Provided

Not Provided

Not Provided

0.91

0.93

0.88

Not Provided

 

Reliability

GradeK123456
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
  1. Justification for each type of reliability reported, given the type and purpose of the tool: Reliability refers to the relative stability with which a test measures the same skills across minor differences in conditions. Two types of reliability are reported in the table below, alternate form reliability and alpha. Alternate form reliability is the correlation between different measures of the same early literacy skills. The coefficient reported is the average correlation among three forms of the measure. Coefficient alpha is a measure of reliability that is widely used in education research and represents the proportion of true score to total variance. Alpha incorporates information about the average inter-test correlation as well as the number of tests.

 

  1. Description of the sample(s), including size and characteristics, for each reliability analysis conducted: The data used for assessing reliability came from kindergarten through sixth grade. The total sample size is 1,255 students from 13 schools within 5 school districts. The sample was drawn from two census regions (Pacific and North Central Midwest).

 

  1. Description of the analysis procedures for each reported type of reliability: Alternate form reliability is reported as the average correlation among three alternate forms of the same test. High alternate form reliability coefficients suggest that these multiple forms are measuring the same construct. Coefficient alpha treats each of the three tests as separate indicators and is calculated using the alternate form reliability, where the number of tests is equal to three.

 

  1. Reliability of performance level score (e.g., model-based, internal consistency, inter-rater reliability).

Type of Reliability

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Alternate Form

K

166

0.66

0.57

0.74

Alternate Form

1

196

0.95

0.57

0.73

Alternate Form

2

219

0.92

0.90

0.94

Alternate Form

3

187

0.97

0.96

0.98

Alternate Form

4

187

0.95

0.93

0.96

Alternate Form

5

195

0.91

0.88

0.93

Alternate Form

6

105

0.91

0.87

0.94

Alpha

K

166

0.85

0.81

0.89

Alpha

1

196

0.98

0.97

0.98

Alpha

2

219

0.97

0.96

0.98

Alpha

3

187

0.99

0.99

0.99

Alpha

4

187

0.98

0.97

0.98

Alpha

5

195

0.97

0.96

0.98

Alpha

6

105

0.97

0.96

0.98

 

Disaggregated Reliability

The following disaggregated reliability data are provided for context and did not factor into the Reliability rating.

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

If your manual cites other published studies on reliability, provide these citations: Dewey, E. N., Powell-Smith, K. A., Good, R. H., & Kaminski, R. A. (2015). Acadience Reading Technical Adequacy Brief. Eugene: Dynamic Measurement Group.

Validity

GradeK123456
RatingEmpty bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble

1.Description of each criterion measure used and explanation as to why each measure is appropriate, given the type and purpose of the tool: The Group Reading Assessment and Diagnostic Evaluation (GRADE) is an untimed, group-administered, norm-referenced reading achievement test appropriate for children in preschool through grade 12. The GRADE is comprised of 16 subtests within five components. Not all 16 subtests are used at each testing level. Various subtest scores are combined to form the Total Test composite score. The GRADE Total Test raw score was compared to all Acadience Reading measures given during the year, providing both predictive criterion-related validity correlations for beginning- and middle-of-year Acadience Reading measures and concurrent criterion-related validity data for end-of-year Acadience Reading measures. The GRADE Total Test score is comprised of scores across subtests of the GRADE that vary by grade level. In kindergarten, the GRADE Total Test score is comprised of measures that assess phonics and phonemic and phonological awareness. In first and second grade, the GRADE Total Test includes word meaning, passage (or sentence) reading, and comprehension measures. In third grade, the GRADE Total Test is comprised of measures assessing word reading, vocabulary, and comprehension. In fourth, fifth, and sixth grade, the GRADE Total Test includes scores from measures of vocabulary and comprehension.

The California Standards Test (CST) is a statewide achievement test produced for California public schools and was designed to assess the California content standards for English/language arts (ELA), mathematics, history–social science, and science in grades two through eleven. The Reading cluster of the ELA portion of the CST was chosen as the criterion. According to a technical report from ETS (2011), the CST items were developed and designed to conform to principles of item writing defined by ETS (ETS, 2002). In addition, the items selected underwent an extensive item review process designed to provide the best standards-based tests possible.

 

2.Description of the sample(s), including size and characteristics, for each validity analysis conducted: The data used for assessing validity came from kindergarten through sixth grade. The total sample size is 5,504 students from 27 schools. The sample was drawn from two census regions (Pacific and North Central Midwest).

 

3.Description of the analysis procedures for each reported type of validity: Predictive validity is the correlation between the Reading Composite Score at the beginning of the year and the GRADE and CST scores at the end of the school year. This coefficient represents the extent to which the Reading Composite Score can predict later reading outcomes. Concurrent validity is the correlation between the Reading Composite Score and the GRADE and CST measures both at the end of the year. This coefficient represents the extent to which the Reading Composite Score is related to important reading outcomes.

 

4.Validity for the performance level score (e.g., concurrent, predictive, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Predictive

K

GRADE

166

0.48

0.35

0.59

Concurrent

K

GRADE

166

0.40

0.26

0.52

Predictive

1

GRADE

196

0.71

0.66

0.77

Concurrent

1

GRADE

196

0.77

0.71

0.82

Predictive

2

CST

1192

0.75

0.73

0.78

Concurrent

2

CST

1192

0.77

0.75

0.79

Predictive

3

CST

1232

0.73

0.70

0.76

Concurrent

3

CST

1232

0.76

 

0.73

0.78

Predictive

4

CST

1183

0.75

0.72

0.77

Concurrent

4

CST

1183

0.76

0.73

0.78

Predictive

5

CST

1218

0.73

0.70

0.76

Concurrent

5

CST

1218

0.72

0.69

0.75

Predictive

6

CST

616

0.73

0.69

0.77

Concurrent

6

CST

616

0.73

0.69

0.76

 

5.Results for other forms of validity (e.g. factor analysis) not conducive to the table format: Not Provided

 

6.Describe the degree to which the provided data support the validity of the tool: Both the concurrent and predictive correlation are high, especially after kindergarten. These strong correlations suggest that the Reading Composite Score is assessing skills relevant to reading outcomes.

Given the wide range of skills assessed on the GRADE and CST, these data support the conclusion that the Reading Composite Score is an excellent indicator of reading proficiency, including reading for meaning, at an adequate rate, with a high degree of accuracy.

 

 

Disaggregated Validity

The following disaggregated validity data are provided for context and did not factor into the Validity rating.

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

Results for other forms of disaggregated validity (e.g. factor analysis) not conducive to the table format: Not Provided  

If your manual cites other published validity studies, provide these citations: Dewey, E. N., Powell-Smith, K. A., Good, R. H., & Kaminski, R. A. (2015). Acadience Reading Technical Adequacy Brief. Eugene: Dynamic Measurement Group.

Sample Representativeness

GradeK123456
Data
  • Regional without Cross-Validation
  • Regional without Cross-Validation
  • Regional without Cross-Validation
  • Regional with Cross-Validation
  • Regional with Cross-Validation
  • Regional with Cross-Validation
  • Regional without Cross-Validation
  • Primary Classification Accuracy Sample

    Criterion 1, Spring

    Grade

    K

    1

    2

    3

    4

    5

    6

    Criterion

    GRADE

    GRADE

    GRADE

    GRADE

    GRADE

    GRADE

    GRADE

    National/Local Representation

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Date

    2011

    2011

    2011

    2011

    2011

    2011

    2011

    Sample Size

    166

    193

    217

    219

    186

    195

    103

    Male

    0

    0

    0

    0

    0

    0

    0

    Female

    0

    0

    0

    0

    0

    0

    0

    Gender Unknown

    166

    193

    217

    219

    186

    195

    103

    Free or Reduced-price Lunch Eligible

    N/A

    N/A

    N/A

    N/A

    N/A

    N/A

    N/A

    White, Non-Hispanic

    0

    0

    0

    0

    0

    0

    0

    Black, Non-Hispanic

    0

    0

    0

    0

    0

    0

    0

    Hispanic

    0

    0

    0

    0

    0

    0

    0

    American Indian/Alaska Native

    0

    0

    0

    0

    0

    0

    0

    Other

    0

    0

    0

    0

    0

    0

    0

    Race/Ethnicity Unknown

    166

    193

    217

    219

    186

    195

    103

    Disability Classification

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

     

    Criterion 2

    Grade

    K

    1

    2

    3

    4

    5

    6

    Criterion

    Not Provided

    Not Provided

    CST Reading

    CST Reading

    CST Reading

    CST Reading

    CST Reading

    National/Local Representation

    Not Provided

    Not Provided

    Pacific

    Pacific

    Pacific

    Pacific

    Pacific

    Date

    Not Provided

    Not Provided

    2011

    2011

    2011

    2011

    2011

    Sample Size

    Not Provided

    Not Provided

    1192

    1227

    1175

    1204

    593

    Male

    Not Provided

    Not Provided

    601

    612

    578

    571

    308

    Female

    Not Provided

    Not Provided

    591

    602

    582

    618

    281

    Gender Unknown

    Not Provided

    Not Provided

    0

    13

    15

    15

    4

    Free or Reduced-price Lunch Eligible

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    White, Non-Hispanic

    Not Provided

    Not Provided

    1

    1

    2

    0

    0

    Black, Non-Hispanic

    Not Provided

    Not Provided

    0

    0

    0

    1

    3

    Hispanic

    Not Provided

    Not Provided

    50

    48

    68

    41

    59

    American Indian/Alaska Native

    Not Provided

    Not Provided

    0

    0

    0

    0

    0

    Other

    Not Provided

    Not Provided

    2

    3

    2

    0

    4

    Race/Ethnicity Unknown

    Not Provided

    Not Provided

    1139

    1173

    1103

    1162

    530

    Disability Classification

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

    303 ELL

    277 ELL

    332 ELL

    209 ELL

    90 ELL

     

    Cross Validation Sample

    Spring

    Grade

    K

    1

    2

    3

    4

    5

    6

    Criterion

    Not Provided

    Not Provided

    Not Provided

    SBAC

    ELA

    SBAC

    ELA

    SBAC

    ELA

    Not Provided

    National/Local Representation

    Not Provided

    Not Provided

    Not Provided

    Pacific

    Pacific

    Pacific

    Not Provided

    Date

    Not Provided

    Not Provided

    Not Provided

    2014-2016

    2014-2016

    2014-2016

    Not Provided

    Sample Size

    Not Provided

    Not Provided

    Not Provided

    121

    118

    117

    Not Provided

    Male

    Not Provided

    Not Provided

    Not Provided

    59

    64

    63

    Not Provided

    Female

    Not Provided

    Not Provided

    Not Provided

    62

    54

    54

    Not Provided

    Gender Unknown

    Not Provided

    Not Provided

    Not Provided

    0

    0

    0

    Not Provided

    Free or Reduced-price Lunch Eligible

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    White, Non-Hispanic

    Not Provided

    Not Provided

    Not Provided

    3

    2

    8

    Not Provided

    Black, Non-Hispanic

    Not Provided

    Not Provided

    Not Provided

    0

    0

    0

    Not Provided

    Hispanic

    Not Provided

    Not Provided

    Not Provided

    20

    15

    10

    Not Provided

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Not Provided

    0

    0

    0

    Not Provided

    Other

    Not Provided

    Not Provided

    Not Provided

    44

    40

    58

    Not Provided

    Race/Ethnicity Unknown

    Not Provided

    Not Provided

    Not Provided

    27

    40

    33

    Not Provided

    Disability Classification

    Not Provided

    Not Provided

    Not Provided

    24

    17

    7

    Not Provided

    First Language

    Not Provided

    Not Provided

    Not Provided

    3

    4

    1

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

     

    Bias Analysis Conducted

    GradeK123456
    RatingYesYesYesYesYesYesYes
    1. Description of the method used to determine the presence or absence of bias: Bias was conceptualized as different classification accuracy between different groups. This was assessed using a Cleary model with the dichotomous outcome of status on the criterion, where the Reading Composite Score, subgroup, and the interaction between the two were used as predictors. If a model with the subgroup and interaction term do not add significantly to model fit, there was evidence that the Reading Composite Score is not biased. Model fit was assessed using the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and the likelihood ratio test (LRT). The effect size for bias was assessed using the difference in AUC for the ROC curves for the different groups. These models were tested for each grade, at each time of year.

     

    1. Description of the subgroups for which bias analyses were conducted: Bias was assessed across genders and among white and non-white students.

     

    1. Description of the results of the bias analyses conducted, including data and interpretative statements: Of the 9 models examining bias across ethnicities the AIC and BIC favored a model without bias all nine times, and the likelihood ratio test showed that adding ethnic group to the logistic regression did not significantly improve model fit. Of the 33 models examining bias across genders, the AIC favored a model without bias 29 times while the BIC favored a model without bias all 33 times. Likewise, the likelihood ratio test favored a model with bias only four out of the 33 times. The results show that the rate of preferring model with bias is near the global Type I error rate of .05, suggesting a lack of bias on the Reading Composite Score.

    Administration Format

    GradeK123456
    Data
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Administration & Scoring Time

    GradeK123456
    Data
  • 4-10 minutes
  • 4-10 minutes
  • 4-10 minutes
  • 4-10 minutes
  • 4-10 minutes
  • 4-10 minutes
  • 4-10 minutes
  • Scoring Format

    GradeK123456
    Data
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Types of Decision Rules

    GradeK123456
    Data
  • Benchmark Goals
  • Benchmark Goals
  • Benchmark Goals
  • Benchmark Goals
  • Benchmark Goals
  • Benchmark Goals
  • Benchmarks Goals
  • Evidence Available for Multiple Decision Rules

    GradeK123456
    Data
  • Yes
  • Yes
  • Yes
  • Yes
  • Yes
  • Yes
  • Yes