Acadience Reading (aka DIBELS Next)

Maze

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

Free Version: No cost

Print Version: $3.64 – $3.72 per student

Mobile Version: $14.90 per student

 

Replacement Cost:

Free Version: No cost

Print Version: $3.62 – $3.74 per student per year

Mobile Version: $14.90 per student per year
Annual license renewal fees subject to change.

 

Included in Cost:

Users adopting the free version can download all materials and supporting documents from https://acadiencelearning.org/acadiencereading.html - however, users may incur some costs when copying/printing materials. Data management and reporting for Acadience Reading paper/pencil users are available from Acadience Data Management, a service provided by Dynamic Measurement Group. Acadience Data Management is optional and costs $1.00 per student per school year.

 

Users purchasing the mobile version will receive access to all materials as well as an online administration platform and data reporting system.

 

Users purchasing the print version will receive a classroom set, which includes all necessary materials for assessing 25 students.

 

Technology Requirements:

  • No technology is required*

*Users who choose Amplify’s mobile device version will need internet access and a tablet or computer.

 

Training Requirements:

  • 1-4 hours of training

 

Qualified Administrators:

  • No minimum qualifications specified*

*No specific expertise is required, but training in the specific administration and scoring procedures should be provided.

 

Accommodations:

Acadience Reading is appropriate for most students for whom an instructional goal is to learn to read in English. For English language learners who are learning to read in English, Acadience Reading is appropriate for assessing and monitoring progress in acquisition of early reading skills. For all Acadience Reading measures, students are never penalized for articulation or dialect differences that are part of their typical speech. In addition, Acadience Reading measures include discontinue rules to prevent student frustration, as well as a set of approved accommodations that assessors may use when appropriate (see Acadience Reading Assessment Manual).

 

There are a few groups of students for whom Acadience Reading is not appropriate: (a) students who are learning to read in a language other than English; (b) students who are deaf; (c) students who have fluency-based speech disabilities such as stuttering (if the stuttering affects the student's response fluency within a one-minute timed assessment) and oral apraxia; and (d) students with severe disabilities for whom learning to read connected text is not an IEP goal.

 

Assessment accommodations are used for those students for whom the standard administration conditions would not produce accurate results. Approved accommodations are those accommodations which are unlikely to change how the assessment functions. When approved accommodations are used, the scores can be reported and interpreted as official Acadience Reading scores (see Acadience Reading Assessment Manual for a list of approved accommodations). Approved accommodations should be used only for students for whom the accommodations are necessary to provide an accurate assessment of student skills.

 

Unapproved accommodations are accommodations that are likely to change how the assessment functions (such as modifying the timing rules). Scores from measures administered with unapproved accommodations should not be treated or reported as official Acadience Reading scores and cannot be compared to other Acadience Reading scores or benchmark goals but can be used to measure individual growth for a student. An unapproved accommodation may be used when (a) a student cannot be tested accurately using the standardized rules or approved accommodations, but the school would still like to measure progress for that student; or (b) a student’s Individualized Education Plan (IEP) requires testing with an unapproved accommodation.

Where to Obtain:

Free Version:

Website: https://acadiencelearning.org/acadiencereading.html

Address: Dynamic Measurement Group, 859 Willamette Street, Suite 320, Eugene, OR 97410

Phone number: 541-431-6931 or toll free 888-943-1240

Email: info@acadiencelearning.org

 

Print Version (published under the name DIBELS Next®):

Website: http://voyagersopris.com

Address: Voyager Sopris Learning, 17855 Dallas Parkway, Suite 400, Dallas, TX 75287-6816  

Telephone: (800) 547-6747

 

Mobile Version (published under the name mCLASS: DIBELS Next®):

Website: www.amplify.com

Address: Amplify, 55 Washington Street, Suite 800, Brooklyn, NY 11201

Telephone: (800) 823-1969


Access to Technical Support:

Dynamic Measurement Group provides customer support for all Acadience Reading assessments, as well as support for the data management and reporting system, Acadience Data Management. Staff are available by phone and email on weekdays from 7am to 5pm Pacific Time at no additional cost. The majority of customer support requests are resolved in less than one business day.

Acadience Reading Maze (also published as DIBELS Next®)1 is a measure of reading comprehension. Maze assesses a student’s ability to construct meaning from text using word recognition skills, background information and prior knowledge, familiarity with linguistic properties such as syntax and morphology, and cause and effect reasoning skills. Maze is just one measure that is part of a broader reading assessment known as Acadience Reading.

 

1 AcadienceTM is a trademark of Dynamic Measurement Group, Inc. (DMG). DMG sold the DIBELS® and DIBELS Next® trademarks to the University of Oregon; DMG's continued use of the DIBELS Next mark is by license from UO. The DIBELS Next copyrighted content and copyright continues to be owned by DMG.

Assessment Format:

  • Direct observation
  • Performance measure
  • One-to-one

 

Administration Time:

  • 5 minutes per student
  • 5 minutes per group

 

Scoring Time:

  • 1-2 minutes per student*

*Scoring is automatic for users who purchase Amplify’s mobile device version or who purchase the Acadience Data Management system.

 

Scoring Method:

  • Calculated manually*

*Scoring is automatic for users who purchase Amplify’s mobile device version or who purchase the Acadience Data Management system.

 

Scores Generated:

  • Raw score
  • Percentile score
  • Developmental benchmarks

 

Classification Accuracy

Grade3456
Criterion 1 FallHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubble
Criterion 1 WinterHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleEmpty bubble
Criterion 1 SpringHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubble
Criterion 2 FallHalf-filled bubbleFull bubbleFull bubbleHalf-filled bubble
Criterion 2 WinterHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleFull bubble
Criterion 2 SpringHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubble

Primary Sample

 

Criterion 1, Fall

Grade

3

4

5

6

Criterion

GRADE

GRADE

GRADE

GRADE

Cut points: Percentile rank on criterion measure

20th Percentile

20th Percentile

20th Percentile

20th Percentile

Cut points: Performance score (numeric) on criterion measure

78

41

39

53

Cut points: Corresponding performance score (numeric) on screener measure

5

10

12

14

Base rate in the sample for children requiring intensive intervention

0.20

0.20

0.20

0.20

False Positive Rate

0.06

0.05

0.07

0.03

False Negative Rate

0.49

0.46

0.58

0.57

Sensitivity

0.51

0.54

0.43

0.43

Specificity

0.95

0.95

0.94

0.98

Positive Predictive Power

0.71

0.70

0.63

0.82

Negative Predictive Power

0.88

0.90

0.86

0.87

Overall Classification Rate

0.85

0.87

0.83

0.86

Area Under the Curve (AUC)

0.86

0.85

0.84

0.84

AUC 95% Confidence Interval Lower Bound

0.82

0.77

0.78

0.75

AUC 95% Confidence Interval Upper Bound

0.90

0.94

0.90

0.94

 

Criterion 1, Winter

Grade

3

4

5

6

Criterion

GRADE

GRADE

GRADE

GRADE

Cut points: Percentile rank on criterion measure

20th Percentile

20th Percentile

20th Percentile

20th Percentile

Cut points: Performance score (numeric) on criterion measure

78

41

39

53

Cut points: Corresponding performance score (numeric) on screener measure

7

12

13

14

Base rate in the sample for children requiring intensive intervention

0.20

0.20

0.20

0.20

False Positive Rate

0.06

0.03

0.03

0.03

False Negative Rate

0.43

0.61

0.67

0.68

Sensitivity

0.58

0.39

0.33

0.32

Specificity

0.94

0.97

0.97

0.98

Positive Predictive Power

0.72

0.74

0.72

0.75

Negative Predictive Power

0.89

0.87

0.85

0.86

Overall Classification Rate

0.86

0.85

0.84

0.85

Area Under the Curve (AUC)

0.88

0.84

0.84

0.80

AUC 95% Confidence Interval Lower Bound

0.82

0.76

0.77

0.69

AUC 95% Confidence Interval Upper Bound

0.93

0.92

0.92

0.92

 

Criterion 1, Spring

Grade

3

4

5

6

Criterion

GRADE

GRADE

GRADE

GRADE

Cut points: Percentile rank on criterion measure

20th Percentile

20th Percentile

20th Percentile

20th Percentile

Cut points: Performance score (numeric) on criterion measure

78

41

39

53

Cut points: Corresponding performance score (numeric) on screener measure

14

20

18

15

Base rate in the sample for children requiring intensive intervention

0.20

0.20

0.20

0.20

False Positive Rate

0.05

0.03

0.02

0.05

False Negative Rate

0.50

0.38

0.73

0.62

Sensitivity

0.50

0.62

0.28

0.38

Specificity

0.95

0.97

0.98

0.95

Positive Predictive Power

0.71

0.82

0.79

0.67

Negative Predictive Power

0.87

0.86

0.84

0.86

Overall Classification Rate

0.85

0.90

0.84

0.84

Area Under the Curve (AUC)

0.84

0.87

0.80

0.84

AUC 95% Confidence Interval Lower Bound

0.76

0.80

0.73

0.75

AUC 95% Confidence Interval Upper Bound

0.92

0.94

0.88

0.92

 

Criterion 2, Fall

Grade

3

4

5

6

Criterion

CST Reading

CST Reading

CST Reading

CST Reading

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

≤ 315

≤ 350

≤ 350

≤ 350

Cut points: Corresponding performance score (numeric) on screener measure

5

10

12

14

Base rate in the sample for children requiring intensive intervention

0.20

0.25

0.27

0.18

False Positive Rate

0.04

0.15

0.16

0.17

False Negative Rate

0.69

0.30

0.29

0.35

Sensitivity

0.31

0.70

0.71

0.65

Specificity

0.96

0.85

0.84

0.84

Positive Predictive Power

0.65

0.52

0.53

0.32

Negative Predictive Power

0.86

0.92

0.92

0.95

Overall Classification Rate

0.84

0.82

0.81

0.81

Area Under the Curve (AUC)

0.84

0.86

0.87

0.83

AUC 95% Confidence Interval Lower Bound

0.81

0.83

0.85

0.79

AUC 95% Confidence Interval Upper Bound

0.84

0.88

0.89

0.86

 

Criterion 2, Winter

Grade

3

4

5

6

Criterion

CST Reading

CST Reading

CST Reading

CST Reading

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

≤ 315

≤ 350

≤ 350

≤ 350

Cut points: Corresponding performance score (numeric) on screener measure

7

12

13

14

Base rate in the sample for children requiring intensive intervention

0.20

0.25

0.27

0.18

False Positive Rate

0.03

0.19

0.21

0.14

False Negative Rate

0.74

0.36

0.36

0.30

Sensitivity

0.27

0.64

0.64

0.70

Specificity

0.97

0.81

0.79

0.86

Positive Predictive Power

0.67

0.36

0.35

0.47

Negative Predictive Power

0.85

0.93

0.93

0.94

Overall Classification Rate

0.83

0.78

0.77

0.84

Area Under the Curve (AUC)

0.82

0.82

0.83

0.86

AUC 95% Confidence Interval Lower Bound

0.78

0.79

0.80

0.83

AUC 95% Confidence Interval Upper Bound

0.85

0.84

0.85

0.90

 

Criterion 2, Spring

Grade

3

4

5

6

Criterion

CST Reading

CST Reading

CST Reading

CST Reading

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

≤ 315

≤ 350

≤ 350

≤ 350

Cut points: Corresponding performance score (numeric) on screener measure

14

20

18

15

Base rate in the sample for children requiring intensive intervention

0.20

0.25

0.27

0.18

False Positive Rate

0.03

0.18

0.33

0.16

False Negative Rate

0.65

0.31

0.41

0.39

Sensitivity

0.35

0.69

0.59

0.61

Specificity

0.97

0.82

0.77

0.84

Positive Predictive Power

0.75

0.41

0.25

0.36

Negative Predictive Power

0.86

0.94

0.94

0.94

Overall Classification Rate

0.85

0.80

0.75

0.81

Area Under the Curve (AUC)

0.85

0.84

0.81

0.84

AUC 95% Confidence Interval Lower Bound

0.82

0.82

0.79

0.81

AUC 95% Confidence Interval Upper Bound

0.87

0.87

0.84

0.88

 

Additional Classification Accuracy

The following are provided for context and did not factor into the Classification Accuracy ratings.

 

Cross-Validation Sample

Fall

Grade

3

4

5

6

Criterion

SBAC

ELA

SBAC

ELA

SBAC

ELA

Not Provided

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

< 2432

< 2473

< 2502

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

5

10

12

Not Provided

Base rate in the sample for children requiring intensive intervention

0.22

0.38

0.34

Not Provided

False Positive Rate

0.21

0.21

0.22

Not Provided

False Negative Rate

0.50

0.31

0.36

Not Provided

Sensitivity

0.50

0.69

0.65

Not Provided

Specificity

0.79

0.79

0.78

Not Provided

Positive Predictive Power

0.15

0.69

0.50

Not Provided

Negative Predictive Power

0.96

0.79

0.86

Not Provided

Overall Classification Rate

0.77

0.76

0.74

Not Provided

Area Under the Curve (AUC)

0.72

0.80

0.81

Not Provided

AUC 95% Confidence Interval Lower Bound

0.64

0.72

0.73

Not Provided

AUC 95% Confidence Interval Upper Bound

0.80

0.89

0.89

Not Provided

 

Winter

Grade

3

4

5

6

Criterion

SBAC

ELA

SBAC

ELA

SBAC

ELA

Not Provided

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

< 2432

< 2473

< 2502

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

7

12

13

Not Provided

Base rate in the sample for children requiring intensive intervention

0.22

0.38

0.34

Not Provided

False Positive Rate

0.20

0.21

0.17

Not Provided

False Negative Rate

0.11

0.23

0.33

Not Provided

Sensitivity

0.89

0.77

0.67

Not Provided

Specificity

0.80

0.79

0.83

Not Provided

Positive Predictive Power

0.19

0.60

0.67

Not Provided

Negative Predictive Power

0.99

0.90

0.83

Not Provided

Overall Classification Rate

0.81

0.79

0.78

Not Provided

Area Under the Curve (AUC)

0.77

0.79

0.87

Not Provided

AUC 95% Confidence Interval Lower Bound

0.68

0.69

0.80

Not Provided

AUC 95% Confidence Interval Upper Bound

0.86

0.89

0.94

Not Provided

 

Spring

Grade

3

4

5

6

Criterion

SBAC ELA

SBAC ELA

SBAC ELA

Not Provided

Cut points: Percentile rank on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Performance score (numeric) on criterion measure

< 2432

< 2473

< 2502

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

14

20

18

Not Provided

Base rate in the sample for children requiring intensive intervention

0.22

0.38

0.34

Not Provided

False Positive Rate

0.20

0.20

0.18

Not Provided

False Negative Rate

0.33

0.21

0.33

Not Provided

Sensitivity

0.67

0.79

0.67

Not Provided

Specificity

0.80

0.80

0.72

Not Provided

Positive Predictive Power

0.15

0.64

0.29

Not Provided

Negative Predictive Power

0.98

0.90

0.93

Not Provided

Overall Classification Rate

0.79

0.80

0.71

Not Provided

Area Under the Curve (AUC)

0.74

0.85

0.74

Not Provided

AUC 95% Confidence Interval Lower Bound

0.66

0.77

0.64

Not Provided

AUC 95% Confidence Interval Upper Bound

0.83

0.92

0.83

Not Provided

 

Reliability

Grade3456
RatingFull bubbleFull bubbleFull bubbleFull bubble

1.Justification for each type of reliability reported, given the type and purpose of the tool: Reliability refers to the relative stability with which a test measures the same skills across minor differences in conditions. Two types of reliability are reported in the table below, alternate form reliability and alpha. Alternate form reliability is the correlation between different measures of the same early literacy skills. The coefficient reported is the average correlation among three forms of the measure. Coefficient alpha is a measure of reliability that is widely used in education research and represents the proportion of true score to total variance. Alpha incorporates information about the average inter-test correlation as well as the number of tests.

 

2.Description of the sample(s), including size and characteristics, for each reliability analysis conducted: The data used for assessing reliability came from third through sixth grade. The total sample size is 674 students from 13 schools within 5 school districts. The sample was drawn from two census regions (Pacific and North Central Midwest).

 

3.Description of the analysis procedures for each reported type of reliability: Alternate form reliability is reported as the average correlation among three alternate forms of the same test. Coefficient alpha treats each of the three tests as separate indicators and is calculated using the alternate form reliability, where the number of tests is equal to three

 

4.Reliability of performance level score (e.g., model-based, internal consistency, inter-rater reliability).

Type of Reliability

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Alternate Form

3

187

0.83

0.78

0.87

Alternate Form

4

1138

0.90

0.88

0.91

Alternate Form

5

195

0.83

0.78

0.87

Alternate Form

6

105

0.79

0.71

0.85

Alpha

3

187

0.94

0.92

0.95

Alpha

4

187

0.90

0.87

0.92

Alpha

5

195

0.94

0.92

0.95

Alpha

6

105

0.92

0.89

0.94

 

Disaggregated Reliability

The following disaggregated reliability data are provided for context and did not factor into the Reliability rating.

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

If your manual cites other published studies on reliability, provide these citations: Dewey, E. N., Powell-Smith, K. A., Good, R. H., & Kaminski, R. A. (2015). Acadience Reading Technical Adequacy Brief. Eugene: Dynamic Measurement Group

Validity

Grade3456
RatingFull bubbleFull bubbleHalf-filled bubbleFull bubble

1.Description of each criterion measure used and explanation as to why each measure is appropriate, given the type and purpose of the tool: The California Standards Test (CST) is a statewide achievement test produced for California public schools and was designed to assess the California content standards for English/language arts (ELA), mathematics, history–social science, and science in grades two through eleven. The Reading cluster of the ELA portion of the CST was chosen as the criterion. According to a technical report from ETS (2011), the CST items were developed and designed to conform to principles of item writing defined by ETS (ETS, 2002). In addition, the items selected underwent an extensive item review process designed to provide the best standards-based tests possible.

The AzMERIT is a statewide achievement test produced for Arizona. Arizona partnered with the American Institutes for Research (AIR) to develop this Arizona-specific assessment linked to, and designed to assess, the Arizona College and Career Ready Standards (ACCRS) for English Language Arts (ELA) and Math. According to the test developer, scores from the AzMERIT can be used to evaluate whether students have achieved the ACCRS by the end of the school year, as well as the effectiveness with which Arizona districts and schools teach students the ACCRS (AIR, 2017).

A rigorous test development process was used to ensure item alignment with the ACCRS. Test blueprints specified the range and depth with which each standard was assessed. Items underwent extensive review by content experts, an internal review with AIR experts, and reviewed by parents/community members. Field-testing was extensive along with research examining the relation of the test to other measures (e.g., NAEP, SBAC, ACT).

The AzMERIT English Language Arts (ELA) score is derived from performance in three areas: Reading for Information, Reading for Literature, and Writing and Language. There are four performance levels for the AzMERIT ELA score: 1-Minimally Proficient,  2-Partially Proficient, 3-Proficient, and 4-Highly Proficient. Scores in the level 3 (Proficient) and 4 (Highly Proficient) range meet grade-level achievement standards. The minimal score to be at level 3 on the ELA portion of the AzMERIT is 2509 for Grade 3 and 2523 for Grade 4.

 

2.Description of the sample(s), including size and characteristics, for each validity analysis conducted: The data used for assessing validity came from third through sixth grade. The total sample size is 4,249 students from 14 schools. The sample was drawn from the Pacific census region.

 

3.Description of the analysis procedures for each reported type of validity: Predictive validity is the correlation between the Acadience Reading Maze Adjusted Score at the beginning of the year and the CST and AzMERIT scores at the end of the school year. This coefficient represents the extent to which Acadience Reading Maze can predict later reading outcomes. Concurrent validity is the correlation between the Acadience Reading Maze Adjusted Score and the CST and AzMERIT measures both at the end of the year. This coefficient represents the extent to which Acadience Reading Maze is related to important reading outcomes.

 

4.Validity for the performance level score (e.g., concurrent, predictive, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Predictive

3

AzMERIT

568

0.65

0.60

0.69

Predictive

4

AzMERIT

639

0.70

0.66

0.74

Predictive

5

CST

1218

0.67

0.64

0.70

Predictive

6

CST

616

0.65

0.60

0.70

Concurrent

3

AzMERIT

568

0.68

0.64

0.72

Concurrent

4

AzMERIT

639

0.77

0.73

0.80

Concurrent

5

CST

1218

0.56

0.52

0.60

Concurrent

6

CST

616

0.65

0.60

0.69

 

5.Results for other forms of validity (e.g. factor analysis) not conducive to the table format: Not Provided

 

6.Describe the degree to which the provided data support the validity of the tool: Both the concurrent and predictive correlation are high. These strong correlations suggest that Acadience Reading Maze assesses skills relevant to broad reading outcomes.

 

Disaggregated Validity

The following disaggregated validity data are provided for context and did not factor into the Validity rating.

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

Results for other forms of disaggregated validity (e.g. factor analysis) not conducive to the table format: Not Provided  

If your manual cites other published validity studies, provide these citations: Dewey, E. N., Powell-Smith, K. A., Good, R. H., & Kaminski, R. A. (2015). Acadience Reading Technical Adequacy Brief. Eugene: Dynamic Measurement Group.

Sample Representativeness

Grade3456
Data
  • Regional with Cross-Validation
  • Regional with Cross-Validation
  • Regional with Cross-Validation
  • Regional without Cross-Validation
  • Primary Classification Accuracy Sample

    Criterion 1, Spring

    Grade

    3

    4

    5

    6

    Criterion

    GRADE

    GRADE

    GRADE

    GRADE

    National/Local Representation

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Pacific, East North Central

    Date

    2011

    2011

    2011

    2011

    Sample Size

    219

    186

    195

    103

    Male

    0

    0

    0

    0

    Female

    0

    0

    0

    0

    Gender Unknown

    219

    186

    195

    103

    Free or Reduced-price Lunch Eligible

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    White, Non-Hispanic

    0

    0

    0

    0

    Black, Non-Hispanic

    0

    0

    0

    0

    Hispanic

    0

    0

    0

    0

    American Indian/Alaska Native

    0

    0

    0

    0

    Other

    0

    0

    0

    0

    Race/Ethnicity Unknown

    219

    186

    195

    103

    Disability Classification

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

    Not Provided

    Not Provided

     

    Criterion 2, Spring

    Grade

    3

    4

    5

    6

    Criterion

    CST Reading

    CST Reading

    CST Reading

    CST Reading

    National/Local Representation

    Pacific

    Pacific

    Pacific

    Pacific

    Date

    2011

    2011

    2011

    2011

    Sample Size

    1227

    1175

    1204

    593

    Male

    612

    578

    571

    308

    Female

    602

    582

    618

    281

    Gender Unknown

    13

    15

    15

    4

    Free or Reduced-price Lunch Eligible

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    White, Non-Hispanic

    1

    2

    0

    0

    Black, Non-Hispanic

    0

    0

    1

     

    Hispanic

    48

    68

    41

    59

    American Indian/Alaska Native

    0

    0

    0

     

    Other

    3

    2

    0

    4

    Race/Ethnicity Unknown

    1173

    1103

    1162

    530

    Disability Classification

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    277 ELL

    332 ELL

    209 ELL

    90 ELL

     

    Cross Validation Sample

    Spring

    Grade

    3

    4

    5

    6

    Criterion

    SBAC

    ELA

    SBAC

    ELA

    SBAC

    ELA

    Not Provided

    National/Local Representation

    Pacific

    Pacific

    Pacific

    Not Provided

    Date

    2014-2016

    2014-2016

    2014-2016

    Not Provided

    Sample Size

    121

    118

    117

    Not Provided

    Male

    59

    64

    63

    Not Provided

    Female

    62

    54

    54

    Not Provided

    Gender Unknown

    0

    0

    0

    Not Provided

    Free or Reduced-price Lunch Eligible

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    White, Non-Hispanic

    3

    2

    8

    Not Provided

    Black, Non-Hispanic

    0

    0

    0

    Not Provided

    Hispanic

    20

    15

    10

    Not Provided

    American Indian/Alaska Native

    0

    0

    0

    Not Provided

    Other

    44

    40

    58

    Not Provided

    Race/Ethnicity Unknown

    27

    40

    33

    Not Provided

    Disability Classification

    24

    17

    7

    Not Provided

    First Language

    3

    4

    1

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

    Not Provided

    Not Provided

     

    Bias Analysis Conducted

    Grade3456
    RatingYesYesYesYes
    1. Description of the method used to determine the presence or absence of bias: Bias was conceptualized as different classification accuracy between different groups. This was assessed using a Cleary model with the dichotomous outcome of status on the criterion, where the Maze Adjusted Score, subgroup, and the interaction between the two were used as predictors. If a model with the subgroup and interaction term do not add significantly to model fit, there was evidence that Maze is not biased. Model fit was assessed using the Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), and the likelihood ratio test (LRT). The effect size for bias was assessed using the difference in AUC for the ROC curves for the different groups. These models were tested for each grade, at each time of year.

     

    1. Description of the subgroups for which bias analyses were conducted: Bias was assessed across genders and among white and non-white students.

     

    1. Description of the results of the bias analyses conducted, including data and interpretative statements: Of the 9 models examining bias across ethnicities the AIC and LRT favored a model without bias eight times, while the BIC favored a model without bias all nine times. Of the 21 models examining bias across genders, the AIC favored a model without bias 17 times while the BIC favored a model without bias 20 times. Likewise, the likelihood ratio test favored a model with bias only three times out of 21 models. The results show that the rate of preferring model with bias is near the global Type I error rate of .05, suggesting a lack of bias on the Maze measure.

     

    Administration Format

    Grade3456
    Data
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Administration & Scoring Time

    Grade3456
    Data
  • 6-8 minutes
  • 6-8 minutes
  • 6-8 minutes
  • 6-8 minutes
  • Scoring Format

    Grade3456
    Data
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Types of Decision Rules

    Grade3456
    Data
  • Benchmark Goals
  • Benchmark Goals
  • Benchmark Goals
  • Benchmarks Goals
  • Evidence Available for Multiple Decision Rules

    Grade3456
    Data
  • Yes
  • Yes
  • Yes
  • Yes