Acadience Reading 7-8
Maze
Summary
Maze is a group-administered measure that assesses general reading comprehension. The student is presented with a passage in which some words are replaced by a multiple-choice box that includes the original word and two distractors. The student reads the passage silently and selects the word in each box that best fits the meaning of the sentence. Each Maze passage has a 3-minute time limit. Each Maze form is composed of a triad of one science, one social studies, and one prose passage. The Maze triad of passages takes 10–12 minutes to complete. Maze is scored with a scoring key. An adjusted score is calculated to correct for guessing.
- Where to Obtain:
- Acadience Learning Inc.
- info@acadiencelearning.org; customerservice@voyagersopris.com
- Acadience Learning Inc. 4710 Village Plaza Loop, Suite 210, Eugene OR 97401; Voyager Sopris Learning 17855 Dallas Parkway, Suite 400 Dallas, TX 75287
- Acadience Learning: (541)431-6931, (888)943-1240; Voyager Sopris Learning: (800)547-6747
- https://acadiencelearning.org/; https://www.voyagersopris.com/
- Initial Cost:
- Free
- Replacement Cost:
- Free
- Included in Cost:
- Additional info on where the tool can be obtained: Acadience Learning Inc. (Free download version in black and white) Voyager Sopris Learning (print materials in color) Website: http://acadiencelearning.org Address: Village Plaza Loop, Suite 210, Eugene, OR 97401 Phone number: 541-431-6931 or toll free 888-943-1240 Email address: info@acadiencelearning.org Download version black and white = $0 per student, per year Print materials: Voyager Sopris Learning (Published print version in color) Website: http://voyagersopris.com Address: 17855 Dallas Parkway, Suite 400, Dallas, TX 75287-6816 Telephone number: (800) 547-6747. Classroom Kits 7 and 8 = $132.00, $5.28 per student Data management via Acadience Learning Online: Manual Entry Licenses for K-6 = $2.50 per student, per year
- Acadience Reading 7–8 is appropriate for most students for whom an instructional goal is gaining the skills required to successfully read content-area text of increasing complexity in English. For English language learners who are learning to read in English, Acadience Reading 7–8 is appropriate for assessing and monitoring progress in the acquisition of those skills. Acadience Reading 7–8 is also appropriate for students in special education for whom reading content-area text is expected or for whom it is an Individualized Education Plan (IEP) goal. For students receiving special education, it may be necessary to adjust goals and timelines and use below-grade materials for progress monitoring. Acadience Reading 7–8 is not appropriate for (a) students who are learning to read in a language other than English or (b) students with severe disabilities for whom reading content-area text is not an IEP goal. Additionally, assessment with some Acadience Reading 7–8 measures (e.g., Oral Reading) may not be appropriate with (a) students who are deaf or (b) students who have fluency based speech disabilities, such as stuttering or oral apraxia. Approved accommodations are any accommodations that will not alter the standardization of the assessment. Approved accommodations include: 1) The use of student materials that have been enlarged or with larger print for students with visual impairments. 2) The use of colored overlays, filters, or lighting adjustments for students with visual impairments. 3) The use of assistive technology, such as hearing aids and assistive listening devices (ALDs), for students with hearing impairments. 4) The use of a marker or ruler to focus student attention on the materials for students who are not able to demonstrate their skills adequately without one. Unapproved accommodations are accommodations that are likely to change how the assessment functions. Scores from measures administered with unapproved accommodations should not be treated or reported as official Acadience Reading 7–8 scores, and cannot be compared to other Acadience Reading 7–8 scores or benchmarks. An unapproved accommodation may be used when (a) a student cannot be tested accurately using the standardized rules or approved accommodations but the school would still like to measure progress for that student or (b) a student’s Individualized Education Plan (IEP) requires testing with an unapproved accommodation. Scores for a student using an unapproved accommodation can be used to measure individual growth for that student.
- Training Requirements:
- Approximately 1-3 hours of training for foundations of Acadience Reading 7-8, as well as administration and scoring of Acadience Reading 7-8 Maze.
- Qualified Administrators:
- Administrator must have adequate training on the administration and scoring of Acadience Reading 7-8 Maze.
- Access to Technical Support:
- Acadience Learning provides customer support for all Acadience Reading assessments, as well as support for the data management and reporting system, Acadience Learning Online. Staff are available by phone and email on weekdays from 7am to 5pm Pacific Time, for no additional cost. The majority of customer support requests are resolved in less than one business day.
- Assessment Format:
-
- Scoring Time:
-
- Scoring is automatic OR
- 1 minutes per student form
- Scores Generated:
-
- Raw score
- Percentile score
- Developmental benchmarks
- Developmental cut points
- Administration Time:
-
- 12 minutes per group
- Scoring Method:
-
- Manually (by hand)
- Automatically (computer-scored)
- Technology Requirements:
-
- Accommodations:
- Acadience Reading 7–8 is appropriate for most students for whom an instructional goal is gaining the skills required to successfully read content-area text of increasing complexity in English. For English language learners who are learning to read in English, Acadience Reading 7–8 is appropriate for assessing and monitoring progress in the acquisition of those skills. Acadience Reading 7–8 is also appropriate for students in special education for whom reading content-area text is expected or for whom it is an Individualized Education Plan (IEP) goal. For students receiving special education, it may be necessary to adjust goals and timelines and use below-grade materials for progress monitoring. Acadience Reading 7–8 is not appropriate for (a) students who are learning to read in a language other than English or (b) students with severe disabilities for whom reading content-area text is not an IEP goal. Additionally, assessment with some Acadience Reading 7–8 measures (e.g., Oral Reading) may not be appropriate with (a) students who are deaf or (b) students who have fluency based speech disabilities, such as stuttering or oral apraxia. Approved accommodations are any accommodations that will not alter the standardization of the assessment. Approved accommodations include: 1) The use of student materials that have been enlarged or with larger print for students with visual impairments. 2) The use of colored overlays, filters, or lighting adjustments for students with visual impairments. 3) The use of assistive technology, such as hearing aids and assistive listening devices (ALDs), for students with hearing impairments. 4) The use of a marker or ruler to focus student attention on the materials for students who are not able to demonstrate their skills adequately without one. Unapproved accommodations are accommodations that are likely to change how the assessment functions. Scores from measures administered with unapproved accommodations should not be treated or reported as official Acadience Reading 7–8 scores, and cannot be compared to other Acadience Reading 7–8 scores or benchmarks. An unapproved accommodation may be used when (a) a student cannot be tested accurately using the standardized rules or approved accommodations but the school would still like to measure progress for that student or (b) a student’s Individualized Education Plan (IEP) requires testing with an unapproved accommodation. Scores for a student using an unapproved accommodation can be used to measure individual growth for that student.
Descriptive Information
- Please provide a description of your tool:
- Maze is a group-administered measure that assesses general reading comprehension. The student is presented with a passage in which some words are replaced by a multiple-choice box that includes the original word and two distractors. The student reads the passage silently and selects the word in each box that best fits the meaning of the sentence. Each Maze passage has a 3-minute time limit. Each Maze form is composed of a triad of one science, one social studies, and one prose passage. The Maze triad of passages takes 10–12 minutes to complete. Maze is scored with a scoring key. An adjusted score is calculated to correct for guessing.
ACADEMIC ONLY: What skills does the tool screen?
- Please describe specific domain, skills or subtests:
- BEHAVIOR ONLY: Which category of behaviors does your tool target?
-
- BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.
Acquisition and Cost Information
Administration
- Are norms available?
- Yes
- Are benchmarks available?
- Yes
- If yes, how many benchmarks per year?
- Three
- If yes, for which months are benchmarks available?
- Beginning of year (months 1-3 of school year), middle of year (months 4-6 of school year), and end of year (months 7-9 of school year)
- BEHAVIOR ONLY: Can students be rated concurrently by one administrator?
- If yes, how many students can be rated concurrently?
Training & Scoring
Training
- Is training for the administrator required?
- Yes
- Describe the time required for administrator training, if applicable:
- Approximately 1-3 hours of training for foundations of Acadience Reading 7-8, as well as administration and scoring of Acadience Reading 7-8 Maze.
- Please describe the minimum qualifications an administrator must possess.
- Administrator must have adequate training on the administration and scoring of Acadience Reading 7-8 Maze.
-
No minimum qualifications
- Are training manuals and materials available?
- Yes
- Are training manuals/materials field-tested?
- Yes
- Are training manuals/materials included in cost of tools?
- No
- If No, please describe training costs:
- The Acadience Reading 7-8 Assessment Manual will be available for free download as of fall of 2023 along with the test materials. In addition to administrator support provided within the Acadience Reading 7-8 Assessment Manual, Acadience Learning offers a variety of training options to meet different needs and at different price points. Training options include asynchronous training, and synchronous training via live online training, and onsite training (hiring a trainer to come out to the school or district). Acadience Learning staff can work with schools, LEAs, regional agencies, and SEAs to develop customized training plans to meet their unique needs. For an individual teacher subscription to the online Acadience Reading 7-8 Essential Workshop, the cost is $129. Please note: Other training options may cost more or less depending on the circumstances and the number of attendees.
- Can users obtain ongoing professional and technical support?
- Yes
- If Yes, please describe how users can obtain support:
- Acadience Learning provides customer support for all Acadience Reading assessments, as well as support for the data management and reporting system, Acadience Learning Online. Staff are available by phone and email on weekdays from 7am to 5pm Pacific Time, for no additional cost. The majority of customer support requests are resolved in less than one business day.
Scoring
- Do you provide basis for calculating performance level scores?
-
Yes
- Does your tool include decision rules?
-
Yes
- If yes, please describe.
- The Acadience Reading 7-8 benchmarks and cut points for risk are based on research that examined the predictive probability of a score on an Acadience Reading 7–8 measure at a particular point in time, compared to later Acadience Reading 7–8 measures and external measures of reading proficiency and achievement. Two outcome criteria were used to develop and evaluate the benchmarks and cut points for risk: (a) the Stanford Achievement Test Series, Tenth Edition– Total Reading score (SAT10; Pearson, 2003); and (b) scores from Acadience Reading 7–8 measures administered at subsequent benchmark assessment time points. The 40th percentile on the SAT10 assessment was used as an indicator that the students had adequate reading skills for their grade. When the Acadience Reading 7–8 measures were used as a criterion, benchmarks were based on the prediction of subsequent benchmark status. For instance, the middle-of-year benchmarks were based on the prediction of end-of-year benchmark status. The exception was when the Acadience Reading outcome was from the beginning of grade 9, where the 40th percentile on Maze was used as the criterion. Benchmarks and cut points for risk were determined by finding the scores on Acadience Reading 7–8 measures that corresponded to the above criteria across two data sets. The goal was to select a benchmark and cut point for risk for each measure so that students who were in the At or Above Benchmark category had a high probability of meeting later benchmarks. Students in the Below Benchmark category had about a 50-50 likelihood of reaching later benchmarks, and students who were in the Well Below Benchmark category were unlikely to meet later benchmarks. In addition, logistic regression curves for predicting whether or not students would meet later benchmarks were constructed. Logistic regression allows us to examine point estimates of the probabilities of meeting later benchmarks. Benchmarks were chosen so that a student who scored exactly at the benchmark had approximately a 60% probability or likelihood of meeting later benchmarks. The cut point for risk was chosen so that students who scored exactly at the cut point had approximately a 40% probability or likelihood of meeting later benchmarks. The final consideration for selecting benchmarks and cut points for risk concerned the marginal percentages. Ideally, a student is described as performing in a benchmark category as a result of reading skill, not artifacts of the test. To this end, benchmarks were chosen so that approximately the same proportion of students were described as At or Above Benchmark at the beginning and end of each time period. For example, if 60% of students were At or Above Benchmark at the end of year, we aimed for 60% of students to be At or Above Benchmark at the middle of year. For further details, please see the Acadience Reading 7-8 Assessment Manual.
- Can you provide evidence in support of multiple decision rules?
-
Yes
- If yes, please describe.
- Research evidence supporting the use of Acadience Reading measures for benchmark assessment three times per year (diagnostic screening) is found here in the following documents: 1) Acadience Reading 7-8 Assessment Manual 2) Acadience Reading 7-8 Benchmarks and Gate Score Worksheets.
- Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.
- Maze is a group administered measure. There are three passages on each form: one science, one social studies, and one prose passage. The assessor asks the student to read the passages silently, with a time limit of 3 minutes per passage and 10-12 minutes for all three passages. In each passage, some words are replaced by a box that contains the original word and two detractor words. Students are asked to circle the word that makes the most sense in the story. If a student completes the assessment before the allotted time (10-12 minutes) is up, the assessor does not prorate the score. The student receives 1 point for each correct word, minus half a point for each incorrect word. A response is correct if the student circled or otherwise marked the correct word. The assessor will mark a slash (/) through any incorrect responses. Incorrect responses include errors, boxes with more than one answer marked, and items left blank (if they occur before the last item the student attempted within the time limit). Items left blank because the student could not get to them before time ran out do not need to be slashed and do not count as incorrect responses. If there are erasure marks, scratched out words, or any other extraneous markings, and the student’s final response is obvious, the assessor should score the item based on that response. Assessors record both scores (correct and incorrect) for each passage on the cover sheet. On the cover sheet, “C” designates correct responses and “I” designates incorrect responses. There are two Adjusted Scores for Maze to compensate for student guessing: a Passage Adjusted Score and a Total Adjusted Score. The formula for the Passage Adjusted Score is Passage Maze Adj. Score = # of correct responses – (# of incorrect responses ÷ 2). The formula for the Total Adjusted Score is Total Maze Adj. Score = P1 Maze Adj. Score + P2 Maze Adj. Score + P3 Maze Adj. Score. The result of the formula should then be rounded to the nearest whole number. Half-points (0.5) should be rounded up. The minimum Adjusted Score is 0. Negative numbers are not recorded. Benchmark assessment with Acadience Reading 7–8 is conducted within a multiple-gating system. This system allows students to be assessed three times per year so that students who may be at risk can be identified throughout the school year. At the same time, the multiple-gating process minimizes assessment time and reduces the number of students who are assessed individually. Students’ gate scores are calculated based upon the triad total scores from each administered measure. Triad scores are based on a science, social studies, and prose passage. First, the total scores are equated to have approximately the same standard deviation as Oral Reading Total Words Correct (i.e., the triad total). Then, a gate score is computed as the average of the equated scores across measures, enabling an equally weighted average of measures. Maze is Gate 1 of the gating system, and is administered to all students. The Maze Total Adjusted Score is equated and used as the Gate 1 Score. Typically, students whose Gate 1 Scores are At or Above Benchmark are not assessed until the next scheduled benchmark. Students whose Gate 1 Scores are Below Benchmark or Well Below Benchmark enter Gate 2.
- Describe the tool’s approach to screening, samples (if applicable), and/or test format, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.
- The Acadience Reading 7-8 measures were developed to provide teachers with information they need to make decisions about instruction. The authors advocate a data-based decision-making model referred to as the Outcomes-Driven Model, because the data are used to make decisions to improve student outcomes by matching the amount and type of instructional support with the needs of the individual students. These steps of the model repeat each trimester as a student progresses through each grade. At the beginning of the semester, the first step is to identify students who may need additional support. At the end of the semester, the final step is to review outcomes, which also facilitates identifying students who need additional support for the next semester. In this manner, educators can ensure that students who are on track to become proficient readers continue to make adequate progress, and that those students who are not on track receive the support they need to become proficient readers. Step 1: Identify need for support. This process occurs during benchmark assessment, and is also referred to as universal screening. The purpose is to identify those students who may need additional instructional support to achieve benchmarks. The benchmark assessment also provides information regarding the performance of all students in the school with respect to benchmarks. All students within a school or grade are assessed with Acadience Reading 7–8 three times per year on grade-level material. The testing occurs at the beginning, middle, and end of the school year. Students who are identified as Below Benchmark at Gate 1 proceed to Gate 2 and are assessed with the SR measure. Students who are Well Below Benchmark after completing Gate 2 move into Gate 3 and are assessed with the OR measure. Each additional measure provides information to staff about the skill areas in which students are having content-area reading difficulties. Step 2: Validate need for support. The purpose of this step is to be reasonably confident that the student needs or does not need additional instructional support. Before making individual student decisions, it is important to consider additional information beyond the initial data obtained during benchmark testing. Teachers can always use additional assessment information and knowledge about a student to validate a score before making decisions about instructional support. If there is a discrepancy in the student’s performance relative to other information available about the student, or if there is a question about the accuracy of a score, the score can be validated by retesting the student using alternate forms of the Acadience Reading 7-8 measures or additional diagnostic assessments as necessary. Step 3: Plan and implement support. In general, for students who are meeting the benchmarks, a good, research-based core classroom curriculum should meet their instructional needs, and they will continue to receive benchmark assessment three times per year to ensure they remain on track. For students who move from Gate 1 to Gate 2 and/or Gate 3, each additional measure provides information to staff about the skill areas in which students are having content-area reading difficulties. Step 4: Evaluate and modify support as needed. Students who are receiving additional support should be progress monitored more frequently to ensure that the instructional support being provided is helping them get back on track. Students should be monitored on the measures that test the skill areas where they are having difficulties and receiving additional instructional support. Monitoring may occur once per month, once every two weeks, or as often as once per week. In general, students who need the most intensive instruction are progress monitored most frequently. Step 5: Review outcomes. By looking at the benchmark assessment data for all students, schools can ensure that their instructional supports—both core curriculum and additional interventions—are working for all students. If a school identifies areas of instructional support that are not working as desired, the school can use the data to help make decisions on how to improve. The use of Acadience Reading measures within the Outcomes-Driven Model is consistent with the most recent reauthorization of the Individuals with Disabilities Education Improvement Act (IDEA), which allows the use of a Response to Intervention (RtI) approach to identify children with learning disabilities. In an RtI approach to identification, early intervention is provided to students who are at risk for the development of learning difficulties. Data are gathered to determine which students are responsive to the intervention provided and which students need more intensive support (Fuchs & Fuchs, 2006). The Outcomes-Driven Model is based on foundational work with a problem-solving model (see Deno, 1989; Shinn, 1995; Tilly, 2008) and the initial application of the problem-solving model to early literacy skills (Kaminski & Good, 1998). The general questions addressed by a problem-solving model include: What is the problem? Why is it happening? What should be done about it? Did it work? (Tilly, 2008). The Outcomes-Driven Model was developed to address these questions, but within a prevention-oriented framework designed to preempt early reading difficulty and ensure step-by-step progress toward outcomes that will result in established, adequate reading achievement.
Technical Standards
Classification Accuracy & Cross-Validation Summary
| Grade |
Grade 7
|
Grade 8
|
|---|---|---|
| Classification Accuracy Fall |
|
|
| Classification Accuracy Winter |
|
|
| Classification Accuracy Spring |
|
|
Legend
Convincing evidencePartially convincing evidence
Unconvincing evidence
Data unavailabledDisaggregated data available
SAT 10 Total Reading
Classification Accuracy
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
-
No
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Cross-Validation
- Has a cross-validation study been conducted?
-
No
- If yes,
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Classification Accuracy - Fall
| Evidence | Grade 7 | Grade 8 |
|---|---|---|
| Criterion measure | SAT 10 Total Reading | SAT 10 Total Reading |
| Cut Points - Percentile rank on criterion measure | 20 | 20 |
| Cut Points - Performance score on criterion measure | 37 | 46 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 16 | 39 |
| Classification Data - True Positive (a) | 10 | 11 |
| Classification Data - False Positive (b) | 11 | 12 |
| Classification Data - False Negative (c) | 1 | 3 |
| Classification Data - True Negative (d) | 43 | 44 |
| Area Under the Curve (AUC) | 0.93 | 0.87 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.84 | 0.79 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 1.00 | 0.96 |
| Statistics | Grade 7 | Grade 8 |
|---|---|---|
| Base Rate | 0.17 | 0.20 |
| Overall Classification Rate | 0.82 | 0.79 |
| Sensitivity | 0.91 | 0.79 |
| Specificity | 0.80 | 0.79 |
| False Positive Rate | 0.20 | 0.21 |
| False Negative Rate | 0.09 | 0.21 |
| Positive Predictive Power | 0.48 | 0.48 |
| Negative Predictive Power | 0.98 | 0.94 |
| Sample | Grade 7 | Grade 8 |
|---|---|---|
| Date | 2017 | 2017 |
| Sample Size | 65 | 70 |
| Geographic Representation | East North Central (MI) Pacific (CA) West North Central (NE, SD) |
East North Central (MI) Pacific (CA) West North Central (NE, SD) |
| Male | ||
| Female | ||
| Other | ||
| Gender Unknown | ||
| White, Non-Hispanic | ||
| Black, Non-Hispanic | ||
| Hispanic | ||
| Asian/Pacific Islander | ||
| American Indian/Alaska Native | ||
| Other | ||
| Race / Ethnicity Unknown | ||
| Low SES | ||
| IEP or diagnosed disability | ||
| English Language Learner |
Classification Accuracy - Winter
| Evidence | Grade 7 | Grade 8 |
|---|---|---|
| Criterion measure | SAT 10 Total Reading | SAT 10 Total Reading |
| Cut Points - Percentile rank on criterion measure | 20 | 20 |
| Cut Points - Performance score on criterion measure | 37 | 46 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 38 | 51 |
| Classification Data - True Positive (a) | 10 | 9 |
| Classification Data - False Positive (b) | 15 | 13 |
| Classification Data - False Negative (c) | 2 | 5 |
| Classification Data - True Negative (d) | 44 | 45 |
| Area Under the Curve (AUC) | 0.92 | 0.81 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.80 | 0.70 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 1.00 | 0.92 |
| Statistics | Grade 7 | Grade 8 |
|---|---|---|
| Base Rate | 0.17 | 0.19 |
| Overall Classification Rate | 0.76 | 0.75 |
| Sensitivity | 0.83 | 0.64 |
| Specificity | 0.75 | 0.78 |
| False Positive Rate | 0.25 | 0.22 |
| False Negative Rate | 0.17 | 0.36 |
| Positive Predictive Power | 0.40 | 0.41 |
| Negative Predictive Power | 0.96 | 0.90 |
| Sample | Grade 7 | Grade 8 |
|---|---|---|
| Date | 2017 | 2017 |
| Sample Size | 71 | 72 |
| Geographic Representation | East North Central (MI) Pacific (CA) West North Central (NE, SD) |
East North Central (MI) Pacific (CA) West North Central (NE, SD) |
| Male | ||
| Female | ||
| Other | ||
| Gender Unknown | ||
| White, Non-Hispanic | ||
| Black, Non-Hispanic | ||
| Hispanic | ||
| Asian/Pacific Islander | ||
| American Indian/Alaska Native | ||
| Other | ||
| Race / Ethnicity Unknown | ||
| Low SES | ||
| IEP or diagnosed disability | ||
| English Language Learner |
Classification Accuracy - Spring
| Evidence | Grade 7 | Grade 8 |
|---|---|---|
| Criterion measure | SAT 10 Total Reading | SAT 10 Total Reading |
| Cut Points - Percentile rank on criterion measure | 20 | 20 |
| Cut Points - Performance score on criterion measure | 37 | 46 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 44 | 52 |
| Classification Data - True Positive (a) | 10 | 10 |
| Classification Data - False Positive (b) | 12 | 14 |
| Classification Data - False Negative (c) | 1 | 4 |
| Classification Data - True Negative (d) | 44 | 44 |
| Area Under the Curve (AUC) | 0.94 | 0.84 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.86 | 0.73 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 1.00 | 0.94 |
| Statistics | Grade 7 | Grade 8 |
|---|---|---|
| Base Rate | 0.16 | 0.19 |
| Overall Classification Rate | 0.81 | 0.75 |
| Sensitivity | 0.91 | 0.71 |
| Specificity | 0.79 | 0.76 |
| False Positive Rate | 0.21 | 0.24 |
| False Negative Rate | 0.09 | 0.29 |
| Positive Predictive Power | 0.45 | 0.42 |
| Negative Predictive Power | 0.98 | 0.92 |
| Sample | Grade 7 | Grade 8 |
|---|---|---|
| Date | 2017 | 2017 |
| Sample Size | 67 | 72 |
| Geographic Representation | East North Central (MI) Pacific (CA) West North Central (NE, SD) |
East North Central (MI) Pacific (CA) West North Central (NE, SD) |
| Male | ||
| Female | ||
| Other | ||
| Gender Unknown | ||
| White, Non-Hispanic | ||
| Black, Non-Hispanic | ||
| Hispanic | ||
| Asian/Pacific Islander | ||
| American Indian/Alaska Native | ||
| Other | ||
| Race / Ethnicity Unknown | ||
| Low SES | ||
| IEP or diagnosed disability | ||
| English Language Learner |
Reliability
| Grade |
Grade 7
|
Grade 8
|
|---|---|---|
| Rating |
|
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Offer a justification for each type of reliability reported, given the type and purpose of the tool.
- Reliability refers to the relative stability with which a test measures the same skills across minor differences in conditions. Two types of reliability are reported in the table below, alternate form reliability, and alpha. Alternate form reliability is the correlation between different measures of the same early literacy skills. The coefficient reported is the average correlation among three forms of the measure. Coefficient alpha is a measure of reliability that is widely used in education research and represents the proportion of true score to total variance. Alpha incorporates information about the average inter-test correlation as well as the number of tests.
- *Describe the sample(s), including size and characteristics, for each reliability analysis conducted.
- The data were collected specifically for the purpose of assessing reliability. Students were given the typical benchmark Maze assessment, and then were given an alternate form of the same Maze measure. The sample sizes were 74 in grade 7 and 63 in grade 8, across multiple states.
- *Describe the analysis procedures for each reported type of reliability.
- Reliability was calculated as either alternate form reliability or alpha. Alternate form reliability was calculated as the correlation between the typical Maze benchmark adjusted score, with an additional Maze measure collected for the purpose of alternate form reliability. The correlation is the correlation between the sum of the adjusted scores for all three passages. Alpha was calculated as the internal consistency of the three Maze passages with each other. Thus, alternate form represents the correlation the Maze from different forms, alpha is the correlation of Maze within each form.
*In the table(s) below, report the results of the reliability analyses described above (e.g., internal consistency or inter-rater reliability coefficients).
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
- Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
- No
If yes, fill in data for each subgroup with disaggregated reliability data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
Validity
| Grade |
Grade 7
|
Grade 8
|
|---|---|---|
| Rating |
|
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.
- The Stanford Achievement Test (SAT)-10 Total Reading scores was used as the criterion measure to validate the 7-8 Maze measure. The Reading subtest of the SAT 10 was used to validate Maze, which consists of both Reading Vocabulary and Reading Comprehesion subtests. The SAT 10 is a widely used measure of reading achievement.
- *Describe the sample(s), including size and characteristics, for each validity analysis conducted.
- The data were collected across multiple states to validate the Maze measure. Samples were over 50 in grade 7 and over 70 in grade 8.
- *Describe the analysis procedures for each reported type of validity.
- Concurrent validity is the correlation between the Acadience 7-8 Maze measure and the SAT-10 Reading score at the same benchmark period, the end of the year. Predictive validity coefficients represent the correlation between an earlier Maze assessment (at the beginning or middle of year), and the SAT 10 Reading score at the end of the year.
*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
- Describe the degree to which the provided data support the validity of the tool.
- The Maze scores for Acadience Reading 7-8 are strongly correlated with the SAT-10 Total Reading score. This provides good evidence for the validity of Acadience Maze as part of the 7-8 suite of assessment.
- Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
- No
If yes, fill in data for each subgroup with disaggregated validity data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
Bias Analysis
| Grade |
Grade 8
|
Grade 7
|
|---|---|---|
| Rating | No | No |
- Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.
- No
- If yes,
- a. Describe the method used to determine the presence or absence of bias:
- b. Describe the subgroups for which bias analyses were conducted:
- c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.
Data Collection Practices
Most tools and programs evaluated by the NCII are branded products which have been submitted by the companies, organizations, or individuals that disseminate these products. These entities supply the textual information shown above, but not the ratings accompanying the text. NCII administrators and members of our Technical Review Committees have reviewed the content on this page, but NCII cannot guarantee that this information is free from error or reflective of recent changes to the product. Tools and programs have the opportunity to be updated annually or upon request.
