aimswebPlus Reading
Reading Composite
Summary
The aimswebPlus Reading Composite is an academic screening tool which combines the scores of three component measures to provide a general indicator of reading and related language arts abilities aligned to grade level expectations. The Reading Composite scores are available for universal and benchmark screening four times per year starting in Grade 2 and extending through high school. Specifically, students can complete testing to obtain a Reading Composite score in the beginning (fall), middle (winter), and end of the standard school year, and may also complete Reading Composite measures during the summer months. Component measures, which combine to provide the Reading Composite score, include Reading Comprehension (Grades 2-HS), Vocabulary (Grades 2-HS), and a reading fluency measure that differs depending on the student's grade level. Reading Composites for Grades 2-3 include Oral Reading Fluency, and Grades 4-HS include Silent Reading Fluency. These measures help to assess a wide range of reading skills students are expected to be learning and demonstrating at each grade level. The component measures available for each testing window within a grade level are equivalent in design and difficulty but unique, so each time students complete the reading composite they are assessed with new items and passages. Reading Comprehension scores are reported in the aimswebPlus web application, where teachers and administrators can track students' scores, observe growth over the school year, and receive feedback about how students are performing relative to peers and a nationally normed data.
- Where to Obtain:
- Pearson
- aimswebsupport@pearson.com
- Pearson Clinical Assessment, 927 E. Sonterra Blvd., Ste 119, San Antonio, TX 78258
- 1-866-313-6194
- www.pearsonassessment.com/aimswebPlus
- Initial Cost:
- $7.50 per student
- Replacement Cost:
- $7.50 per student per year
- Included in Cost:
- aimswebPlus is a subscription-based online solution that includes digital editions of training manuals and testing materials within the application. The per-student cost of $7.50 for one year grants access to all measures (Reading and Math). The per-student cost for one year's access to only Reading is $4.25. An aimswebPlus Unlimited Subscription is available for districts with enrollment of 2,500 students or fewer. It includes all aimswebPlus measures (Reading and Math) and these supplemental measures: Shaywitz DyslexiaScreen, BASC-3 BESS Teacher and Student Forms, WriteToLearn, and RAN Objects, Colors and Shapes. The cost for one year is $4995.00
- In general, accommodations consistent with IEPs and 504 plans are permitted with aimswebPlus, but modifications which bias or conflict with a measure's scoring logic are not recommended. Unique exceptions permitting modifications depend on the nature of the modification, and if the necessity and benefit to testing goals significantly outweigh any consequence to scoring. Extensive guidance on test accommodations, modifications, and their appropriate use is provided in the aimswebPlus Manual and other training materials.
- Training Requirements:
- Less than one hour of administrator training is required to learn best practices for giving one individually-administered measure and for proctoring group test sessions.
- Qualified Administrators:
- Administrators may be paraprofessional or professional members of the educational staff. All test administrators must understand the administration and scoring guidelines for all measures.
- Access to Technical Support:
- Pearson provides an extensive online Help database and offers both phone- and email-based support. A customer forum facilitates asking and answering questions, and additional on-site, virtual, and on-demand training may be purchased.
- Assessment Format:
-
- Scoring Time:
-
- Scoring is automatic OR
- 3 minutes per student
- Scores Generated:
-
- Raw score
- Percentile score
- Lexile score
- Error analysis
- Composite scores
- Subscale/subtest scores
- Administration Time:
-
- 45 minutes per student
- Scoring Method:
-
- Automatically (computer-scored)
- Other : For Oral Reading Fluency, test items are scored by the examiner on a digital record form as the the student responds. Responses can be reviewed before submitting each measure to be scored using the aimswebPlus platform.
- Technology Requirements:
-
- Computer or tablet
- Internet connection
- Accommodations:
- In general, accommodations consistent with IEPs and 504 plans are permitted with aimswebPlus, but modifications which bias or conflict with a measure's scoring logic are not recommended. Unique exceptions permitting modifications depend on the nature of the modification, and if the necessity and benefit to testing goals significantly outweigh any consequence to scoring. Extensive guidance on test accommodations, modifications, and their appropriate use is provided in the aimswebPlus Manual and other training materials.
Descriptive Information
- Please provide a description of your tool:
- The aimswebPlus Reading Composite is an academic screening tool which combines the scores of three component measures to provide a general indicator of reading and related language arts abilities aligned to grade level expectations. The Reading Composite scores are available for universal and benchmark screening four times per year starting in Grade 2 and extending through high school. Specifically, students can complete testing to obtain a Reading Composite score in the beginning (fall), middle (winter), and end of the standard school year, and may also complete Reading Composite measures during the summer months. Component measures, which combine to provide the Reading Composite score, include Reading Comprehension (Grades 2-HS), Vocabulary (Grades 2-HS), and a reading fluency measure that differs depending on the student's grade level. Reading Composites for Grades 2-3 include Oral Reading Fluency, and Grades 4-HS include Silent Reading Fluency. These measures help to assess a wide range of reading skills students are expected to be learning and demonstrating at each grade level. The component measures available for each testing window within a grade level are equivalent in design and difficulty but unique, so each time students complete the reading composite they are assessed with new items and passages. Reading Comprehension scores are reported in the aimswebPlus web application, where teachers and administrators can track students' scores, observe growth over the school year, and receive feedback about how students are performing relative to peers and a nationally normed data.
ACADEMIC ONLY: What skills does the tool screen?
- Please describe specific domain, skills or subtests:
- BEHAVIOR ONLY: Which category of behaviors does your tool target?
-
- BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.
Acquisition and Cost Information
Administration
- Are norms available?
- Yes
- Are benchmarks available?
- Yes
- If yes, how many benchmarks per year?
- 4
- If yes, for which months are benchmarks available?
- Fall (August-November), Winter (December - March 15), Spring (March 16 - May), Summer (June-July).
- BEHAVIOR ONLY: Can students be rated concurrently by one administrator?
- If yes, how many students can be rated concurrently?
Training & Scoring
Training
- Is training for the administrator required?
- Yes
- Describe the time required for administrator training, if applicable:
- Less than one hour of administrator training is required to learn best practices for giving one individually-administered measure and for proctoring group test sessions.
- Please describe the minimum qualifications an administrator must possess.
- Administrators may be paraprofessional or professional members of the educational staff. All test administrators must understand the administration and scoring guidelines for all measures.
-
No minimum qualifications
- Are training manuals and materials available?
- Yes
- Are training manuals/materials field-tested?
- No
- Are training manuals/materials included in cost of tools?
- Yes
- If No, please describe training costs:
- Can users obtain ongoing professional and technical support?
- Yes
- If Yes, please describe how users can obtain support:
- Pearson provides an extensive online Help database and offers both phone- and email-based support. A customer forum facilitates asking and answering questions, and additional on-site, virtual, and on-demand training may be purchased.
Scoring
- Do you provide basis for calculating performance level scores?
-
Yes
- Does your tool include decision rules?
-
No
- If yes, please describe.
- Can you provide evidence in support of multiple decision rules?
-
No
- If yes, please describe.
- Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.
- The aimswebPlus Reading composite is calculated by summing the scores of two or more measures. The composite for students in Grades 2 and 3 is formed using scores from Oral Reading Fluency, Vocabulary, and Reading Comprehension; specifically READ = (1/2*ORF) + VO + RC. The composite for students in Grades 4 through 8 is formed using scores from Silent Reading Fluency, Vocabulary, and Reading Comprehension; specifically READ = (1/2*SRF) + VO + RC. Because some of these measures assess fluency, and because Reading Comprehension is offered as a fixed form and a computer-adapted format, the form type and each student's skill competency will determine how many items are answered. Oral Reading Fluency measures a student's ability to read fictional English texts aloud quickly and accurately. For benchmark screening, the student sees a one-page story and reads as much of the story as possible in 1 minute, then a second one-page story is administered. Depending on the stories, and the student's ability and grade, as many as 400 items may be answered. One point is given for each word read correctly within the time limit, and student's raw score is the average words read correctly per minute based on the scores of both stories. Reading accuracy is also reported as the percentage of the words read correctly within the total number of words attempted. Administrators may optionally mark categories of miscues and qualitative observations to enhance score interpretation. Vocabulary measures the range of a student's knowledge of grade-appropriate words in isolation. The student chooses from multiple-choice options the response that best matches the meaning of vocabulary words. Students in Grade 2 take 16 items; students in Grades 3 through 8 take 22 items. The student's raw score is based on 1 point for each correct answer. Reading Comprehension measures a student's ability to read grade-appropriate informational and literary texts and demonstrate accurate understanding in multiple components of reading comprehension. The student reads passages silently and answers multiple-choice questions about each passage. Depending on the form, a student will take between 22 and 24 items. Both the fixed form and computer-adaptive formats are untimed. The fixed form presents the same ordered lists of passages and items to all students in the same grade and the computer-adaptive form presents a unique sequence of passages and items to each student adapted to their individual performance levels. Silent Reading Fluency measures a student's ability to quickly read and answer questions about grade-appropriate story segments when immediate feedback is given. The student reads three stories silently, one part/segment at a time, and answers multiple-choice questions about each segment under timed conditions. Each student takes 12 items. A student's raw score is a reading rate: the number of words read per minute based on the time spent reading story segments (not including time spent responding to items). Students must demonstrate sufficient evidence that they have understood a story segment for its reading rate to be scored. To obtain a valid score, students must show they understood at least two of the three stories. Demonstrating sufficient comprehension of a story requires three questions answered correctly out of the four. If sufficient comprehension is demonstrated, the median reading rate of each story read with sufficient accuracy is used to identify the student's median reading rate overall. Combining scores on essential reading-related skills into a composite that can be tracked over time presents a comprehensive and efficient evaluation of a student's on-grade-level performance.
- Describe the tool’s approach to screening, samples (if applicable), and/or test format, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.
- aimswebPlus was designed for the universal screening of Reading abilities, to document the academic performance data for students who likely have wide-ranging levels of ability, and to be an essential part of a school's strategy to meet state and federal accountability requirements. In the fall, winter, spring, and summer, the Reading benchmark screening measures provide data to identify students' academic strengths and weaknesses. When this benchmark screening reveals gaps among students, aimswebPlus provides information to help education teams decide next steps. Score reports can help identify specific areas of academic concern and may reveal patterns for creating instructional groups of students with similar learning needs. Oral Reading Fluency is administered individually, and requires administrator observation, judgment, and support (e.g., prompting). To facilitate this, aimswebPlus uses a computer- and tablet-compatible digital record form (DRF) to make it easy to capture and score student responses. The DRF includes all administration instructions; the words said aloud to the student, and practice items; the test items and method for capturing correct/incorrect responses; and other tools (e.g., timer or audio recorder) needed for accurate recording and scoring. Once the DRF with the student's responses are submitted to the aimswebPlus system, a score for each measure is generated automatically. TestNav, Pearson's secure, online testing portal, is used to administer Vocabulary, Reading Comprehension (both formats), and Silent Reading Fluency. Computer-based administration is required to offer computer-adaptive test forms and to capture the precise screen-exposure time required by Silent Reading Fluency. All of these measures may be given individually, or to groups of students who complete the test independently. The testing format allows for individual students or groups of students to complete the screening measures at the same time. Once the test session begins, students see all the instructions and test items on screens where they enter their responses. Instructions may be read or heard as often as is permitted, allowing students to work until all items are completed or until the time limit expires. Scores are calculated automatically, and reports are generated typically within 15 minutes. To support assessment with diverse populations, steps were taken during the development of each measure, beginning with the review of content by qualified experts to minimize bias and to ensure the appropriate skill coverage by age/grade. Test instructions are intentionally brief and rely on simple, grade-appropriate language. Pilot studies included students with diverse skill levels and backgrounds, and psychometric analyses were performed to show that items and measures are fair and free from bias. To ensure an equitable testing experience for all students, tools and guidance are offered to establish rapport and to address challenges for students related to test-taking, language, or physical differences. TestNav offers a range of accommodation tools, including the ability to magnify or zoom content and select different text and background color combinations. When allowed by a student's IEP accommodations, enlarging materials or adapting the physical environment are also permitted. For language-related differences, sign language or alternative modalities may be used to give test directions. Oral Reading Fluency includes specific guidance to avoid penalizing students for pronouncing words using regional dialects or articulation differences and may be given to assess reading speed and accuracy in English or Spanish, or in both languages. More generally, aimswebPlus offers guidance about when giving test items in Spanish is appropriate.
Technical Standards
Classification Accuracy & Cross-Validation Summary
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Classification Accuracy Fall |
|
|
|
|
|
|
|
| Classification Accuracy Winter |
|
|
|
|
|
|
|
| Classification Accuracy Spring |
|
|
|
|
|
|
|
Legend
Convincing evidencePartially convincing evidence
Unconvincing evidence
Data unavailabledDisaggregated data available
Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5
Classification Accuracy
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Grades 2-5: Alabama Comprehensive Assessment Program – English Language Arts (ACAP-ELA) was used as the criterion measure for grades 2-5. ACAP-ELA is a state-summative assessment measuring reading, vocabulary, language, and writing abilities with a focus on reading comprehension. This summative assessment is used to indicate a student's end of year proficiency in ELA skills generally and identify students' performing below expectation who may need intensive intervention. The ACAP-ELA has no direct relation to the aimswebPlus Reading Composite. ACAP-ELA was developed separately using data from unique samples of students and published by an organization that separate from the aimswebPlus research and development team.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- GRADES 2-5: Cut points on the ACAP-ELA were chosen to identify students with intensive needs in accordance with the definition provided by the NCII’s Technical Review Committee. Specifically, cut points were determined separately for each grade level as the ACAP-ELA scale score associated the 20th percentile of performance. Students with external criterion scores below the 20th percentile were labeled as “at-risk”; students with external criterion scores equal to or greater than the 20th percentile were labeled as “not at-risk” The cut points on the aimswebPlus Reading Composite score scale were determined by identifying the scores that maximized the sum of sensitivity and specificity with the classification outcomes of the ACAP-ELA. Students with Reading Composite scores below the cut point were classified as “at-risk”; students with Reading Composite scores at or above the cut point were classified as “not at-risk”. Lastly, classification indices were calculated using the formulates provided by the NCII Classification worksheet.
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
-
Yes
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
- Grades 2-5: We did not receive data specific to the types of interventions students were receiving in addition to standard classroom instruction. However, verbal reports from the school district we partnered with for analyses with ACAP-ELA data lead us to conclude it is likely students scoring below the 20th percentile received some type of intervention.
Cross-Validation
- Has a cross-validation study been conducted?
-
No
- If yes,
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8
Classification Accuracy
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Grades 6-8: Tennessee Comprehensive Assessment Program – English Language Arts (TCAP-ELA) was used as the criterion measure for grades 6-8. TCAP-ELA is a state-summative assessment measuring reading, vocabulary, language, and writing abilities with a focus on reading comprehension. This summative assessment is used to indicate a student's end of year proficiency in ELA skills generally and identify students' performing below expectations who may need intensive intervention. The TCAP-ELA was developed by Pearson School Assessments, which is a different division of Pearson Inc. There was no shared content across TCAP and aimswebPlus measures. The data was collected from unique samples of students and published by another Pearson division that is separate from the aimswebPlus research and development team.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- GRADES 6-8: Cut points on the TCAP-ELA were chosen to identify students with intensive needs in accordance with the definition provided by the NCII’s Technical Review Committee. Specifically, cut points were determined separately for each grade level as the TCAP-ELA scale score associated the 20th percentile of performance. Students with external criterion scores below the 20th percentile were labeled as “at-risk”; students with external criterion scores equal to or greater than the 20th percentile were labeled as “not at-risk” The cut points on the aimswebPlus Reading Composite score scale were determined by identifying the scores that maximized the sum of sensitivity and specificity with the classification outcomes of the TCAP-ELA. Students with Reading Composite scores below the cut point were classified as “at-risk”; students with Reading Composite scores at or above the cut point were classified as “not at-risk”. Lastly, classification indices were calculated using the formulates provided by the NCII Classification worksheet.
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
-
Yes
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
- Grades 6-8: We did not receive data specific to the types of interventions students were receiving in addition to standard classroom instruction. However, verbal reports from the school district we partnered with for analyses with TCAP-ELA data lead us to conclude it is likely students scoring below the 20th percentile received some type of intervention.
Cross-Validation
- Has a cross-validation study been conducted?
-
No
- If yes,
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Classification Accuracy - Fall
| Evidence | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Criterion measure | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 |
| Cut Points - Percentile rank on criterion measure | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
| Cut Points - Performance score on criterion measure | 460 | 467 | 453 | 457 | 288 | 298 | 287 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 266 | 319 | 406 | 423 | 464 | 473 | 492 |
| Classification Data - True Positive (a) | 40 | 15 | 43 | 47 | 474 | 358 | 407 |
| Classification Data - False Positive (b) | 63 | 15 | 61 | 38 | 593 | 351 | 387 |
| Classification Data - False Negative (c) | 5 | 3 | 2 | 7 | 90 | 81 | 96 |
| Classification Data - True Negative (d) | 355 | 176 | 321 | 321 | 2305 | 1365 | 1188 |
| Area Under the Curve (AUC) | 0.92 | 0.92 | 0.95 | 0.95 | 0.90 | 0.89 | 0.86 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.89 | 0.87 | 0.92 | 0.92 | 0.88 | 0.88 | 0.84 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.95 | 0.97 | 0.98 | 0.98 | 0.91 | 0.91 | 0.88 |
| Statistics | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Base Rate | 0.10 | 0.09 | 0.11 | 0.13 | 0.16 | 0.20 | 0.24 |
| Overall Classification Rate | 0.85 | 0.91 | 0.85 | 0.89 | 0.80 | 0.80 | 0.77 |
| Sensitivity | 0.89 | 0.83 | 0.96 | 0.87 | 0.84 | 0.82 | 0.81 |
| Specificity | 0.85 | 0.92 | 0.84 | 0.89 | 0.80 | 0.80 | 0.75 |
| False Positive Rate | 0.15 | 0.08 | 0.16 | 0.11 | 0.20 | 0.20 | 0.25 |
| False Negative Rate | 0.11 | 0.17 | 0.04 | 0.13 | 0.16 | 0.18 | 0.19 |
| Positive Predictive Power | 0.39 | 0.50 | 0.41 | 0.55 | 0.44 | 0.50 | 0.51 |
| Negative Predictive Power | 0.99 | 0.98 | 0.99 | 0.98 | 0.96 | 0.94 | 0.93 |
| Sample | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Date | |||||||
| Sample Size | 463 | 209 | 427 | 413 | 3462 | 2155 | 2078 |
| Geographic Representation | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (TN) | East South Central (TN) | East South Central (TN) |
| Male | 48.6% | 45.9% | 47.8% | 47.2% | |||
| Female | 51.4% | 54.1% | 52.2% | 52.8% | |||
| Other | |||||||
| Gender Unknown | |||||||
| White, Non-Hispanic | |||||||
| Black, Non-Hispanic | 19.0% | 21.5% | 23.7% | 22.8% | |||
| Hispanic | 22.7% | 23.4% | 20.8% | 20.3% | |||
| Asian/Pacific Islander | 1.3% | 1.0% | 1.2% | 2.2% | |||
| American Indian/Alaska Native | 0.2% | 1.2% | 0.5% | ||||
| Other | |||||||
| Race / Ethnicity Unknown | 56.8% | 54.1% | 53.2% | 54.2% | 100.0% | 100.0% | 100.0% |
| Low SES | |||||||
| IEP or diagnosed disability | |||||||
| English Language Learner |
Classification Accuracy - Winter
| Evidence | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Criterion measure | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 |
| Cut Points - Percentile rank on criterion measure | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
| Cut Points - Performance score on criterion measure | 460 | 467 | 453 | 457 | 288 | 298 | 287 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 311 | 364 | 414 | 449 | 469 | 464 | 501 |
| Classification Data - True Positive (a) | 44 | 16 | 41 | 50 | 507 | 303 | 381 |
| Classification Data - False Positive (b) | 59 | 23 | 60 | 49 | 573 | 171 | 270 |
| Classification Data - False Negative (c) | 1 | 2 | 6 | 4 | 75 | 90 | 48 |
| Classification Data - True Negative (d) | 364 | 173 | 324 | 318 | 2348 | 991 | 751 |
| Area Under the Curve (AUC) | 0.96 | 0.94 | 0.93 | 0.96 | 0.92 | 0.90 | 0.89 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.94 | 0.89 | 0.89 | 0.93 | 0.91 | 0.88 | 0.87 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.98 | 0.99 | 0.96 | 0.98 | 0.93 | 0.91 | 0.91 |
| Statistics | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Base Rate | 0.10 | 0.08 | 0.11 | 0.13 | 0.17 | 0.25 | 0.30 |
| Overall Classification Rate | 0.87 | 0.88 | 0.85 | 0.87 | 0.82 | 0.83 | 0.78 |
| Sensitivity | 0.98 | 0.89 | 0.87 | 0.93 | 0.87 | 0.77 | 0.89 |
| Specificity | 0.86 | 0.88 | 0.84 | 0.87 | 0.80 | 0.85 | 0.74 |
| False Positive Rate | 0.14 | 0.12 | 0.16 | 0.13 | 0.20 | 0.15 | 0.26 |
| False Negative Rate | 0.02 | 0.11 | 0.13 | 0.07 | 0.13 | 0.23 | 0.11 |
| Positive Predictive Power | 0.43 | 0.41 | 0.41 | 0.51 | 0.47 | 0.64 | 0.59 |
| Negative Predictive Power | 1.00 | 0.99 | 0.98 | 0.99 | 0.97 | 0.92 | 0.94 |
| Sample | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Date | |||||||
| Sample Size | 468 | 214 | 431 | 421 | 3503 | 1555 | 1450 |
| Geographic Representation | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (TN) | East South Central (TN) | East South Central (TN) |
| Male | 48.7% | 46.3% | 47.8% | 46.6% | |||
| Female | 51.3% | 53.7% | 52.2% | 53.4% | |||
| Other | |||||||
| Gender Unknown | |||||||
| White, Non-Hispanic | |||||||
| Black, Non-Hispanic | 19.4% | 22.0% | 23.7% | 22.8% | |||
| Hispanic | 22.9% | 22.9% | 21.1% | 20.4% | |||
| Asian/Pacific Islander | 1.3% | 0.9% | 1.2% | 2.1% | |||
| American Indian/Alaska Native | 0.2% | 1.2% | 0.5% | ||||
| Other | |||||||
| Race / Ethnicity Unknown | 56.2% | 54.2% | 52.9% | 54.2% | 100.0% | 100.0% | 100.0% |
| Low SES | |||||||
| IEP or diagnosed disability | |||||||
| English Language Learner |
Classification Accuracy - Spring
| Evidence | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Criterion measure | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Alabama Comprehensive Assessment Program - English Language Arts (ACAP-ELA) Grades 2-5 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 | Tennessee Comprehensive Assessment Program - English Language Arts (TCAP-ELA) Grades 6-8 |
| Cut Points - Percentile rank on criterion measure | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
| Cut Points - Performance score on criterion measure | 460 | 467 | 453 | 457 | 288 | 298 | 287 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 348 | 388 | 435 | 440 | 475 | 478 | 505 |
| Classification Data - True Positive (a) | 41 | 19 | 48 | 44 | 500 | 390 | 423 |
| Classification Data - False Positive (b) | 62 | 32 | 85 | 22 | 692 | 454 | 480 |
| Classification Data - False Negative (c) | 3 | 1 | 2 | 10 | 51 | 85 | 62 |
| Classification Data - True Negative (d) | 365 | 164 | 305 | 351 | 2194 | 2085 | 2032 |
| Area Under the Curve (AUC) | 0.95 | 0.94 | 0.94 | 0.95 | 0.92 | 0.91 | 0.91 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.93 | 0.86 | 0.91 | 0.92 | 0.91 | 0.89 | 0.90 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.97 | 0.99 | 0.97 | 0.98 | 0.93 | 0.92 | 0.93 |
| Statistics | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Base Rate | 0.09 | 0.09 | 0.11 | 0.13 | 0.16 | 0.16 | 0.16 |
| Overall Classification Rate | 0.86 | 0.85 | 0.80 | 0.93 | 0.78 | 0.82 | 0.82 |
| Sensitivity | 0.93 | 0.95 | 0.96 | 0.81 | 0.91 | 0.82 | 0.87 |
| Specificity | 0.85 | 0.84 | 0.78 | 0.94 | 0.76 | 0.82 | 0.81 |
| False Positive Rate | 0.15 | 0.16 | 0.22 | 0.06 | 0.24 | 0.18 | 0.19 |
| False Negative Rate | 0.07 | 0.05 | 0.04 | 0.19 | 0.09 | 0.18 | 0.13 |
| Positive Predictive Power | 0.40 | 0.37 | 0.36 | 0.67 | 0.42 | 0.46 | 0.47 |
| Negative Predictive Power | 0.99 | 0.99 | 0.99 | 0.97 | 0.98 | 0.96 | 0.97 |
| Sample | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Date | |||||||
| Sample Size | 471 | 216 | 440 | 427 | 3437 | 3014 | 2997 |
| Geographic Representation | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (TN) | East South Central (TN) | East South Central (TN) |
| Male | 49.0% | 46.3% | 47.0% | 46.6% | |||
| Female | 51.0% | 53.7% | 53.0% | 53.4% | |||
| Other | |||||||
| Gender Unknown | |||||||
| White, Non-Hispanic | |||||||
| Black, Non-Hispanic | 19.1% | 21.8% | 24.3% | 23.0% | |||
| Hispanic | 23.4% | 23.6% | 21.4% | 20.4% | |||
| Asian/Pacific Islander | 1.3% | 0.9% | 1.1% | 2.1% | |||
| American Indian/Alaska Native | 0.2% | 1.1% | 0.5% | ||||
| Other | |||||||
| Race / Ethnicity Unknown | 56.1% | 53.7% | 52.0% | 54.1% | 100.0% | 100.0% | 100.0% |
| Low SES | |||||||
| IEP or diagnosed disability | |||||||
| English Language Learner |
Reliability
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Rating |
|
|
|
|
|
|
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Offer a justification for each type of reliability reported, given the type and purpose of the tool.
- Reading Composite is used to measure reading abilities as a single general construct reflecting performance on a combination of key skills students are expected to learn and develop in Grades 2-8. To test the reliability of the Reading Composite, we calculated coefficient omega using a model-based approach for each grade level and each benchmark screening season (Fall, Winter, and Spring). Coefficient omega indicates how well the component measures of the Reading Composite reliably measure the same general construct. This reliability approach accounts for how the Reading composite score is calculated as the combination of scores from unique component measures, accounting for differences between component measure scores and their relationship to the general measure of Reading ability. (Reference: McDonald, R. P. (1978). Generalizability in factorable domains: “domain validity and generalizability”: 1. Educational and Psychological Measurement, 38 (1), 75-79.)
- *Describe the sample(s), including size and characteristics, for each reliability analysis conducted.
- The data sample was comprised of students from Grades 2-8 (n = 23,114) from 34 states. Demographic data regarding sex, ethnicity, and English as a second language (ESL) were analyzed. Student data showed that 48.6% of the population was female and 51.4% was male. Furthermore, an analysis of the ethnicities that comprised the sample showed the following : American-Indian (1.6%), Asian (4.9%), Black (16.3%), Hispanic (24.7%), Multi-Racial (6.0%), Pacific-Islander (0.6%), and White (46.0%). Most students in the sample spoke English as their first language (91.4%) and some were ESL students (8.6%).
- *Describe the analysis procedures for each reported type of reliability.
- Omega coefficients were calculated for each Reading grade level (Grade 2 through Grade 8) and benchmark screening season (Fall, Winter, and Spring). Analyses were conducted in R. Omega (total) coefficients were calculated using the “omega” function from the “psych” package. This method first performs a factor analysis on the scores of the Reading composite’s component measures to extract a single common factor. Omega total is then calculated as the proportion of total variance in the scores attributed to the single common factor. 95% confidence intervals of the reliability coefficient were estimated through a bootstrapping approach using the “boot” package.
*In the table(s) below, report the results of the reliability analyses described above (e.g., internal consistency or inter-rater reliability coefficients).
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
- Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
If yes, fill in data for each subgroup with disaggregated reliability data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
Validity
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Rating |
|
|
|
|
|
|
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.
- Two external criterion measures were used in our validity analyses: the English Language Arts end-of-year assessment from the Tennessee Comprehensive Assessment Program (TCAP-ELA) and the English Language Arts end-of-year assessment from the Alabama Comprehensive Assessment Program (ACAP-ELA). ACAP was also used as our external criterion measure for concurrent and predictive validity analyses for grades 2-5. TCAP-ELA was used as our external criterion measure for our concurrent and predictive validity analyses for grades 3-8. Both measures are appropriate for concurrent validity analyses focused on evaluating the construct validity of the aimswebPlus Reading Composite. Both the Reading composite and the two criterion measures are intended to measure Reading comprehension abilities with literary and informational texts aligned to grade-level state standard expectations as well as related language arts skills like Vocabulary. Using both external measures for predictive validity analyses aligns with how aimswebPlus Reading Composite scores are intended to provide teachers and school districts with valid indications of how students may perform on summative end of year.
- *Describe the sample(s), including size and characteristics, for each validity analysis conducted.
- Samples of students from Tennessee and Alabama were included in our validity analyses. The Tennessee sample comes from a large school district with students represented across 45 elementary schools (Grades 3-5) and 21 middle schools (Grades 6-8) in urban, suburban, and rural regions. Students of all ability levels in this district completed the Reading Composite as a part of their universal screening assessments at the beginning, middle, and end of the school year. The Alabama sample comes from a school district with students represented across 4 schools (Grades 2-5) of varying sizes and locations around a medium sized suburban city. Demographic data indicate the Alabama sample was drawn from a diverse district composed of multiple ethnic and socioeconomic backgrounds. For predictive validity analyses, students in the Tennessee sample in Grades 3-8 completed Fall and Winter Reading Composite measures during the 2022-2023 school year and the TCAP-ELA assessment in the Spring of 2023. Students in the Alabama sample in Grades 2-5 completed Fall and Winter Reading Composite measures during the 2023-2024 school year, and ACAP-ELA assessments in the Spring of 2024. For concurrent validity analyses, students in the Tennessee sample in Grades 3-8 completed Reading Composite measures and the TCAP-ELA assessment in the Spring of 2023. Students in the Alabama sample in Grades 2-5 completed Reading Composite measures and the TCAP-ELA assessment in the Spring of 2024.
- *Describe the analysis procedures for each reported type of validity.
- Two types of validity analyses were conducted with Reading and the external criterion measures: predictive validity and concurrent validity. Predictive Validity analyses for Grades 3-8 were conducted by examining the strength of the Pearson correlation coefficient between Reading scores gathered multiple months prior to TCAP-ELA scores for the same students. The predictive validity analyses for grades 3-8 correlation coefficients were calculated between scores from Reading tests given in the Fall of 2022 (August-November) and TCAP-ELA given in the Spring (March) of 2023. For the predictive validity analysis for Grades 2-5, correlation coefficients were calculated between scores from Reading tests given in the Spring of 2021 and ACAP scores of the same students assessed at the end of Grade 3 in the Spring of 2023. Concurrent Validity analyses were conducted for Grades 3-8 by examining the strength of the Pearson correlation coefficient between Reading scores and the external criterion measures given from TCAP-ELA. For the concurrent validity analysis for Grades 2-5 correlation coefficients were calculated between Pearson correlation coefficient and the ACAP Reading test scores (end of January to beginning of February) For predictive and concurrent validity analyses, 95% confidence intervals for the correlation coefficients using the Fischer z-transformation.
*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
- Describe the degree to which the provided data support the validity of the tool.
- Results of the concurrent and predictive validity analyses for all grade levels present strong evidence that the validity of aimswebPlus Reading Composite scores. Concurrent validity analyses provide strong indications of construct validity, with positive associations between Reading Composite scores and TCAP and ACAP scores designed to measure similar reading abilities aligned to grade-level standards. Predictive validity results provide additional evidence that the Grade 2-Grade 8 Reading Composites have strong predictive utility for the skills students are expected demonstrate on end-of-year state summative exams. The same was observed in Grades 3-Grade 8 with strong correlations observed in Reading Composites and TCAP reading scores. Together, predictive and concurrent validity results provide convergent evidence that the Reading Composites measure the reading abilities they are designed to assess and can provide valuable predictive insights for future performance.
- Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
- No
If yes, fill in data for each subgroup with disaggregated validity data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
Bias Analysis
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Rating | Not Provided | Not Provided | Not Provided | Not Provided | Not Provided | Not Provided | Not Provided |
- Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.
- No
- If yes,
- a. Describe the method used to determine the presence or absence of bias:
- b. Describe the subgroups for which bias analyses were conducted:
- c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.
Data Collection Practices
Most tools and programs evaluated by the NCII are branded products which have been submitted by the companies, organizations, or individuals that disseminate these products. These entities supply the textual information shown above, but not the ratings accompanying the text. NCII administrators and members of our Technical Review Committees have reviewed the content on this page, but NCII cannot guarantee that this information is free from error or reflective of recent changes to the product. Tools and programs have the opportunity to be updated annually or upon request.
