aimswebPlus Math
Math Composite
Summary
The aimswebPlus Math Composite is an academic screening tool which combines the scores of three component measures to provide a general indicator of math abilities aligned to grade level expectations. The Math Composite scores are available for universal and benchmark screening four times per year starting in Grade 2 and extending through high school. Specifically, students can complete testing to obtain a Math Composite score in the beginning (fall), middle (winter), and end of the standard school year, and may also complete Math Composite measures during the summer months. Component measures, which combine to provide the Math Composite score, include Concepts and Applications, Number Comparison Fluency – Triads, and Mental Computation Fluency. These measures help to assess a wide range of math skills students are expected to be learning and demonstrating at each grade level. The component measures available for each testing window within a grade level are equivalent in design and difficulty but unique, so each time students complete the Math Composite they are assessed with new items. Math Composite scores are reported in the aimswebPlus web application, where teachers and administrators can track students' scores, observe growth over the school year, and receive feedback about how students are performing relative to peers and nationally normed data.
- Where to Obtain:
- Pearson
- aimswebsupport@pearson.com
- Pearson Clinical Assessment, 927 E. Sonterra Blvd., Ste 119, San Antonio, TX 78258
- 1-866-313-6194
- www.pearsonassessment.com/aimswebPlus
- Initial Cost:
- $7.50 per student
- Replacement Cost:
- $7.50 per student per year
- Included in Cost:
- aimswebPlus is a subscription-based online solution that includes digital editions of training manuals and testing materials within the application. The per-student cost of $7.50 for one year grants access to all measures (Reading and Math). The per-student cost for one year's access to only Math is $4.25. An aimswebPlus Unlimited Subscription is available for districts with enrollment of 2,500 students or fewer. It includes all aimswebPlus measures (Reading and Math) and these supplemental measures: Shaywitz DyslexiaScreen, BASC-3 BESS Teacher and Student Forms, WriteToLearn, and RAN Objects, Colors and Shapes. The cost for one year is $4995.00
- In general, accommodations consistent with IEPs and 504 plans are permitted with aimswebPlus, but modifications which bias or conflict with a measure's scoring logic are not recommended. Unique exceptions permitting modifications depend on the nature of the modification, and if the necessity and benefit to testing goals significantly outweigh any consequence to scoring. Extensive guidance on test accommodations, modifications, and their appropriate use is provided in the aimswebPlus Manual and other training materials.
- Training Requirements:
- Less than one hour of administrator training is required to learn best practices for proctoring group test sessions.
- Qualified Administrators:
- Administrators may be paraprofessional or professional members of the educational staff. All test administrators must understand the administration and scoring guidelines for all measures.
- Access to Technical Support:
- Pearson provides an extensive online Help database and offers both phone- and email-based support. A customer forum facilitates asking and answering questions, and additional on-site, virtual, and on-demand training may be purchased.
- Assessment Format:
-
- Scoring Time:
-
- Scoring is automatic
- Scores Generated:
-
- Raw score
- Percentile score
- Error analysis
- Composite scores
- Subscale/subtest scores
- Other: Quantile score
- Administration Time:
-
- 40 minutes per student
- Scoring Method:
-
- Automatically (computer-scored)
- Technology Requirements:
-
- Computer or tablet
- Internet connection
- Accommodations:
- In general, accommodations consistent with IEPs and 504 plans are permitted with aimswebPlus, but modifications which bias or conflict with a measure's scoring logic are not recommended. Unique exceptions permitting modifications depend on the nature of the modification, and if the necessity and benefit to testing goals significantly outweigh any consequence to scoring. Extensive guidance on test accommodations, modifications, and their appropriate use is provided in the aimswebPlus Manual and other training materials.
Descriptive Information
- Please provide a description of your tool:
- The aimswebPlus Math Composite is an academic screening tool which combines the scores of three component measures to provide a general indicator of math abilities aligned to grade level expectations. The Math Composite scores are available for universal and benchmark screening four times per year starting in Grade 2 and extending through high school. Specifically, students can complete testing to obtain a Math Composite score in the beginning (fall), middle (winter), and end of the standard school year, and may also complete Math Composite measures during the summer months. Component measures, which combine to provide the Math Composite score, include Concepts and Applications, Number Comparison Fluency – Triads, and Mental Computation Fluency. These measures help to assess a wide range of math skills students are expected to be learning and demonstrating at each grade level. The component measures available for each testing window within a grade level are equivalent in design and difficulty but unique, so each time students complete the Math Composite they are assessed with new items. Math Composite scores are reported in the aimswebPlus web application, where teachers and administrators can track students' scores, observe growth over the school year, and receive feedback about how students are performing relative to peers and nationally normed data.
ACADEMIC ONLY: What skills does the tool screen?
- Please describe specific domain, skills or subtests:
- BEHAVIOR ONLY: Which category of behaviors does your tool target?
-
- BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.
Acquisition and Cost Information
Administration
- Are norms available?
- Yes
- Are benchmarks available?
- Yes
- If yes, how many benchmarks per year?
- 4
- If yes, for which months are benchmarks available?
- all
- BEHAVIOR ONLY: Can students be rated concurrently by one administrator?
- If yes, how many students can be rated concurrently?
Training & Scoring
Training
- Is training for the administrator required?
- Yes
- Describe the time required for administrator training, if applicable:
- Less than one hour of administrator training is required to learn best practices for proctoring group test sessions.
- Please describe the minimum qualifications an administrator must possess.
- Administrators may be paraprofessional or professional members of the educational staff. All test administrators must understand the administration and scoring guidelines for all measures.
-
No minimum qualifications
- Are training manuals and materials available?
- Yes
- Are training manuals/materials field-tested?
- No
- Are training manuals/materials included in cost of tools?
- Yes
- If No, please describe training costs:
- Can users obtain ongoing professional and technical support?
- Yes
- If Yes, please describe how users can obtain support:
- Pearson provides an extensive online Help database and offers both phone- and email-based support. A customer forum facilitates asking and answering questions, and additional on-site, virtual, and on-demand training may be purchased.
Scoring
- Do you provide basis for calculating performance level scores?
-
Yes
- Does your tool include decision rules?
-
No
- If yes, please describe.
- Can you provide evidence in support of multiple decision rules?
-
No
- If yes, please describe.
- Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.
- The aimswebPlus Math composite is calculated by summing the scores of two or more measures. The composite for students in Grades 2 through 8 is formed by the sum of Number Comparison Fluency–Triads, Mental Computation Fluency, and Concepts & Applications scores. Because some of these measures assess fluency, and because Concepts & Applications is offered as a fixed form and a computer-adapted format, the form type and each student's skill competency will determine how many items are answered. Number Comparison Fluency–Triads measures a student's ability to assess the magnitude of numbers and make comparisons quickly and accurately. The student answers multiple-choice items making number comparisons with 3 response options within a 3-minute time limit. Depending on the student's ability, they may answer as many as 40 items. A student's raw score is based on 1 point for each correct answer within the time limit. Mental Computation Fluency measures a student's ability to solve math computation problems quickly and accurately. The student answers multiple-choice items, choosing the solution to one- or two-step mental computations from 3 response options within a 4-minute time limit. Depending on the student's ability, they may answer as many as 42 items. A student's raw score is based on 1 point for each correct answer within the time limit. Concepts & Applications measures a student's understanding of essential math concepts and their ability to apply that knowledge to solve math problems aligned to common expectations at each grade level. Depending on the form, a student will take about 30 items. Both the fixed form and computer-adaptive formats are untimed, whereas the fixed form presents the same ordered list of items to all students in the same grade and the computer-adaptive form presents a unique sequence of items to each student adapted to their individual performance levels. The computer-adaptive form includes mostly multiple-choice items, but additional item types are also included (e.g., a student may show knowledge of number magnitude by clicking points on a number line). Combining the scores on essential math skills into a composite that can be tracked over time presents a comprehensive and efficient evaluation of a student's on-grade-level performance.
- Describe the tool’s approach to screening, samples (if applicable), and/or test format, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.
- aimswebPlus was designed for the universal screening of Math abilities, to document the academic performance data for students who likely have wide-ranging levels of ability, and to be an essential part of a school's strategy to meet state and federal accountability requirements. In the fall, winter, spring, and summer, the Math benchmark screening measures provide data to identify students' academic strengths and weaknesses. When this benchmark screening reveals gaps among students, aimswebPlus provides information to help education teams decide next steps. Score reports can help identify specific areas of academic concern and may reveal patterns for creating instructional groups of students with similar learning needs. TestNav, Pearson's secure, online testing portal, is used to administer Number Comparison Fluency–Triads, Mental Computation Fluency, and Concepts & Applications scores. Computer-based administration is required to offer computer-adaptive test forms. All of these measures may be given individually, or to groups of students who complete the test independently. The testing format allows for individual students or groups of students to complete the screening measures at the same time. Once the test session begins, students see all the instructions and test items on screens where they enter their responses. Instructions may be read or heard as often as is permitted, allowing students to work until all items are completed or until the time limit expires. Scores are calculated automatically, and reports are generated typically within 15 minutes. To support assessment with diverse populations, steps were taken during the development of each measure, beginning with the review of content by qualified experts to minimize bias and to ensure the appropriate skill coverage by age/grade. Test instructions are intentionally brief and rely on simple, grade-appropriate language. Pilot studies included students with diverse skill levels and backgrounds, and psychometric analyses were performed to show that items and measures are fair and free from bias. To ensure an equitable testing experience for all students, tools and guidance are offered to establish rapport and to address challenges for students related to test-taking, language, or physical differences. TestNav offers a range of accommodation tools, including the ability to magnify or zoom content and select different text and background color combinations. When allowed by a student's IEP accommodations, enlarging materials or adapting the physical environment are also permitted. For language-related differences, sign language or alternative modalities may be used to give test directions. Students who speak Spanish may be given the administration instructions in either Spanish or English. aimswebPlus also offers general guidance for when it's appropriate to give test items in Spanish.
Technical Standards
Classification Accuracy & Cross-Validation Summary
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Classification Accuracy Fall |
|
|
|
|
|
|
|
| Classification Accuracy Winter |
|
|
|
|
|
|
|
| Classification Accuracy Spring |
|
|
|
|
|
|
|
Legend
Convincing evidencePartially convincing evidence
Unconvincing evidence
Data unavailabledDisaggregated data available
Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5
Classification Accuracy
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- GRADES 2-5: Alabama Comprehensive Assessment Program – Mathematics (ACAP-Math) was used as the criterion measure for grades 2-5. ACAP-Math is a state-summative assessment measuring a combination of math abilities aligned to grade-level specific state standards for math achievement. This summative assessment is used to indicate a student's end of year proficiency in math and identify students' performing below expectations who may need intensive intervention. The ACAP-Math has no direct relation to the aimswebPlus Mathematics Composite. ACAP-Math was developed separately using data from unique samples of students and published by an organization that is separate from the aimswebPlus research and development team.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- GRADES 2-5: Cut points on the ACAP-Math were chosen to identify students with intensive needs in accordance with the definition provided by the NCII’s Technical Review Committee. Specifically, cut points were determined separately for each grade level as the ACAP-Math scale score associated the 20th percentile of performance. Students with external criterion scores below the 20th percentile were labeled as “at-risk”; students with external criterion scores equal to or greater than the 20th percentile were labeled as “not at-risk” The cut points on the aimswebPlus Math Composite score scale were determined by identifying the scores that maximized the sum of sensitivity and specificity with the classification outcomes of the ACAP-Math. Students with Math Composite scores below the cut point were classified as “at-risk”; students with Math Composite scores at or above the cut point were classified as “not at-risk”. Lastly, classification indices were calculated using the formulates provided by the NCII Classification worksheet.
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
-
Yes
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
- Grades 2-5: We did not receive data specific to the types of interventions students were receiving in addition to standard classroom instruction. However, verbal reports from the school district we partnered with for analyses with ACAP-Math data lead us to conclude it is likely students scoring below the 20th percentile received some type of intervention.
Cross-Validation
- Has a cross-validation study been conducted?
-
No
- If yes,
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8
Classification Accuracy
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- GRADES 6-8: Tennessee Comprehensive Assessment Program – Math (TCAP-Math) was used as the criterion measure for grades 6-8. TCAP-Math is a state-summative assessment measuring a combination of math abilities aligned to grade-level specific state standards for math achievement. This summative assessment is used to indicate a student's end of year proficiency in math and identify students' performing below expectations who may need intensive intervention. The TCAP-Math has no direct relation to the aimswebPlus Mathematics Composite. The TCAP-Math was developed by Pearson School Assessments, which is a different division of Pearson Inc. There was no shared content across TCAP and aimswebPlus measures. The data was collected from unique samples of students and published by another Pearson division that is separate from the aimswebPlus research and development team.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- GRADES 6-8: Cut points on the TCAP-Math were chosen to identify students with intensive needs in accordance with the definition provided by the NCII’s Technical Review Committee. Specifically, cut points were determined separately for each grade level as the TCAP-Math scale score associated the 20th percentile of performance. Students with external criterion scores below the 20th percentile were labeled as “at-risk”; students with external criterion scores equal to or greater than the 20th percentile were labeled as “not at-risk” The cut points on the aimswebPlus Math Composite score scale were determined by identifying the scores that maximized the sum of sensitivity and specificity with the classification outcomes of the TCAP-Math. Students with Math Composite scores below the cut point were classified as “at-risk”; students with Math Composite scores at or above the cut point were classified as “not at-risk”. Lastly, classification indices were calculated using the formulates provided by the NCII Classification worksheet.
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
-
Yes
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
- Grades 6-8: We did not receive data specific to the types of interventions students were receiving in addition to standard classroom instruction. However, verbal reports from the school district we partnered with for analyses with TCAP-Math data lead us to conclude it is likely students scoring below the 20th percentile received some type of intervention.
Cross-Validation
- Has a cross-validation study been conducted?
-
No
- If yes,
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Classification Accuracy - Fall
| Evidence | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Criterion measure | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 |
| Cut Points - Percentile rank on criterion measure | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
| Cut Points - Performance score on criterion measure | 473 | 478 | 476 | 477 | 294 | 285 | 274 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 145 | 167 | 187 | 194 | 201 | 206 | 207 |
| Classification Data - True Positive (a) | 25 | 13 | 41 | 27 | 645 | 391 | 424 |
| Classification Data - False Positive (b) | 161 | 39 | 60 | 45 | 652 | 495 | 542 |
| Classification Data - False Negative (c) | 1 | 0 | 5 | 4 | 104 | 76 | 90 |
| Classification Data - True Negative (d) | 279 | 158 | 325 | 340 | 2587 | 1381 | 909 |
| Area Under the Curve (AUC) | 0.85 | 0.91 | 0.94 | 0.92 | 0.90 | 0.86 | 0.78 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.79 | 0.86 | 0.90 | 0.88 | 0.89 | 0.85 | 0.76 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.91 | 0.96 | 0.97 | 0.96 | 0.91 | 0.88 | 0.81 |
| Statistics | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Base Rate | 0.06 | 0.06 | 0.11 | 0.07 | 0.19 | 0.20 | 0.26 |
| Overall Classification Rate | 0.65 | 0.81 | 0.85 | 0.88 | 0.81 | 0.76 | 0.68 |
| Sensitivity | 0.96 | 1.00 | 0.89 | 0.87 | 0.86 | 0.84 | 0.82 |
| Specificity | 0.63 | 0.80 | 0.84 | 0.88 | 0.80 | 0.74 | 0.63 |
| False Positive Rate | 0.37 | 0.20 | 0.16 | 0.12 | 0.20 | 0.26 | 0.37 |
| False Negative Rate | 0.04 | 0.00 | 0.11 | 0.13 | 0.14 | 0.16 | 0.18 |
| Positive Predictive Power | 0.13 | 0.25 | 0.41 | 0.38 | 0.50 | 0.44 | 0.44 |
| Negative Predictive Power | 1.00 | 1.00 | 0.98 | 0.99 | 0.96 | 0.95 | 0.91 |
| Sample | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Date | |||||||
| Sample Size | 466 | 210 | 431 | 416 | 3988 | 2343 | 1965 |
| Geographic Representation | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (TN) | East South Central (TN) | East South Central (TN) |
| Male | 48.9% | 46.2% | 48.0% | 47.1% | |||
| Female | 51.1% | 53.8% | 52.0% | 52.9% | |||
| Other | |||||||
| Gender Unknown | |||||||
| White, Non-Hispanic | |||||||
| Black, Non-Hispanic | 18.9% | 21.9% | 23.4% | 22.6% | |||
| Hispanic | 23.0% | 23.3% | 21.3% | 20.4% | |||
| Asian/Pacific Islander | 1.3% | 1.0% | 1.2% | 2.2% | |||
| American Indian/Alaska Native | 0.2% | 1.2% | 0.5% | ||||
| Other | |||||||
| Race / Ethnicity Unknown | 56.7% | 53.8% | 52.9% | 54.3% | 100.0% | 100.0% | 100.0% |
| Low SES | |||||||
| IEP or diagnosed disability | |||||||
| English Language Learner |
Classification Accuracy - Winter
| Evidence | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Criterion measure | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 |
| Cut Points - Percentile rank on criterion measure | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
| Cut Points - Performance score on criterion measure | 473 | 478 | 476 | 477 | 294 | 285 | 274 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 152 | 188 | 204 | 220 | 201 | 209 | 210 |
| Classification Data - True Positive (a) | 29 | 16 | 42 | 30 | 651 | 427 | 341 |
| Classification Data - False Positive (b) | 61 | 38 | 75 | 60 | 576 | 518 | 463 |
| Classification Data - False Negative (c) | 0 | 0 | 4 | 2 | 113 | 54 | 61 |
| Classification Data - True Negative (d) | 386 | 161 | 311 | 331 | 2708 | 1134 | 586 |
| Area Under the Curve (AUC) | 0.96 | 0.95 | 0.92 | 0.96 | 0.91 | 0.85 | 0.75 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.95 | 0.92 | 0.88 | 0.93 | 0.90 | 0.83 | 0.73 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.98 | 0.99 | 0.96 | 0.98 | 0.92 | 0.87 | 0.78 |
| Statistics | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Base Rate | 0.06 | 0.07 | 0.11 | 0.08 | 0.19 | 0.23 | 0.28 |
| Overall Classification Rate | 0.87 | 0.82 | 0.82 | 0.85 | 0.83 | 0.73 | 0.64 |
| Sensitivity | 1.00 | 1.00 | 0.91 | 0.94 | 0.85 | 0.89 | 0.85 |
| Specificity | 0.86 | 0.81 | 0.81 | 0.85 | 0.82 | 0.69 | 0.56 |
| False Positive Rate | 0.14 | 0.19 | 0.19 | 0.15 | 0.18 | 0.31 | 0.44 |
| False Negative Rate | 0.00 | 0.00 | 0.09 | 0.06 | 0.15 | 0.11 | 0.15 |
| Positive Predictive Power | 0.32 | 0.30 | 0.36 | 0.33 | 0.53 | 0.45 | 0.42 |
| Negative Predictive Power | 1.00 | 1.00 | 0.99 | 0.99 | 0.96 | 0.95 | 0.91 |
| Sample | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Date | |||||||
| Sample Size | 476 | 215 | 432 | 423 | 4048 | 2133 | 1451 |
| Geographic Representation | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (TN) | East South Central (TN) | East South Central (TN) |
| Male | 48.3% | 46.5% | 47.9% | 46.6% | |||
| Female | 51.7% | 53.5% | 52.1% | 53.4% | |||
| Other | |||||||
| Gender Unknown | |||||||
| White, Non-Hispanic | |||||||
| Black, Non-Hispanic | 19.1% | 21.9% | 23.4% | 22.9% | |||
| Hispanic | 23.9% | 23.3% | 21.5% | 20.6% | |||
| Asian/Pacific Islander | 1.3% | 0.9% | 1.2% | 2.1% | |||
| American Indian/Alaska Native | 0.2% | 1.2% | 0.5% | ||||
| Other | |||||||
| Race / Ethnicity Unknown | 55.5% | 54.0% | 52.8% | 53.9% | 100.0% | 100.0% | 100.0% |
| Low SES | |||||||
| IEP or diagnosed disability | |||||||
| English Language Learner |
Classification Accuracy - Spring
| Evidence | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Criterion measure | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Alabama Comprehensive Assessment Program - Mathematics (ACAP-Math) Grades 2-5 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 | Tennessee Comprehensive Assessment Program - Mathematics (TCAP-Math) Grades 6-8 |
| Cut Points - Percentile rank on criterion measure | 20 | 20 | 20 | 20 | 20 | 20 | 20 |
| Cut Points - Performance score on criterion measure | 473 | 478 | 476 | 477 | 294 | 285 | 274 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 178 | 181 | 203 | 215 | 217 | 216 | 212 |
| Classification Data - True Positive (a) | 28 | 15 | 44 | 27 | 705 | 554 | 515 |
| Classification Data - False Positive (b) | 59 | 8 | 34 | 28 | 664 | 708 | 690 |
| Classification Data - False Negative (c) | 2 | 3 | 6 | 4 | 53 | 63 | 102 |
| Classification Data - True Negative (d) | 392 | 194 | 363 | 370 | 2620 | 2414 | 1663 |
| Area Under the Curve (AUC) | 0.97 | 0.95 | 0.95 | 0.96 | 0.92 | 0.89 | 0.83 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.94 | 0.92 | 0.92 | 0.94 | 0.92 | 0.88 | 0.82 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.99 | 0.99 | 0.97 | 0.99 | 0.93 | 0.90 | 0.85 |
| Statistics | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Base Rate | 0.06 | 0.08 | 0.11 | 0.07 | 0.19 | 0.17 | 0.21 |
| Overall Classification Rate | 0.87 | 0.95 | 0.91 | 0.93 | 0.82 | 0.79 | 0.73 |
| Sensitivity | 0.93 | 0.83 | 0.88 | 0.87 | 0.93 | 0.90 | 0.83 |
| Specificity | 0.87 | 0.96 | 0.91 | 0.93 | 0.80 | 0.77 | 0.71 |
| False Positive Rate | 0.13 | 0.04 | 0.09 | 0.07 | 0.20 | 0.23 | 0.29 |
| False Negative Rate | 0.07 | 0.17 | 0.12 | 0.13 | 0.07 | 0.10 | 0.17 |
| Positive Predictive Power | 0.32 | 0.65 | 0.56 | 0.49 | 0.51 | 0.44 | 0.43 |
| Negative Predictive Power | 0.99 | 0.98 | 0.98 | 0.99 | 0.98 | 0.97 | 0.94 |
| Sample | Grade 2 | Grade 3 | Grade 4 | Grade 5 | Grade 6 | Grade 7 | Grade 8 |
|---|---|---|---|---|---|---|---|
| Date | |||||||
| Sample Size | 481 | 220 | 447 | 429 | 4042 | 3739 | 2970 |
| Geographic Representation | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (AL) | East South Central (TN) | East South Central (TN) | East South Central (TN) |
| Male | 48.6% | 45.9% | 47.0% | 46.6% | |||
| Female | 51.4% | 54.1% | 53.0% | 53.4% | |||
| Other | |||||||
| Gender Unknown | |||||||
| White, Non-Hispanic | |||||||
| Black, Non-Hispanic | 18.7% | 21.4% | 23.9% | 22.8% | |||
| Hispanic | 24.9% | 24.1% | 22.4% | 20.7% | |||
| Asian/Pacific Islander | 1.2% | 0.9% | 1.1% | 2.1% | |||
| American Indian/Alaska Native | 0.2% | 1.1% | 0.5% | ||||
| Other | |||||||
| Race / Ethnicity Unknown | 54.9% | 53.6% | 51.5% | 53.8% | 100.0% | 100.0% | 100.0% |
| Low SES | |||||||
| IEP or diagnosed disability | |||||||
| English Language Learner |
Reliability
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Rating |
|
|
|
|
|
|
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Offer a justification for each type of reliability reported, given the type and purpose of the tool.
- Math Composite is used to measure Math abilities as a single general construct reflecting performance on a combination of key skills students are expected to learn and develop in Grades 2-8. To test the reliability of the Math Composite, we calculated coefficient omega using a model-based approach for each grade level and each benchmark screening season (Fall, Winter, and Spring). Coefficient omega indicates how well the component measures of the Math Composite reliably measure the same general construct. This reliability approach accounts for how the Math composite score is calculated as the combination of scores from unique component measures, accounting for differences between component measure scores and their relationship to a general measure of Math ability. (Reference: McDonald, R. P. (1978). Generalizability in factorable domains: “domain validity and generalizability”: 1. Educational and Psychological Measurement, 38 (1), 75-79.)
- *Describe the sample(s), including size and characteristics, for each reliability analysis conducted.
- The data sample was comprised of students from Grades 2-8 students (n = 64,514) from 37 states. Demographic data regarding sex, ethnicity, and English as a second language (ESL) were analyzed. Student data showed that 45.8% of the population was female and 54.2% was male. Analysis of the ethnicities that comprised the sample showed the following: American-Indian (0.7%), Asian (7.2%), Black (3.8%), Hispanic (8.4%), Multi-Racial (18.7%), Pacific-Islander (0.4%), and White (60.8%). In addition, most students in the sample spoke English as their first language (88.9%), some were ESL students (0.6%) with 10.6% of the sample not reporting.
- *Describe the analysis procedures for each reported type of reliability.
- Omega coefficients were calculated for each Math grade level (Grades 2 through Grade 8) and benchmark screening season (Fall, Winter, and Spring). Analyses were conducted in R. Omega (total) coefficients were calculated using the “omega” function from the “psych” package. This method first performs a factor analysis on the scores of the Math composite’s component measures to extract a single common factor. Omega total is then calculated as the proportion of total variance in the scores attributed to the single common factor. 95% confidence intervals of the reliability coefficient were estimated through a bootstrapping approach using the “boot” package.
*In the table(s) below, report the results of the reliability analyses described above (e.g., internal consistency or inter-rater reliability coefficients).
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
- Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
If yes, fill in data for each subgroup with disaggregated reliability data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
Validity
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Rating |
|
|
|
|
|
|
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.
- Two external criterion measures were used in our validity analyses: the Math end-of-year assessment from the Tennessee Comprehensive Assessment Program (TCAP-Math) and the Math end-of-year assessment from the Alabama Comprehensive Assessment Program (ACAP-Math). ACAP was also used as our external criterion measure for concurrent and predictive validity analyses for grades 2-5. TCAP-Math was used as our external criterion measure for our concurrent and predictive validity analyses for grades 3-8. Both measures are appropriate for concurrent validity analyses focused on evaluating the construct validity of the aimswebPlus Math Composite. Both the Math composite and the two criterion measures are intended to measure math abilities aligned to grade-level state standard expectations as well as related math skills. Using both external measures for predictive validity analyses aligns with how aimswebPlus Math Composite scores are intended to provide teachers and school districts with valid indications of how students may perform on summative end of year.
- *Describe the sample(s), including size and characteristics, for each validity analysis conducted.
- Samples of students from Tennessee and Alabama were included in our validity analyses. The Tennessee sample comes from a large school district with students represented across 45 elementary schools (Grades 3-5) and 21 middle schools (Grades 6-8) in urban, suburban, and rural regions. Students of all ability levels in this district completed the Math Composite as a part of their universal screening assessments at the beginning, middle, and end of the school year. The Alabama sample comes from a school district with students represented across 4 schools (Grades 2-5) of varying sizes and locations around a medium sized suburban city. Demographic data indicate the Alabama sample was drawn from a diverse district composed of multiple ethnic and socioeconomic backgrounds. For predictive validity analyses, students in the Tennessee sample in Grades 3-8 completed Fall and Winter Math Composite measures during the 2022-2023 school year and the TCAP-Math assessment in the Spring of 2023. Students in the Alabama sample in Grades 2-5 completed Fall and Winter Math Composite measures during the 2023-2024 school year, and ACAP-Math assessments in the Spring of 2024. For concurrent validity analyses, students in the Tennessee sample in Grades 3-8 completed Math Composite measures and the TCAP-Math assessment in the Spring of 2023. Students in the Alabama sample in Grades 2-5 completed Math Composite measures and the TCAP-Math assessment in the Spring of 2024.
- *Describe the analysis procedures for each reported type of validity.
- Two types of validity analyses were conducted with Math and the external criterion measures: predictive validity and concurrent validity. Predictive Validity analyses for Grades 3-8 were conducted by examining the strength of the Pearson correlation coefficient between Math scores gathered multiple months prior to TCAP-Math scores for the same students. The predictive validity analyses for grades 3-8 correlation coefficients were calculated between scores from Math tests given in the Fall of 2022 (August-November) and TCAP-Math given in the Spring (March) of 2023. For the predictive validity analysis for Grades 2-5, correlation coefficients were calculated between scores from Math tests given in the Spring of 2021 and ACAP scores of the same students assessed at the end of Grade 3 in the Spring of 2023. Concurrent Validity analyses were conducted for Grades 3-8 by examining the strength of the Pearson correlation coefficient between Math scores and the external criterion measures given from TCAP-Math. For the concurrent validity analysis for Grades 2-5 correlation coefficients were calculated between Pearson correlation coefficient and the ACAP Math test scores (end of January to beginning of February) For predictive and concurrent validity analyses, 95% confidence intervals for the correlation coefficients using the Fischer z-transformation.
*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
- Describe the degree to which the provided data support the validity of the tool.
- Results of the concurrent and predictive validity analyses for all grade levels present strong evidence that the validity of aimswebPlus Math Composite scores. Concurrent validity analyses provide strong indications of construct validity, with positive associations between Math Composite scores and TCAP and ACAP scores designed to measure similar math abilities aligned to grade-level standards. Predictive validity results provide additional evidence that the Grade 2-Grade 8 Math Composites have strong predictive utility for the skills students are expected to demonstrate on end-of-year state summative exams. The same was observed in Grades 3-Grade 8 with strong correlations observed in Math Composites and TCAP math scores. Together, predictive and concurrent validity results provide convergent evidence that the Math Composites measure the math abilities they are designed to assess and can provide valuable predictive insights for future performance.
- Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
If yes, fill in data for each subgroup with disaggregated validity data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
Bias Analysis
| Grade |
Grade 2
|
Grade 3
|
Grade 4
|
Grade 5
|
Grade 6
|
Grade 7
|
Grade 8
|
|---|---|---|---|---|---|---|---|
| Rating | Not Provided | Not Provided | Not Provided | Not Provided | Not Provided | Not Provided | Not Provided |
- Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.
- No
- If yes,
- a. Describe the method used to determine the presence or absence of bias:
- b. Describe the subgroups for which bias analyses were conducted:
- c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.
Data Collection Practices
Most tools and programs evaluated by the NCII are branded products which have been submitted by the companies, organizations, or individuals that disseminate these products. These entities supply the textual information shown above, but not the ratings accompanying the text. NCII administrators and members of our Technical Review Committees have reviewed the content on this page, but NCII cannot guarantee that this information is free from error or reflective of recent changes to the product. Tools and programs have the opportunity to be updated annually or upon request.
