aimswebPlus Math
Early Numeracy Composite

Descriptive Information

Please provide a description of your tool:

The aimswebPlus Early Numeracy Composite is an academic screening tool which combines the scores of three component measures to provide a general indicator of early numeracy abilities aligned to grade level expectations. The Early Numeracy Composite scores are available for universal and benchmark screening four times per year starting in kindergarten and extending through high school. Specifically, students can complete testing to obtain an Early Numeracy Composite score in the beginning (fall), middle (winter), and end (Spring) of the standard school year, and may also complete Early Numeracy Composite measures during the summer months. Component measures, which combine to provide the Early Numeracy Composite score, in Kindergarten include Concepts and Applications, Number Naming Fluency, and Quantity Total Fluency. In Grade 1, the Early Numeracy Composite includes scores from Concepts and Applications, Number Naming Fluency, and Number Comparison Fluency-Pairs. These measures help to assess a wide range of math skills students are expected to be learning and demonstrating at each grade level. The component measures available for each testing window within a grade level are equivalent in design and difficulty but unique, so each time students complete the Early Numeracy Composite they are assessed with new items. Early Numeracy Composite scores are reported in the aimswebPlus web application, where teachers and administrators can track students' scores, observe growth over the school year, and receive feedback about how students are performing relative to peers and a nationally normed data.

The tool is intended for use with the following grade(s).

Preschool / Pre - kindergarten
Kindergarten
First grade
Second grade
Third grade
Fourth grade
Fifth grade
Sixth grade
Seventh grade
Eighth grade
Ninth grade
Tenth grade
Eleventh grade
Twelfth grade

The tool is intended for use with the following age(s).

0-4 years old
5 years old
6 years old
7 years old
8 years old
9 years old
10 years old
11 years old
12 years old
13 years old
14 years old
15 years old
16 years old
17 years old
18 years old

The tool is intended for use with the following student populations.

Students in general education
Students with disabilities
English language learners

ACADEMIC ONLY: What skills does the tool screen?

Reading

Phonological processing:

RAN
Memory
Awareness
Letter sound correspondence
Phonics
Structural analysis

Word ID

Accuracy
Speed

Nonword

Accuracy
Speed

Spelling

Accuracy
Speed

Passage

Accuracy
Speed

Reading comprehension:

Multiple choice questions
Cloze
Constructed Response
Retell
Maze
Sentence verification
Other (please describe):

Listening comprehension:

Multiple choice questions
Cloze
Constructed Response
Retell
Maze
Sentence verification
Vocabulary
Expressive
Receptive

Mathematics

Global Indicator of Math Competence

Accuracy
Speed
Multiple Choice
Constructed Response

Early Numeracy

Accuracy
Speed
Multiple Choice
Constructed Response

Mathematics Concepts

Accuracy
Speed
Multiple Choice
Constructed Response

Mathematics Computation

Accuracy
Speed
Multiple Choice
Constructed Response

Mathematic Application

Accuracy
Speed
Multiple Choice
Constructed Response

Fractions/Decimals

Accuracy
Speed
Multiple Choice
Constructed Response

Algebra

Accuracy
Speed
Multiple Choice
Constructed Response

Geometry

Accuracy
Speed
Multiple Choice
Constructed Response

Other (please describe):
For some measures, test items are scored by the examiner on a digital record form as the the student responds. Responses can be reviewed before submitting each measure to be scored using the aimswebPlus platform.

Please describe specific domain, skills or subtests:

BEHAVIOR ONLY: Which category of behaviors does your tool target?

Internalizing
Externalizing
Internalizing and Externalizing

BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.

Acquisition and Cost Information

Where to obtain:

Email Address

aimswebsupport@pearson.com

Address

Pearson Clinical Assessment, 927 E. Sonterra Blvd., Ste 119, San Antonio, TX 78258

Phone Number

1-866-313-6194

Website

www.pearsonassessment.com/aimswebPlus

Initial cost for implementing program:

Cost

$7.50

Unit of cost

student

Replacement cost per unit for subsequent use:

Cost

$7.50

Unit of cost

student

Duration of license

year

Additional cost information:

Describe basic pricing plan and structure of the tool. Provide information on what is included in the published tool, as well as what is not included but required for implementation.

aimswebPlus is a subscription-based online solution that includes digital editions of training manuals and testing materials within the application. The per-student cost of $7.50 for one year grants access to all measures (Reading and Math). The per-student cost for one year's access to only Math is $4.25. An aimswebPlus Unlimited Subscription is available for districts with enrollment of 2,500 students or fewer. It includes all aimswebPlus measures (Reading and Math) and these supplemental measures: Shaywitz DyslexiaScreen, BASC-3 BESS Teacher and Student Forms, WriteToLearn, and RAN Objects, Colors and Shapes. The cost for one year is $4995.00

Provide information about special accommodations for students with disabilities.

In general, accommodations consistent with IEPs and 504 plans are permitted with aimswebPlus, but modifications which bias or conflict with a measure's scoring logic are not recommended. Unique exceptions permitting modifications depend on the nature of the modification, and if the necessity and benefit to testing goals significantly outweigh any consequence to scoring. Extensive guidance on test accommodations, modifications, and their appropriate use is provided in the aimswebPlus Manual and other training materials.

Administration

BEHAVIOR ONLY: What type of administrator is your tool designed for?

General education teacher
Special education teacher
Parent
Child
External observer
Other

If other, please specify:

What is the administration setting?

Direct observation
Rating scale
Checklist
Performance measure
Questionnaire
Direct: Computerized
One-to-one
Other

If other, please specify:

Does the tool require technology?

Yes

If yes, what technology is required to implement your tool? (Select all that apply)

Computer or tablet
Internet connection
Other technology (please specify)

If your program requires additional technology not listed above, please describe the required technology and the extent to which it is combined with teacher small-group instruction/intervention:

What is the administration context?

Individual
Small group   If small group, n=
Large group   If large group, n=
Computer-administered
Other

If other, please specify:

What is the administration time?

Time in minutes

20

per (student/group/other unit)

student

Additional scoring time:

Time in minutes

3

per (student/group/other unit)

student

ACADEMIC ONLY: What are the discontinue rules?

No discontinue rules provided
Basals
Ceilings
Other

If other, please specify:

Are norms available?

Yes

Are benchmarks available?

Yes

If yes, how many benchmarks per year?

4

If yes, for which months are benchmarks available?

Fall (August-November), Winter (December - March 15), Spring (March 16 - May), Summer (June-July).

BEHAVIOR ONLY: Can students be rated concurrently by one administrator?

If yes, how many students can be rated concurrently?

Training & Scoring

Training

Is training for the administrator required?

Yes

Describe the time required for administrator training, if applicable:

Less than one hour of administrator training is required.

Please describe the minimum qualifications an administrator must possess.

Administrators may be paraprofessional or professional members of the educational staff. All test administrators must understand the administration and scoring guidelines for all measures.

No minimum qualifications

Are training manuals and materials available?

Yes

Are training manuals/materials field-tested?

No

Are training manuals/materials included in cost of tools?

Yes

If No, please describe training costs:

Can users obtain ongoing professional and technical support?

Yes

If Yes, please describe how users can obtain support:

Pearson provides an extensive online Help database and offers both phone- and email-based support. A customer forum facilitates asking and answering questions, and additional on-site, virtual, and on-demand training may be purchased.

Scoring

How are scores calculated?

Manually (by hand)
Automatically (computer-scored)
Other

If other, please specify:

Test items are scored by the examiner on a digital record form as the student responds. Responses can be reviewed before submitting each measure to be scored using the aimswebPlus platform.

Do you provide basis for calculating performance level scores?

Yes

What is the basis for calculating performance level and percentile scores?

Age norms
Grade norms
Classwide norms
Schoolwide norms
Stanines
Normal curve equivalents

What types of performance level scores are available?

Raw score
Standard score
Percentile score
Grade equivalents
IRT-based score
Age equivalents
Stanines
Normal curve equivalents
Developmental benchmarks
Developmental cut points
Equated
Probability
Lexile score
Error analysis
Composite scores
Subscale/subtest scores
Other

If other, please specify:

Does your tool include decision rules?

No

If yes, please describe.

Can you provide evidence in support of multiple decision rules?

No

If yes, please describe.

Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.

The aimswebPlus Early Numeracy composite is calculated by summing the scores of two or more measures. The composite for kindergarten students is formed using scores from Number Naming Fluency, Quantity Total Fluency, and Concepts & Applications; specifically, EN = (1/3*NNF) + QTF + CA. For students in Grade 1, the composite is formed by the sum of Number Comparison Fluency–Pairs, Math Facts Fluency–1 Digit, and Concepts & Applications. Because some of these measures assess fluency, each student's skill competency will determine how many items are answered within the time limit. Number Naming Fluency measures a student's ability to recognize and name numbers from 0 through 20 quickly and accurately. The student sees rows of numbers, then points to and names individual numbers within a 1-minute time limit. Depending on the student's ability, they may answer as many as 80 items. A student's raw score is based on 1 point for each correct answer within the time limit. Quantity Total Fluency measures a student's ability to count objects in sets of up to 10 quickly and accurately. The student sees boxes or pairs of boxes containing blue dots, then states the total number of dots in each box or pair of boxes within a 1-minute time limit. Depending on the student's ability, they may answer as many as 38 items. A student's raw score is based on 1 point for each correct answer within the time limit. Concepts & Applications measures a student's understanding of essential math concepts and their ability to solve math problems. Unless the discontinue rule is met (i.e., the student incorrectly answers the first 5 items), students are encouraged to attempt all 25 items. A student's raw score is based on 1 point for each correct answer. Number Comparison Fluency–Pairs measures a student's ability to compare the magnitude of two numbers quickly and accurately. The student sees rows of number pairs, then points to and names the larger number in each pair within a 1-minute time limit. Depending on the student's ability, they may answer as many as 50 items. A student's raw score is based on 1 point for each correct answer within the time limit. Math Facts Fluency–1 Digit measures a student's ability to mentally add and subtract numbers from 0 through 10 quickly and accurately. The student responds orally to as many addition and subtraction problems as possible within the 1-minute time limit. Depending on the student's ability, they may answer as many as 40 items. A student's raw score is based on 1 point for each correct answer within the time limit. Combining the scores of fluency and standards-based measures of essential early math skills into a composite that can be tracked over time presents a comprehensive and efficient evaluation of a student's on-grade-level performance.

Describe the tool’s approach to screening, samples (if applicable), and/or test format, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.

aimswebPlus was designed to be an essential part of a school's strategy to meet state and federal accountability requirements and to document the academic performance data for students who have wide-ranging ability levels. In the fall, winter, spring, and summer, the Early Numeracy benchmark screening measures provide data to identify students' academic strengths and weaknesses. When this benchmark screening reveals gaps among students, aimswebPlus provides information to help education teams decide next steps. Score reports can display student performance by subdomain or item-level error analysis. These reports help identify specific areas of academic concern and may reveal patterns for creating instructional groups of students with similar learning needs. The individually administered measures given to obtain an Early Numeracy composite include tasks that require administrator observation, judgment, and support (e.g., teaching new tasks and applying discontinue rules). To facilitate this, aimswebPlus uses a computer- and tablet-compatible digital record form (DRF) to make it easy to capture and score student responses. The DRF includes all administration instructions; the words said aloud to the student, and practice items; the test items and method for capturing correct/incorrect responses; and other tools (e.g., a timer) needed for accurate recording and scoring. Once the DRF with the student's responses are submitted to the aimswebPlus system, a score for each measure is generated automatically. Student materials use colorful art, grade-appropriate fonts, and other design features to engage young test-takers. Printed stimulus books are required that include practice and test pages, response booklets, or other materials specified in the directions for administering and scoring each measure. One printed stimulus book may be reused to test many students. All student materials are available as downloadable PDFs and are included in the per-student cost of aimswebPlus. To support assessment with diverse populations, steps were taken during the development of each measure, beginning with the review of content by qualified experts to minimize bias and to ensure the appropriate skill coverage by age/grade. Test instructions are intentionally brief and rely on simple, grade-appropriate language. Pilot studies included students with diverse skill levels and backgrounds, and psychometric analyses were performed to show that items and measures are fair and free from bias. To ensure an equitable testing experience for all students, guidance is offered to establish rapport and to address challenges for students related to language or physical differences. For students who speak Spanish, the administration instructions for the measures that make up the Early Numeracy composite may be given in either Spanish or English. Materials and scoring guidelines for students who speak Spanish are included in the per-student cost of aimswebPlus. aimswebPlus also offers general guidance for when it's appropriate to give test items in Spanish. When allowed by a student’s IEP accommodations, adapting the physical environment is permitted.

Classification Accuracy & Cross-Validation Summary

Grade	Kindergarten	Grade 1
Classification Accuracy Fall
Classification Accuracy Winter
Classification Accuracy Spring

Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)

Classification Accuracy

Select time of year

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Alabama Comprehensive Assessment Program – Mathematics (ACAP-Math) was used as the criterion measure for grades K-1. ACAP-Math is a state-summative assessment measuring a combination of math abilities aligned to grade-level specific state standards for math achievement. This summative assessment is used to indicate a student's end of year proficiency in math and identify students' performing below expectations who may need intensive intervention. The ACAP-Math has no direct relation to the aimswebPlus Early Numeracy Composite. The ACAP-Math was developed separately using data from unique samples of students and published by an organization that is separate from the aimswebPlus research and development team.

Do the classification accuracy analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

aimswebPlus Kindergarten and Grade 1 Early Numeracy Composite screening measures were administered three times during the standard school year, in the Fall, Winter, and Spring. The ACAP-Math was administered 1-2 years later as an end of year summative exam to the same students in Grade 2 in the Spring. Therefore, all classification accuracy analyses used a predictive method on Fall, Winter, and Spring Early Numeracy Composite scores predicting risk of students in need of intensive interventions by the end of the school year based on outcomes of ACAP-Math testing.

Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Cut points on the ACAP-Math were chosen to identify students with intensive needs in accordance with the definition provided by the NCII’s Technical Review Committee. Specifically, cut points were determined separately for each grade level as the ACAP-Math scale score associated the 20th percentile of performance. Students with external criterion scores below the 20th percentile were labeled as “at-risk”; students with external criterion scores equal to or greater than the 20th percentile were labeled as “not at-risk” The cut points on the aimswebPlus Early Numeracy Composite score scale were determined by identifying the scores that maximized the sum of sensitivity and specificity with the classification outcomes of the ACAP-Math. Students with Early Numeracy Composite scores below the cut point were classified as “at-risk”; students with Early Numeracy Composite scores at or above the cut point were classified as “not at-risk”. Lastly, classification indices were calculated using the formulates provided by the NCII Classification worksheet.

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

Yes

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

We did not receive data specific to the types of interventions students were receiving in addition to standard classroom instruction. However, verbal reports from the school district we partnered with for analyses with ACAP-Math data lead us to conclude it is likely students scoring below the 20th percentile received some type of intervention.

Cross-Validation

Has a cross-validation study been conducted?

No

If yes,

Select time of year.

Fall

Winter

Spring

Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.

Do the cross-validation analyses examine concurrent and/or predictive classification?

Concurrent
Predictive

Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.

Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).

Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?

If yes, please describe the intervention, what children received the intervention, and how they were chosen.

---

Classification Accuracy - Fall

Evidence	Kindergarten	Grade 1
Criterion measure	Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)	Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)
Cut Points - Percentile rank on criterion measure	20	20
Cut Points - Performance score on criterion measure	473	473
Cut Points - Corresponding performance score (numeric) on screener measure	28	36
Classification Data - True Positive (a)	15	21
Classification Data - False Positive (b)	122	106
Classification Data - False Negative (c)	0	0
Classification Data - True Negative (d)	223	297
Area Under the Curve (AUC)	0.84	0.92
AUC Estimate’s 95% Confidence Interval: Lower Bound	0.77	0.88
AUC Estimate’s 95% Confidence Interval: Upper Bound	0.91	0.96

Statistics	Kindergarten	Grade 1
Base Rate	0.04	0.05
Overall Classification Rate	0.66	0.75
Sensitivity	1.00	1.00
Specificity	0.65	0.74
False Positive Rate	0.35	0.26
False Negative Rate	0.00	0.00
Positive Predictive Power	0.11	0.17
Negative Predictive Power	1.00	1.00

Sample	Kindergarten	Grade 1
Date
Sample Size	360	424
Geographic Representation	East South Central (AL)	East South Central (AL)
Male	48.9%	49.5%
Female	50.6%	50.5%
Other
Gender Unknown
White, Non-Hispanic
Black, Non-Hispanic	17.2%	17.9%
Hispanic	21.7%	21.9%
Asian/Pacific Islander	1.4%	1.4%
American Indian/Alaska Native	0.3%	0.2%
Other	58.9%	58.5%
Race / Ethnicity Unknown
Low SES
IEP or diagnosed disability
English Language Learner

Classification Accuracy - Winter

Evidence	Kindergarten	Grade 1
Criterion measure	Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)	Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)
Cut Points - Percentile rank on criterion measure	20	20
Cut Points - Performance score on criterion measure	473	473
Cut Points - Corresponding performance score (numeric) on screener measure	39	49
Classification Data - True Positive (a)	17	20
Classification Data - False Positive (b)	89	59
Classification Data - False Negative (c)	0	1
Classification Data - True Negative (d)	261	347
Area Under the Curve (AUC)	0.92	0.94
AUC Estimate’s 95% Confidence Interval: Lower Bound	0.88	0.90
AUC Estimate’s 95% Confidence Interval: Upper Bound	0.96	0.98

Statistics	Kindergarten	Grade 1
Base Rate	0.05	0.05
Overall Classification Rate	0.76	0.86
Sensitivity	1.00	0.95
Specificity	0.75	0.85
False Positive Rate	0.25	0.15
False Negative Rate	0.00	0.05
Positive Predictive Power	0.16	0.25
Negative Predictive Power	1.00	1.00

Sample	Kindergarten	Grade 1
Date
Sample Size	367	427
Geographic Representation	East South Central (AL)	East South Central (AL)
Male	48.8%	48.9%
Female	50.7%	51.1%
Other
Gender Unknown
White, Non-Hispanic
Black, Non-Hispanic	16.9%	17.8%
Hispanic	21.8%	22.2%
Asian/Pacific Islander	1.4%	1.4%
American Indian/Alaska Native	0.3%	0.2%
Other	59.1%	58.3%
Race / Ethnicity Unknown
Low SES
IEP or diagnosed disability
English Language Learner

Classification Accuracy - Spring

Evidence	Kindergarten	Grade 1
Criterion measure	Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)	Alabama Comprehension Assessment Program - Mathematics (ACAP-Math)
Cut Points - Percentile rank on criterion measure	20	20
Cut Points - Performance score on criterion measure	473	473
Cut Points - Corresponding performance score (numeric) on screener measure	49	51
Classification Data - True Positive (a)	17	21
Classification Data - False Positive (b)	68	42
Classification Data - False Negative (c)	0	3
Classification Data - True Negative (d)	281	370
Area Under the Curve (AUC)	0.95	0.93
AUC Estimate’s 95% Confidence Interval: Lower Bound	0.92	0.90
AUC Estimate’s 95% Confidence Interval: Upper Bound	0.98	0.97

Statistics	Kindergarten	Grade 1
Base Rate	0.05	0.06
Overall Classification Rate	0.81	0.90
Sensitivity	1.00	0.88
Specificity	0.81	0.90
False Positive Rate	0.19	0.10
False Negative Rate	0.00	0.13
Positive Predictive Power	0.20	0.33
Negative Predictive Power	1.00	0.99

Sample	Kindergarten	Grade 1
Date
Sample Size	366	436
Geographic Representation	East South Central (AL)	East South Central (AL)
Male	48.9%	49.3%
Female	50.8%	50.7%
Other
Gender Unknown
White, Non-Hispanic
Black, Non-Hispanic	16.7%	18.1%
Hispanic	22.1%	22.9%
Asian/Pacific Islander	1.4%	1.4%
American Indian/Alaska Native	0.3%
Other	59.3%	56.9%
Race / Ethnicity Unknown		0.7%
Low SES
IEP or diagnosed disability
English Language Learner

Reliability

Grade	Kindergarten	Grade 1
Rating

*Offer a justification for each type of reliability reported, given the type and purpose of the tool.

Early Numeracy Composite is used to measure early numeracy abilities as a single general construct reflecting performance on a combination of key skills students are expected to learn and develop in Kindergarten and Grade 1. To test the reliability of the Early Numeracy Composite, we calculated coefficient omega using a model-based approach for each grade level and each benchmark screening season (Fall, Winter, and Spring). Coefficient omega indicates how well the component measures of the Early Numeracy Composite reliably measure the same general construct. This reliability approach accounts for how the Early Numeracy composite score is calculated as the combination of scores from unique component measures. (Reference: McDonald, R. P. (1978). Generalizability in factorable domains: “domain validity and generalizability”: 1. Educational and Psychological Measurement, 38 (1), 75-79.)

*Describe the sample(s), including size and characteristics, for each reliability analysis conducted.

The data sample was comprised of Kindergarten and Grade 1 students (n = 58,950) from 41 states. Demographic data regarding sex, ethnicity, and English as a second language (ESL) were analyzed. Student data showed that 49.1% of the population was female and 50.9% was male. Analysis of the ethnicities that comprised the sample showed the following: American-Indian (1.6%), Asian (4.6%), Black (6.9%), Hispanic (9.4%), Multi-Racial (17.0%), Pacific-Islander (0.9%), and White (59.8%). In addition, most students in the sample spoke English as their first language (81.8%), some were ESL students (1.4%) with 16.8% of the sample not reporting.

*Describe the analysis procedures for each reported type of reliability.

Omega coefficients were calculated for each Early Numeracy grade level (Kindergarten and Grade 1) and benchmark screening season (Fall, Winter, and Spring). Analyses were conducted in R. Omega (total) coefficients were calculated using the “omega” function from the “psych” package. This method first performs a factor analysis on the scores of the Early Numeracy composite’s component measures to extract a single common factor. Omega total is then calculated as the proportion of total variance in the scores attributed to the single common factor. 95% confidence intervals of the reliability coefficient were estimated through a bootstrapping approach using the “boot” package.

*In the table(s) below, report the results of the reliability analyses described above (e.g., internal consistency or inter-rater reliability coefficients).

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

Provide citations for additional published studies.

Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?

If yes, fill in data for each subgroup with disaggregated reliability data.

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of reliability analysis not compatible with above table format:

Manual cites other published reliability studies:

Provide citations for additional published studies.

Validity

Grade	Kindergarten	Grade 1
Rating

*Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.

Two external criterion measures were used in our validity analyses. The NWEA Map Growth Mathematics (MAP Growth Math) was used for concurrent validity analyses for Kindergarten and Grade 1 and the mathematics state summative section of the Alabama Comprehensive Assessment Program (ACAP-Math) was used for predictive validity analyses. Concurrent validity analyses provide evidence for construct validity, and MAP Growth Math is an appropriate external criterion measure because both the aimswebPlus Early Numeracy (EN) Composite and MAP Growth Math are assessment tools designed to measure a variety of math and numeracy skills aligned to academic state standards for each grade level. Additionally, this external measure may be given multiple times in a school year (Fall, Winter, & Spring). Predictive Validity analyses assessed the strength of how well the screening of early numeracy and math abilities assessed by the EN Composite can predict students’ future mathematical abilities at the end of Grade 2. ACAP Math is first administered to students in Alabama at the end of Grade 2, and it measures all essential state standard aligned math skills students are expected to learn in each grade. Similar to the EN Composite, ACAP Math overall scale scores provide a general indication of a student’s math ability. The Grade 2 ACAP Math assessment provides an appropriate external criterion measure for predictive validity analyses evaluating how well EN Composite scores in Kindergarten and Grade 1 can predict overall math proficiency 1-2.5 years later.

*Describe the sample(s), including size and characteristics, for each validity analysis conducted.

Concurrent and Predictive validity analyses were conducted on two separate samples of students. Concurrent validity analyses with NWEA Map Growth Math scores (MAP Growth Math) included students from multiple elementary schools of varying sizes and locations around a medium sized city in Illinois. Demographic data indicates the students in the sample were drawn from a school district with diverse families representing multiple ethnic and socioeconomic backgrounds. Students of all ability levels in this sample completed the EN Composite measures as a part of their universal screening assessments at the beginning (fall), middle (winter), and end (spring) of the school year. Concurrent validity analyses for Kindergarten and Grade 1 included all students in the sample who had completed the EN Composite and NWEA Math in the Spring of 2023 around the same time (within 2 months). Predictive validity analyses using ACAP Math scores included students from two elementary schools in a suburban city in Alabama. Students of all ability levels completed the EN Composite measures in Kindergarten and Grade 1 as a part of their district’s universal screening assessments at the beginning (fall), middle (winter), and end (spring) of the school year. Predictive validity analyses for the Kindergarten EN Composite included all students with a valid Grade 2 ACAP Math score (Spring 2024) who had completed Kindergarten EN Composite testing 2 years prior in the fall, winter, and spring of the 2021-2022 school year. Analyses for the Grade 1 EN Composite included all students with a valid Grade 2 ACAP Math score (Spring 2024) who had completed Grade 1 EN Composite testing 1 year prior in the fall, winter, and spring of the 2022-2023 school year.

*Describe the analysis procedures for each reported type of validity.

Two types of validity analyses were conducted with Early Numeracy (EN) Composite scores and the external criterion measures: predictive validity and concurrent validity. Predictive Validity analyses for Kindergarten and Grade 1 examined the strength of the Pearson correlation coefficient between ACAP-Math scores collected in Spring 2024 of Grade 2 and EN Composite scores from the same students collected in Fall, Winter, and Spring of Kindergarten (2 years prior) and Grade 1 (1 year prior). Concurrent Validity analyses for Kindergarten and Grade 1 examined the strength of the Pearson correlation coefficient between EN Composite scores and MAP Growth Math scores collected in the Spring of 2024. For predictive and concurrent validity analyses, 95% confidence intervals for the correlation coefficients were calculated using the Fischer z-transformation.

*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of validity analysis not compatible with above table format:

Manual cites other published reliability studies:

Provide citations for additional published studies.

Describe the degree to which the provided data support the validity of the tool.

The provided data show strong evidence that the aimswebPlus Early Numeracy (EN) Composite scores provide a valid indication of Kindergarten and Grade 1 students’ emerging math abilities. Predictive validity results provide strong and compelling evidence that the Kindergarten and Grade 1 EN Composites measure early numeracy abilities that are foundational for future proficiency in Mathematics. Specifically, Kindergarten EN scores show a moderate-to-strong positive association with ACAP summative state test results in mathematics observed 2-2.5 years later in Grade 2. Even stronger correlations were observed in Grade 1 EN Composites predicting Grade 2 ACAP math score observed 1-1.5 years later. Concurrent validity results indicate that EN Composite scores show a moderate-to-strong positive relationship with similar assessment tools like MAP Growth Math, which measure the same general early numeracy and math skills. Confidence intervals around the observed concurrent validity correlation coefficients, especially for Kindergarten results, were relatively broad. This may reflect the smaller size of the student sample available for analysis among other plausible sources of variability present when testing young students. Together, predictive and concurrent validity results provide convergent evidence that the EN Composites measure the foundational math abilities they are designed to assess and can provide valuable predictive insights for future performance in math.

Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?

If yes, fill in data for each subgroup with disaggregated validity data.

Type of	Subgroup	Informant	Age / Grade	Test or Criterion	n	Median Coefficient	95% Confidence Interval Lower Bound	95% Confidence Interval Upper Bound

Results from other forms of validity analysis not compatible with above table format:

Manual cites other published reliability studies:

Provide citations for additional published studies.

Bias Analysis

Grade	Kindergarten	Grade 1
Rating	Not Provided	Not Provided

Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.

No

If yes,

a. Describe the method used to determine the presence or absence of bias:

b. Describe the subgroups for which bias analyses were conducted:

c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.