Early Learning Quick Assessments
Kindergarten Literacy
Summary
Early Learning Quick Assessments (ELQA) are a series of quick assessments that monitor progress in early literacy during kindergarten.
- Where to Obtain:
- University of Oklahoma Center for Early Childhood Professional Development
- elqa@ou.edu
- 1801 N. MOORE AVENUE MOORE, OK 73160
- 844-349-5519
- www.elqa.ou.edu
- Initial Cost:
- $195.00 per classroom
- Replacement Cost:
- $195.00 per classroom per year
- Included in Cost:
- $195 per classroom per year. Bulk rate: $180.00 per classroom, reached with 26 classrooms or more.
- Assessment administrators should ensure all learners are provided with the basic accommodations to which they are entitled during instruction and testing: comfortable seating/ventilation, adequate lighting, and minimized distraction. We recommend that the classroom teacher administer the ELQA to each child. Classroom teachers know the children in their classroom well and will most likely obtain accurate results. Accommodations for Students with Disabilities: The ELQA assessments are compatible with relevant Setting, Time/Schedule, Response, and Presentation Approved Accommodations outlined in the Oklahoma State Department of Education Special Education Services Accommodations Guide (2014). Setting: All ELQA are administered individually to each child (S1, S2). The ELQA are computer-based; the child can be seated anywhere there is access to a computer device, including in an adaptive environment, a separate location, or even a remote location (S3, S4, S6). The computer screen may be brightened or dimmed to accommodate the student, as needed (S5). Time/Schedule: The ELQA assessments are administered individually to each child, thus the time at which each child is assessed is flexible (T1). All ELQA assessments are made up of sub-assessments, allowing for administration in several “mini-sessions” (T2). For most children, ELQA assessments can be completed in approximately 30 minutes. However, the ELQA are not timed assessments. Breaks can be built-in between each sub-assessment, as needed (T3). Response: The teacher (or other assessment administrator) records all student responses on the computer as the assessments are being given, eliminating the need for a student answer sheet (R1, R2, R4). The ELQA is a computer-based assessment and the teacher controls what a student sees on the screen; students answer questions by pointing or speaking (R3). Presentation: The computer screen may be maximized to enlarge the print and pictures the child sees; volume, brightness, and the child’s proximity to the screen are all adjustable (P1, P3). The assessment administrator reads all questions to the student (P4, P13). Practice items for sub-assessments allow students opportunity to understand what they will be doing before the assessment begins (P6). The child does not use an answer form and only one question at a time will be visible on the screen (P10, P11, P14). Paper & pencil versions of the ELQA assessments are available, by request (P16). Accommodations for Students with Limited English Proficiency: For students whose primary language is not English, assessment directions may be provided in the child’s primary language by an assessment administrator who is fluent in the language. For students whose primary language is Spanish and who use dual language supports in the classroom, the Dual Language ELQA assessments may be appropriate. The Dual Language ELQA assessments provide translation of the assessment directions, questions, and acceptable answers on the computer screen. The assessor can indicate whether the question was asked in English or Spanish and whether the child answered in English or Spanish.
- Training Requirements:
- Training not required
- Qualified Administrators:
- No minimum qualifications specified.
- Access to Technical Support:
- On the ELQA website or use the ELQA Support desk by emailing elqa@ou.edu or using the toll-free number: 844-349-5519
- Assessment Format:
-
- Scoring Time:
-
- Scoring is automatic
- Scores Generated:
-
- Raw score
- Percentile score
- Developmental benchmarks
- Developmental cut points
- Composite scores
- Subscale/subtest scores
- Administration Time:
-
- 30 minutes per student
- Scoring Method:
-
- Automatically (computer-scored)
- Technology Requirements:
-
- Computer or tablet
- Internet connection
- Other technology : A single children's book with at least 2 lines of text on any page for the print concepts subtest.
- Accommodations:
- Assessment administrators should ensure all learners are provided with the basic accommodations to which they are entitled during instruction and testing: comfortable seating/ventilation, adequate lighting, and minimized distraction. We recommend that the classroom teacher administer the ELQA to each child. Classroom teachers know the children in their classroom well and will most likely obtain accurate results. Accommodations for Students with Disabilities: The ELQA assessments are compatible with relevant Setting, Time/Schedule, Response, and Presentation Approved Accommodations outlined in the Oklahoma State Department of Education Special Education Services Accommodations Guide (2014). Setting: All ELQA are administered individually to each child (S1, S2). The ELQA are computer-based; the child can be seated anywhere there is access to a computer device, including in an adaptive environment, a separate location, or even a remote location (S3, S4, S6). The computer screen may be brightened or dimmed to accommodate the student, as needed (S5). Time/Schedule: The ELQA assessments are administered individually to each child, thus the time at which each child is assessed is flexible (T1). All ELQA assessments are made up of sub-assessments, allowing for administration in several “mini-sessions” (T2). For most children, ELQA assessments can be completed in approximately 30 minutes. However, the ELQA are not timed assessments. Breaks can be built-in between each sub-assessment, as needed (T3). Response: The teacher (or other assessment administrator) records all student responses on the computer as the assessments are being given, eliminating the need for a student answer sheet (R1, R2, R4). The ELQA is a computer-based assessment and the teacher controls what a student sees on the screen; students answer questions by pointing or speaking (R3). Presentation: The computer screen may be maximized to enlarge the print and pictures the child sees; volume, brightness, and the child’s proximity to the screen are all adjustable (P1, P3). The assessment administrator reads all questions to the student (P4, P13). Practice items for sub-assessments allow students opportunity to understand what they will be doing before the assessment begins (P6). The child does not use an answer form and only one question at a time will be visible on the screen (P10, P11, P14). Paper & pencil versions of the ELQA assessments are available, by request (P16). Accommodations for Students with Limited English Proficiency: For students whose primary language is not English, assessment directions may be provided in the child’s primary language by an assessment administrator who is fluent in the language. For students whose primary language is Spanish and who use dual language supports in the classroom, the Dual Language ELQA assessments may be appropriate. The Dual Language ELQA assessments provide translation of the assessment directions, questions, and acceptable answers on the computer screen. The assessor can indicate whether the question was asked in English or Spanish and whether the child answered in English or Spanish.
Descriptive Information
- Please provide a description of your tool:
- Early Learning Quick Assessments (ELQA) are a series of quick assessments that monitor progress in early literacy during kindergarten.
ACADEMIC ONLY: What skills does the tool screen?
- Please describe specific domain, skills or subtests:
- BEHAVIOR ONLY: Which category of behaviors does your tool target?
-
- BEHAVIOR ONLY: Please identify which broad domain(s)/construct(s) are measured by your tool and define each sub-domain or sub-construct.
Acquisition and Cost Information
Administration
- Are norms available?
- Yes
- Are benchmarks available?
- Yes
- If yes, how many benchmarks per year?
- 5
- If yes, for which months are benchmarks available?
- August/September, November, January, March, May/June.
- BEHAVIOR ONLY: Can students be rated concurrently by one administrator?
- If yes, how many students can be rated concurrently?
Training & Scoring
Training
- Is training for the administrator required?
- No
- Describe the time required for administrator training, if applicable:
- Optional trainings are available and are 3 hours. Administrators can utilize the teacher's guide and user's manual to conduct the assessment without formal training.
- Please describe the minimum qualifications an administrator must possess.
-
No minimum qualifications
- Are training manuals and materials available?
- Yes
- Are training manuals/materials field-tested?
- Yes
- Are training manuals/materials included in cost of tools?
- Yes
- If No, please describe training costs:
- Can users obtain ongoing professional and technical support?
- Yes
- If Yes, please describe how users can obtain support:
- On the ELQA website or use the ELQA Support desk by emailing elqa@ou.edu or using the toll-free number: 844-349-5519
Scoring
- Do you provide basis for calculating performance level scores?
-
Yes
- Does your tool include decision rules?
-
Yes
- If yes, please describe.
- Decisions for RTI are given via cut-off scores.
- Can you provide evidence in support of multiple decision rules?
-
Yes
- If yes, please describe.
- There are cut-offs as well as very low benchmarks (10%) for helping to decide for tier 3 intervention.
- Please describe the scoring structure. Provide relevant details such as the scoring format, the number of items overall, the number of items per subscale, what the cluster/composite score comprises, and how raw scores are calculated.
- There are 13 subtests with 176 items. The 13 subtests and their number of items are: Alliteration: 10 items; Comprehension: 8 items; Expressive Vocabulary: 15 items; Fluency: 10 items; Letter Sounds: 26 items; Lowercase Alphabet: 26 items; Uppercase Alphabet: 26 items; Phoneme Deletion and Substitution: 10 items; Phoneme Blending and Segmenting: 10 items; Phonics: 10 items; Print Concepts: 10 items; Rhyming: 5 items; Syllable Segmentation: 10 items. Our cluster score comprises a grand mean of all subtests, with each of the 13 subtests weighted equally. Raw scores are the number of items correct out of each subtest.
- Describe the tool’s approach to screening, samples (if applicable), and/or test format, including steps taken to ensure that it is appropriate for use with culturally and linguistically diverse populations and students with disabilities.
- The tool is designed to be given by the teacher, one-on-one with the child. The teacher follows on-screen instructions and there is a Spanish version available for children who speak Spanish. We have conducted bias analyses to ensure questions are appropriate for use with a variety of students with diverse backgrounds.
Technical Standards
Classification Accuracy & Cross-Validation Summary
| Grade |
Kindergarten
|
|---|---|
| Classification Accuracy Fall |
|
| Classification Accuracy Winter |
|
| Classification Accuracy Spring |
|
Legend
Convincing evidencePartially convincing evidence
Unconvincing evidenceData unavailabledDisaggregated data available
Oklahoma State Testing Program's English Language Arts Assessment
Classification Accuracy
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- The outcome measure was performance on the Oklahoma State Testing Program's English Language Arts Assessment taken by all 3rd graders in Oklahoma. It is entirely independent of the ELQA.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the classification analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- The cut-point scores were derived from a longitudinal study of 729 Oklahoma students from kindergarten (2017) to 3rd grade and their middle of year (time 3) ELQA-K Literacy score and OSTP ELA scores in the spring of their 3rd grade year. This analysis was conducted in early 2023 using data acquired from the Oklahoma State Department of Education (OSDE). Children who performed above and below the 20th percentile on the OSTP ELA were contrasted.
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
-
No
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Cross-Validation
- Has a cross-validation study been conducted?
-
No
- If yes,
- Describe the criterion (outcome) measure(s) including the degree to which it/they is/are independent from the screening measure.
- Describe when screening and criterion measures were administered and provide a justification for why the method(s) you chose (concurrent and/or predictive) is/are appropriate for your tool.
- Describe how the cross-validation analyses were performed and cut-points determined. Describe how the cut points align with students at-risk. Please indicate which groups were contrasted in your analyses (e.g., low risk students versus high risk students, low risk students versus moderate risk students).
- Were the children in the study/studies involved in an intervention in addition to typical classroom instruction between the screening measure and outcome assessment?
- If yes, please describe the intervention, what children received the intervention, and how they were chosen.
Classification Accuracy - Winter
| Evidence | Kindergarten |
|---|---|
| Criterion measure | Oklahoma State Testing Program's English Language Arts Assessment |
| Cut Points - Percentile rank on criterion measure | 20 |
| Cut Points - Performance score on criterion measure | 253 |
| Cut Points - Corresponding performance score (numeric) on screener measure | 75.95 |
| Classification Data - True Positive (a) | 76 |
| Classification Data - False Positive (b) | 133 |
| Classification Data - False Negative (c) | 25 |
| Classification Data - True Negative (d) | 540 |
| Area Under the Curve (AUC) | 0.86 |
| AUC Estimate’s 95% Confidence Interval: Lower Bound | 0.82 |
| AUC Estimate’s 95% Confidence Interval: Upper Bound | 0.89 |
| Statistics | Kindergarten |
|---|---|
| Base Rate | 0.13 |
| Overall Classification Rate | 0.80 |
| Sensitivity | 0.75 |
| Specificity | 0.80 |
| False Positive Rate | 0.20 |
| False Negative Rate | 0.25 |
| Positive Predictive Power | 0.36 |
| Negative Predictive Power | 0.96 |
| Sample | Kindergarten |
|---|---|
| Date | 2017 - 2022 |
| Sample Size | 774 |
| Geographic Representation | West South Central (OK) |
| Male | 49.5% |
| Female | 44.7% |
| Other | |
| Gender Unknown | |
| White, Non-Hispanic | 58.7% |
| Black, Non-Hispanic | 2.2% |
| Hispanic | 11.6% |
| Asian/Pacific Islander | 1.3% |
| American Indian/Alaska Native | 11.4% |
| Other | 8.7% |
| Race / Ethnicity Unknown | |
| Low SES | 47.4% |
| IEP or diagnosed disability | 18.9% |
| English Language Learner | 6.6% |
Reliability
| Grade |
Kindergarten
|
|---|---|
| Rating |
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Offer a justification for each type of reliability reported, given the type and purpose of the tool.
- Inter-Rater Reliability – the consistency with which different raters score the same responses; when the test results produced by two independent raters or scorers are compared.
- *Describe the sample(s), including size and characteristics, for each reliability analysis conducted.
- The sample for the inter-rater reliability was 55 Oklahoma kindergarteners at 2 sites, one a private K-12 school and another a public school. See page 9 in the technical manual for more information.
- *Describe the analysis procedures for each reported type of reliability.
- For inter-rater reliability, a trained research staff member administered the ELQA-K Literacy and recorded the answers privately as another trained staff member observed and independently scored the test. Accuracy and Cohen’s Kappa were calculated using Classical Test Theory reliability analysis. See technical manual pg. 8 for more information.
*In the table(s) below, report the results of the reliability analyses described above (e.g., internal consistency or inter-rater reliability coefficients).
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
- Do you have reliability data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
- No
If yes, fill in data for each subgroup with disaggregated reliability data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of reliability analysis not compatible with above table format:
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
Validity
| Grade |
Kindergarten
|
|---|---|
| Rating |
|
Legend
Convincing evidencePartially convincing evidenceUnconvincing evidenceData unavailabledDisaggregated data available
- *Describe each criterion measure used and explain why each measure is appropriate, given the type and purpose of the tool.
- For predictive validity, we used the Oklahoma State Testing Programs English Language Arts test in 3rd grade, which also supports our classification accuracy. For Concurrent Validity, we used the NWEA MAP. Additionally we provide content validity, and construct validity.
- *Describe the sample(s), including size and characteristics, for each validity analysis conducted.
- For concurrent validity, we used 208 Oklahoma kindergarteners from three sites in 2021. The student sample is representative of students across all performance levels. The samples were collected from 3 schools, a charter school, a public school, and a private school, to ensure a full range of performance on the NWEA MAP and the ELQA. A small number of outliers (both extremely low and extremely high) were removed from each time point based on Cook's distance being greater than 4.5 Median Absolute Deviations from the median performance for that time point. For the concurrent validity analysis reported below, 143 of the 208 students had data for both the ELQA and NWEA MAP at the middle of the year time point and were used for that analysis. For construct validity, the CFA was completed with full item and subtest ELQA-K Literacy data from 1226 Oklahoma kindergartners from the middle of year (time 3) 2017.
- *Describe the analysis procedures for each reported type of validity.
- Concurrent validity: We conducted ELQA-K assessments in the academic year, 2021-2022 and provided 5 time-period ELQA-K literacy data to ECEI. In addition, CECPD provided NWEA MAP data including three-time points (BOY, MOY, and EOY). ECEI selected children who completed all subsets of ELQA-K in the first, third, and fifth time periods and matched them with NWEA MAP data. In other words, we selected children who completed both ELQA-K and NWEA MAP at three-time points across the year and conducted Spearman correlations between the ELQA-K percent correct score and NWEA MAP RIT score for each time point (beginning of year (BOY), middle of year (MOY), and end of year (EOY). Construct validity: A second-order confirmatory factor analysis is used to evaluate construct validity when several related first-order factors (e.g., Phonics, Vocabulary, etc.) are expected to represent a broader underlying construct (e.g., Literacy). It tests whether observed items measure their intended dimensions and whether those dimensions, in turn, load onto a higher-order factor. Support for the model suggests that the subscales are distinct yet sufficiently related to justify interpreting them as components of one overarching construct and, when appropriate, using an overall composite score.
*In the table below, report the results of the validity analyses described above (e.g., concurrent or predictive validity, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Construct validity: A Second Order Confirmatory Factor Analysis (CFA) confirmed that each item in each subtest was significantly related to the subtest construct (e.g., Phonics item 1, Phonics item 2, … are related to Phonics; Print Concepts item 1, Print concepts item 2, … are related to Print Concepts, etc.), all at p < 0.01. Additionally, each subtest (e.g., Phonics, Comprehension) was significantly related to a single 2nd Order Factor (i.e., Literacy; each at p < 0.01) and had good model fit criteria (CFI = .993, TLI = .993, RMSEA = .030, SRMR = .079).
- Manual cites other published reliability studies:
- Provide citations for additional published studies.
- Describe the degree to which the provided data support the validity of the tool.
- Do you have validity data that are disaggregated by gender, race/ethnicity, or other subgroups (e.g., English language learners, students with disabilities)?
- No
If yes, fill in data for each subgroup with disaggregated validity data.
| Type of | Subgroup | Informant | Age / Grade | Test or Criterion | n | Median Coefficient | 95% Confidence Interval Lower Bound |
95% Confidence Interval Upper Bound |
|---|
- Results from other forms of validity analysis not compatible with above table format:
- Predictive validity for the ELQA-K Literacy is good, with a Positive Predictive Value of .78 and a Negative Predictive Value of .84. These values indicate that the ELQA-K Literacy performs strongly in accurately identifying students who are at risk of reading deficiency in 3rd grade while also not categorizing students at risk when they are likely to be reading on grade level by 3rd grade.
- Manual cites other published reliability studies:
- No
- Provide citations for additional published studies.
Bias Analysis
| Grade |
Kindergarten
|
|---|---|
| Rating | Provided |
- Have you conducted additional analyses related to the extent to which your tool is or is not biased against subgroups (e.g., race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)? Examples might include Differential Item Functioning (DIF) or invariance testing in multiple-group confirmatory factor models.
- Yes
- If yes,
- a. Describe the method used to determine the presence or absence of bias:
- A differential item functioning test was done using the Mantel-Haenszel Test on the test scores on the ELQA from the previous 5 years. Item performance across gender and ethnicity groups were compared and no significant difference in the functioning across groups was found in any subtest or question.
- b. Describe the subgroups for which bias analyses were conducted:
- Gender (Male and Female) and Race/Ethnicity (Black/African American, Hispanic/Latino, Asian, American Indian or Pacific Islander, White)
- c. Describe the results of the bias analyses conducted, including data and interpretative statements. Include magnitude of effect (if available) if bias has been identified.
- Data on the ELQA were tested for differential item functioning using the Mantel-Haenszel Test for each subgroup against the total sample, utilizing data from 2019-2024 and totalling over 4000 students. Each item was examined by reviewing the p-value of the Chi-Square statistic of the Mantel-Haenzel test, items that were significant at p > .05 were then evaluated for their effect size. Those which were significant were examined for their change in the Mantzel-Haenzel score for their effect size, where a change of greater than 1.5 is considered a large effect between 1.5 and 1 a moderate effect size, and below 1 a negligible effect size. Five items were potentially biased with large or moderate effects but were removed in iterative development of the ELQA, all current items show no differential functioning for any subgroup.
Data Collection Practices
Most tools and programs evaluated by the NCII are branded products which have been submitted by the companies, organizations, or individuals that disseminate these products. These entities supply the textual information shown above, but not the ratings accompanying the text. NCII administrators and members of our Technical Review Committees have reviewed the content on this page, but NCII cannot guarantee that this information is free from error or reflective of recent changes to the product. Tools and programs have the opportunity to be updated annually or upon request.
