Classworks Universal Screener

Mathematics

 

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

$5 per student*

*Classworks Universal Screener is purchased as part of a comprehensive Response to Intervention (RtI) program. When the full Classworks suite is purchased the screener is included.

 

Replacement Cost:

No information provided; contact vendor for details.

 

Included in Cost:

Classworks professional development includes free online training and virtual training sessions. Training days can be purchased at $1,500 - $1,800 per day depending on volume of days purchased.

 

Technology Requirements:

  • Computer or tablet
  • Internet connection

 

Training Requirements:

  • Less than 1 hour of training

 

Qualified Administrators:

  • No minimum qualifications specified

 

Accommodations:

Classworks assessments and instruction are designed to be accessible for most students. Accessible design features included should aid most students who typically require testing accommodations such as large print, audio support, or extra time.

 

Accommodations included are as follows:

  • Extra time—Students may need extra time to complete the task. The assessment may be stopped and started as needed to allow students needing extra time to finish, is untimed, and can be administered in multiple test sessions.
  • Administrations—Students may be given the amount of days necessary to complete the test.
  • Presentation—Classworks diagnostic items are presented in a large, easily legible format specifically chosen for its readability. With HTML5, you have the ability to change the screen size and font size.
  • Audio Support–Audio support is available for all grades.
  • Setting—Classworks assessments are web-based. They can be completed on any device with internet access that meets technical requirements. Students may use headphones to benefit from audio support.
  • Response—Classworks assessments are easily completed on a computer using point and click or on a tablet device using touch screens for those with motor impairment.

Where to Obtain:

Website: www.curriculumadvantage.com

Address: 5185 Peachtree Parkway Suite 285, Peachtree Corners, GA 30092

Phone number: 770-325-5555


Access to Technical Support:

Classworks support is available online via in product chat, by phone, and by email.

 

Classworks Universal Screeners for Mathematics are formal assessments used to measure readiness for grade level instruction, help identify baseline learning levels, and measure growth. The Universal Screeners were specifically designed for the purpose of screening students who may need additional intervention and can be used as part of the RtI process.

 

In addition to reporting an overall scaled score based on the total test, Classworks provides student strengths and weaknesses for key strands. Key strands include a minimum of four test questions to provide a reasonable estimate of student strengths and weaknesses.

Assessment Format:

  • Direct: Computerized

 

Administration Time:

  • 30 minutes per student
  • 30 minutes per group

 

Scoring Time:

  • Scoring is automatic

 

Scoring Method:

  • Calculated automatically

 

Scores Generated:

  • Raw score
  • IRT-based score
  • Developmental benchmarks
  • Equated
  • Strand-level proficiency feedback

 

Classification Accuracy

Grade234567
Criterion 1 FallHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleEmpty bubble
Criterion 1 Winterdashdashdashdashdashdash
Criterion 1 Springdashdashdashdashdashdash
Criterion 2 Falldashdashdashdashdashdash
Criterion 2 Winterdashdashdashdashdashdash
Criterion 2 Springdashdashdashdashdashdash

Primary Sample

 

Criterion 1, Fall

Grade

2

3

4

5

6

7

Criterion

MAP Growth

MAP Growth

MAP Growth

MAP Growth

MAP Growth

MAP Growth

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

20th percentile

20th percentile

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

MAP Growth < 168

MAP Growth < 182

MAP Growth < 193

MAP Growth < 202

MAP Growth < 208

MAP Growth < 214

Cut points: Corresponding performance score (numeric) on screener measure

Classworks Screener < 290

Classworks Screener < 330

Classworks Screener < 370

Classworks Screener < 410

Classworks Screener < 440

Classworks Screener < 470

Base rate in the sample for children requiring intensive intervention

0.11

0.13

0.19

0.22

0.25

0.30

False Positive Rate

0.14

0.25

0.22

0.17

0.13

0.29

False Negative Rate

0.23

0.06

0.32

0.20

0.31

0.36

Sensitivity

0.77

0.94

0.68

0.80

0.69

0.64

Specificity

0.86

0.75

0.78

0.83

0.88

0.71

Positive Predictive Power

0.98

0.96

0.93

0.94

0.94

0.84

Negative Predictive Power

0.32

0.64

0.36

0.54

0.48

0.45

Overall Classification Rate

0.78

0.91

0.70

0.80

0.73

0.66

Area Under the Curve (AUC)

0.82

0.84

0.73

0.81

0.78

0.68

AUC 95% Confidence Interval Lower Bound

0.75

0.77

0.67

0.76

0.72

0.57

AUC 95% Confidence Interval Upper Bound

0.89

0.92

0.79

0.87

0.84

0.78

 

Reliability

Grade234567
RatingHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubbleHalf-filled bubble
  1. Justification for each type of reliability reported, given the type and purpose of the tool: The Classworks Screener affords the means to screen students on multiple occasions (e.g., Fall, Winter, Spring) during the school year. Thus, test-retest reliability is necessary, and we estimate test-retest reliability via the Pearson correlation between Classworks Screener scores of students taking test in two terms within the school year (Fall/Winter, and Winter/Spring).

The second reliability test is Cronbach’s Alpha, a measure of internal consistency. This analysis was conducted on a sample of students who had posted scores for three sets of Classworks Screener questions, all of which aimed to measure a single construct--student’s proficiency in mathematics.

 

  1. Description of the sample(s), including size and characteristics, for each reliability analysis conducted: See sample size information in the data.

 

  1. Description of the analysis procedures for each reported type of reliability: See above question 1.

 

  1. Reliability of performance level score (e.g., model-based, internal consistency, inter-rater reliability).

Type of Reliability

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Test-Retest (Fall/Winter)

2

1237

0.60

0.56

0.63

Test-Retest (Winter/Spring)

2

967

0.70

0.67

0.74

Test-Retest (Fall/Winter)

3

1212

0.67

0.63

0.70

Test-Retest (Winter/Spring)

3

908

0.76

0.73

0.79

Test-Retest (Fall/Winter)

4

1350

0.65

0.62

0.68

Test-Retest (Winter/Spring)

4

1140

0.72

0.69

0.75

Test-Retest (Fall/Winter)

5

1421

0.71

0.68

0.73

Test-Retest (Winter/Spring)

5

1245

0.72

0.69

0.75

Test-Retest (Fall/Winter)

6

1771

0.70

0.68

0.73

Test-Retest (Winter/Spring)

6

1264

0.71

0.68

0.74

Test-Retest (Fall/Winter)

7

1286

0.65

0.62

0.68

Test-Retest (Winter/Spring)

7

922

0.66

0.62

0.69

Cronbach’s Alpha

2

851

0.83

0.81

0.85

Cronbach’s Alpha

3

802

0.86

0.84

0.88

Cronbach’s Alpha

4

916

0.86

0.85

0.88

Cronbach’s Alpha

5

1072

0.88

0.87

0.89

Cronbach’s Alpha

6

1016

0.88

0.86

0.89

Cronbach’s Alpha

7

791

0.84

0.82

0.86

 

Disaggregated Reliability

The following disaggregated reliability data are provided for context and did not factor into the Reliability rating.

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

Validity

Grade234567
RatingEmpty bubbleFull bubbleHalf-filled bubbleFull bubbleFull bubbleEmpty bubble
  1. Description of each criterion measure used and explanation as to why each measure is appropriate, given the type and purpose of the tool: The validity evidence for the Classworks Screener comes from the relationships of Classworks Screener test scores to NWEA MAP Growth test scores. These relationships include a) the concurrent performance of students on Classworks Screener tests with their performance on MAP Growth tests and b) the predictive relationship between students’ performance on Classworks Screener tests with their performance, two testing terms later, on MAP Growth tests.

The Measures of Academic Progress (MAP) is used as the outcome measure. Published by the NWEA the MAP Growth is regarded as a highly valid and reliable measure of broad reading ability. The NWEA website states, “Our tools are trusted by educators in 140 countries and more than half the schools in the US” which indicates it can be considered an excellent outcome measure for classification studies.

 

  1. Description of the sample(s), including size and characteristics, for each validity analysis conducted: See data chart for sample size information.

 

  1. Description of the analysis procedures for each reported type of validity: For the validity analysis conducted, we used concurrent and predictive validity. Concurrent validity was estimated as the Pearson correlation coefficient between student scores from Fall 2016 and the same students’ total scale score on the Map Growth assessment (also administered in Fall 2016). Predictive validity was estimated as the Pearson correlation coefficient between student scores from a given term (Fall 2016) and the same students’ total scale score on the MAP Growth assessment administered in Spring 2017.

 

  1. Validity for the performance level score (e.g., concurrent, predictive, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Concurrent (Fall/Fall)

2

Map Growth

262

0.63

0.55

0.70

Predictive (Fall/Winter)

2

Map Growth

263

0.66

0.59

0.72

Concurrent (Fall/Fall)

3

Map Growth

273

0.77

0.71

0.81

Predictive (Fall/Winter)

3

Map Growth

269

0.78

0.72

0.82

Concurrent (Fall/Fall)

4

Map Growth

310

0.67

0.60

0.72

Predictive (Fall/Winter)

4

Map Growth

309

0.64

0.57

0.71

Concurrent (Fall/Fall)

5

Map Growth

295

0.74

0.69

0.79

Predictive (Fall/Winter)

5

Map Growth

292

0.73

0.67

0.78

Concurrent (Fall/Fall)

6

Map Growth

192

0.71

0.63

0.77

Predictive (Fall/Winter)

6

Map Growth

191

0.71

0.63

0.77

Concurrent (Fall/Fall)

7

Map Growth

95

0.66

0.53

0.76

Predictive (Fall/Winter)

7

Map Growth

94

0.68

0.56

0.78

 

  1. Results for other forms of validity (e.g. factor analysis) not conducive to the table format: Not Provided

 

  1. Describe the degree to which the provided data support the validity of the tool: Concurrent and predictive validity coefficients, for each grade and each time of year, were consistently in the mid to high 0.60s to 0.70s. This validity evidence demonstrates a strong relationship between the Classworks Screener and the MAP Growth assessments across the grades and times of year reported.

 

Disaggregated Validity

The following disaggregated validity data are provided for context and did not factor into the Validity rating.

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

 

Results for other forms of disaggregated validity (e.g. factor analysis) not conducive to the table format: Not Provided

Sample Representativeness

Grade234567
Data
  • No Evidence
  • No Evidence
  • No Evidence
  • No Evidence
  • No Evidence
  • No Evidence
  • Primary Classification Accuracy Sample

     

    Grade

    2

    3

    4

    5

    6

    7

    Criterion

    MAP Growth

    MAP Growth

    MAP Growth

    MAP Growth

    MAP Growth

    MAP Growth

    National/Local Representation

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Date

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Sample Size

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Male

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Female

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Gender Unknown

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Free or Reduced-price Lunch Eligible

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    White, Non-Hispanic

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Black, Non-Hispanic

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Hispanic

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Disability Classification

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

     

    Bias Analysis Conducted

    Grade234567
    RatingNoNoNoNoNoNo
    1. Description of the method used to determine the presence or absence of bias: Not Provided  

     

    1. Description of the subgroups for which bias analyses were conducted: Not Provided  

     

    1. Description of the results of the bias analyses conducted, including data and interpretative statements: Not Provided

    Administration Format

    Grade234567
    Data
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Administration & Scoring Time

    Grade234567
    Data
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • Scoring Format

    Grade234567
    Data
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Manual
  • Automatic
  • Types of Decision Rules

    Grade234567
    Data
  • None
  • None
  • None
  • None
  • None
  • None
  • Evidence Available for Multiple Decision Rules

    Grade234567
    Data
  • No
  • No
  • No
  • No
  • No
  • No