easyCBM

Reading Comprehension

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

$5 per student per year*

*In Year 1 of subscription, buyers must complete three training webinars. One webinar is provided at no charge and the remaining two cost $200 each.

 

Replacement Cost:

$5 per student per year.

Annual license renewal fee subject to change.

 

Included in Cost:

easyCBM is available through Houghton Mifflin Harcourt on an annual subscription license for districts. The price includes manuals and use of the assessments.

 

easyCBM is also available directly through the University of Oregon for individual classroom teacher use (limited to one teacher per building, maximum of 200 students). This teacher subscription includes the online training that is part of the system.

 

Teachers have unlimited access to the system and reports. Training is available within the system. Students can access any assigned test on line.

 

Technology Requirements:

  • Computer or tablet
  • Internet connection

 

Training Requirements:

  • Less than 1 hour of training

 

Qualified Administrators:

  • Paraprofessionals
  • Professionals

 

Accommodations:

All measures were developed following Universal Design for Assessment guidelines to reduce the need for accommodations. However, districts are directed to develop their own practices for accommodations as needed.

Where to Obtain:

Contact Information for Districts:

Houghton Mifflin Harcourt Website:

http://www.hmhco.com/hmh-assessments/response-to-intervention/easycbm

Address:

Houghton Mifflin Harcourt, Attention Customer Experience Support—Assessments, 255 38th Avenue, Suite L, St. Charles, IL  60174

Phone Number:

800.323.9540

Email:

easyCBM@hmhco.com


Contact Information for Individual Teachers:

Website:
www.easyCBM.com   

Address:

BRT, University of Oregon, Eugene, OR 97403

Phone Number:
541.346.3535

Email:
support@easycbm.com   


Access to Technical Support:

Help desk via email and phone.

easyCBM® is a web-based assessment system that includes both benchmarking and progress monitoring assessments combined with a comprehensive array of reports. The assessments in easyCBM are curriculum-based general outcome measures, or CBMs, which are standardized measures that sample from a year’s worth of curriculum to assess the degree to which students have mastered the skills and knowledge deemed critical at each grade level.

 

In Grades K–8, easyCBM provides three forms of a screening measure to be used locally for establishing benchmarks and multiple forms (generally 17 in reading) to be used to monitor progress. All measures have been developed with reference to specific content in reading (National Reading Panel) and developed using Item Response Theory (IRT).

 

Assessment Format:

  • Individual
  • Group
  • Computer-administered

 

Administration Time:

  • 30 minutes per student
  • 30 minutes per group

 

Scoring Time:

  • Scoring is automatic

 

Scoring Method:

  • Calculated manually
  • Calculated automatically

 

Scores Generated:

  • Percentile Score
  • Raw Score

 

 

Reliability

Grade345678
RatingEmpty bubbledEmpty bubbledEmpty bubbledEmpty bubbledEmpty bubbledEmpty bubbled

Justify the appropriateness of each type of reliability reported:

Internal consistency was investigated using Cronbach’s alpha, perhaps the most common measure of internal consistency. The authors supplemented these indices with split-half reliability estimates using all possible split-half comparisons. The mean of the split-half estimates are reported in the table below.

 

Describe the sample characteristics for each reliability analysis conducted:

A large sample collected across the country was used for investigation. Measures from each of the fall, winter, and spring test windows were evaluated in Grades 3–8. The proportion of male students across grades varied from 50.8% to 52.4%. Across grades, approximately 3–4% of students identified as American Indian or Alaskan Native; 3–4% identified as Asian; 11–12% identified as Black or African American; 0–1% identified as Native Hawaiian or Other Pacific Islander; 5–10% identified as two or more races; 59–62% identified as White; and 10–16% were unknown or refused to report. 

 

Describe the analysis procedures for each reported type of reliability:

Item-level data were extracted from the easyCBM database. Correct responses were coded 1 while incorrect responses were coded 0. The psych package in R was used to calculate both indicators of internal consistency.

Type of Reliability

Age or Grade

n

Coefficient

Confidence Interval

Cronbach’s Alpha (Fall)

Grade 3

32,761

0.71

 

Split Half Reliability (Fall)

Grade 3  

32,761

0.74

 

Cronbach’s Alpha (Winter)


Grade 3

31,741

0.59

 

Split Half Reliability (Winter)


Grade 3

31,741

0.67

 

Cronbach’s Alpha (Spring)


Grade 3

30,689

0.79

 

Split Half Reliability (Spring)


Grade 3

30,689

0.804

 

Cronbach’s Alpha (Fall)


Grade 4

37,397

0.78

 

Split Half Reliability (Fall)


Grade 4

37,397

0.79

 

Cronbach’s Alpha (Winter)


Grade 4

35,692

0.77

 

Split Half Reliability (Winter)


Grade 4

35,692

0.78

 

Cronbach’s Alpha (Spring)


Grade 4

34,416

0.79

 

Split Half Reliability (Spring)


Grade 4

34,416

0.804

 

Cronbach’s Alpha (Fall)


Grade 5

37,770

0.75

 

Split Half Reliability (Fall)


Grade 5

37,770

0.78

 

Cronbach’s Alpha (Winter)


Grade 5

36,244

0.805

 

Split Half Reliability (Winter)


Grade 5

36,244

0.81

 

Cronbach’s Alpha (Spring)


Grade 5

34,583

0.74

 

Split Half Reliability (Spring)


Grade 5

34,583

0.79

 

Cronbach’s Alpha (Fall)


Grade 6

29,398

0.71

 

Split Half Reliability (Fall)


Grade 6

29,398

0.73

 

Cronbach’s Alpha (Winter)


Grade 6

27,224

0.75

 

Split Half Reliability (Winter)


Grade 6

27,224

0.77

 

Cronbach’s Alpha (Spring)


Grade 6

26,425

0.69

 

Split Half Reliability (Spring)


Grade 6

26,425

0.74

 

Cronbach’s Alpha (Fall)


Grade 7

27,409

0.74

 

Split Half Reliability (Fall)


Grade 7

27,409

0.76

 

Cronbach’s Alpha (Winter)


Grade 7

24,326

0.73

 

Split Half Reliability (Winter)


Grade 7

24,326

0.76

 

Cronbach’s Alpha (Spring)


Grade 7

26,486

0.65

 

Split Half Reliability (Spring)


Grade 7

26,486

0.71

 

Cronbach’s Alpha (Fall)


Grade 8

26,360

0.72

 

Split Half Reliability (Fall)


Grade 8

26,360

0.75

 

Cronbach’s Alpha (Winter)


Grade 8

22,859

0.66

 

Split Half Reliability (Winter)


Grade 8

22,859

0.68

 

Cronbach’s Alpha (Spring)


Grade 8

23,361

0.68

 

Split Half Reliability (Spring)


Grade 8

23,361

0.71

 

 

Disaggregated Reliability Data

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

Confidence Interval

Cronbach’s Alpha (Fall)

White

Grade 3

19,839

0.72

 

Cronbach’s Alpha (Fall)

American Indian or Alaskan Native

Grade 3

1,002

0.66

 

Cronbach’s Alpha (Fall)

Asian

Grade 3

1,024

0.72

 

Cronbach’s Alpha (Fall)

Two or more races

Grade 3

1,461

0.7

 

Cronbach’s Alpha (Fall)

Black or African American

Grade 3

4,452

0.64

 

Cronbach’s Alpha (Fall)

Native Hawaiian or Other Pacific Islander

Grade 3

136

0.68

 

Cronbach’s Alpha (Fall)

Unknown race/ethnicity

Grade 3

4,589

0.71

 

Cronbach’s Alpha (Winter)

White

Grade 3

19,113

0.59

 

Cronbach’s Alpha (Winter)

American Indian or Alaskan Native

Grade 3

993

0.53

 

Cronbach’s Alpha (Winter)

Asian

Grade 3

940

0.6

 

Cronbach’s Alpha (Winter)

Two or more races

Grade 3

1,466

0.57

 

Cronbach’s Alpha (Winter)

Black or African American

Grade 3

4,160

0.49

 

Cronbach’s Alpha (Winter)

Native Hawaiian or Other Pacific Islander

Grade 3

141

0.56

 

Cronbach’s Alpha (Winter)

Unknown race/ethnicity

Grade 3

4,682

0.58

 

Cronbach’s Alpha (Fall)

Male

Grade 3

15,698

0.7

 

Cronbach’s Alpha (Fall)

Female

Grade 3

16,966

0.72

 

Cronbach’s Alpha (Spring)

White

Grade 3

18,200

0.79

 

Cronbach’s Alpha (Spring)

American Indian or Alaskan Native

Grade 3

960

0.74

 

Cronbach’s Alpha (Spring)

Asian

Grade 3

944

0.77

 

Cronbach’s Alpha (Spring)

Two or more races

Grade 3

1,369

0.78

 

Cronbach’s Alpha (Spring)

Black or African American

Grade 3

4,147

0.75

 

Cronbach’s Alpha (Spring)

Native Hawaiian or Other Pacific Islander

Grade 3

130

0.77

 

Cronbach’s Alpha (Spring)

Unknown race/ethnicity

Grade 3

4,736

--

 

Split Half Reliability (Fall)

White

Grade 3

19,839

0.74

 

Split Half Reliability (Fall)

American Indian or Alaskan Native

Grade 3

1,002

0.68

 

Split Half Reliability (Fall)

Asian

Grade 3

1,024

0.74

 

Split Half Reliability (Fall)

Two or more races

Grade 3

1,461

0.72

 

Split Half Reliability (Fall)

Black or African American

Grade 3

4,452

0.66

 

Split Half Reliability (Fall)

Native Hawaiian or Other Pacific Islander

Grade 3

136

0.71

 

Split Half Reliability (Fall)

Unknown race/ethnicity

Grade 3

4,589

0.74

 

Split Half Reliability (Fall)

Male

Grade 3

15,698

0.72

 

Split Half Reliability (Fall)

Female

Grade 3

16,966

0.75

 

Cronbach’s Alpha (Winter)

Male

Grade 3

15,308

0.56

 

Cronbach’s Alpha (Winter)

Female

Grade 3

16,319

0.6

 

Split Half Reliability (Winter)

White

Grade 3

19,113

0.67

 

Split Half Reliability (Winter)

American Indian or Alaskan Native

Grade 3

993

0.62

 

Split Half Reliability (Winter)

Asian

Grade 3

940

0.67

 

Split Half Reliability (Winter)

Two or more races

Grade 3

1,466

0.66

 

Split Half Reliability (Winter)

Black or African American

Grade 3

4,160

0.62

 

Split Half Reliability (Winter)

Native Hawaiian or Other Pacific Islander

Grade 3

141

0.71

 

Split Half Reliability (Winter)

Unknown race/ethnicity

Grade 3

4,682

0.67

 

Split Half Reliability (Winter)

Male

Grade 3

15,308

0.67

 

Split Half Reliability (Winter)

Female

Grade 3

16,319

0.67

 

Cronbach’s Alpha (Spring)

Male

Grade 3

15,760

0.78

 

Cronbach’s Alpha (Spring)

Female

Grade 3

14,762

0.79

 

Split Half Reliability (Spring)

White

Grade 3

18,200

0.8

 

Split Half Reliability (Spring)

American Indian or Alaskan Native

Grade 3

960

0.74

 

Split Half Reliability (Spring)

Asian

Grade 3

944

0.79

 

Split Half Reliability (Spring)

Two or more races

Grade 3

1,369

0.79

 

Split Half Reliability (Spring)

Black or African American

Grade 3

4,147

0.75

 

Split Half Reliability (Spring)

Native Hawaiian or Other Pacific Islander

Grade 3

130

0.78

 

Split Half Reliability (Spring)

Unknown race/ethnicity

Grade 3

4,736

--

 

Split Half Reliability (Spring)

Male

Grade 3

15,760

0.79

 

Split Half Reliability (Spring)

Female

Grade 3

14,762

0.8

 

Cronbach’s Alpha (Fall)

White

Grade 4

22,283

0.78

 

Cronbach’s Alpha (Fall)

American Indian or Alaskan Native

Grade 4

958

0.75

 

Cronbach’s Alpha (Fall)

Asian

Grade 4

1,300

0.78

 

Cronbach’s Alpha (Fall)

Two or more races

Grade 4

2,615

0.77

 

Cronbach’s Alpha (Fall)

Black or African American

Grade 4

4,929

0.71

 

Cronbach’s Alpha (Fall)

Native Hawaiian or Other Pacific Islander

Grade 4

161

0.72

 

Cronbach’s Alpha (Fall)

Unknown race/ethnicity

Grade 4

4,764

0.78

 

Cronbach’s Alpha (Fall)

Male

Grade 4

19,047

0.78

 

Cronbach’s Alpha (Fall)

Female

Grade 4

18,237

0.78

 

Split Half Reliability (Fall)

White

Grade 4

22,283

0.79

 

Split Half Reliability (Fall)

American Indian or Alaskan Native

Grade 4

958

0.75

 

Split Half Reliability (Fall)

Asian

Grade 4

1,300

0.79

 

Split Half Reliability (Fall)

Two or more races

Grade 4

2,615

0.77

 

Split Half Reliability (Fall)

Black or African American

Grade 4

4,929

0.71

 

Split Half Reliability (Fall)

Native Hawaiian or Other Pacific Islander

Grade 4

161

0.73

 

Split Half Reliability (Fall)

Unknown race/ethnicity

Grade 4

4,764

0.79

 

Split Half Reliability (Fall)

Male

Grade 4

19,047

0.78

 

Split Half Reliability (Fall)

Female

Grade 4

18,237

0.79

 

Cronbach’s Alpha (Winter)

White

Grade 4

22,283

0.76

 

Cronbach’s Alpha (Winter)

American Indian or Alaskan Native

Grade 4

958

0.74

 

Cronbach’s Alpha (Winter)

Asian

Grade 4

1,300

0.72

 

Cronbach’s Alpha (Winter)

Two or more races

Grade 4

2,615

0.76

 

Cronbach’s Alpha (Winter)

Black or African American

Grade 4

4,929

0.75

 

Cronbach’s Alpha (Winter)

Native Hawaiian or Other Pacific Islander

Grade 4

161

0.77

 

Cronbach’s Alpha (Winter)

Unknown race/ethnicity

Grade 4

4,764

0.76

 

Cronbach’s Alpha (Winter)

Male

Grade 4

18,172

0.77

 

Cronbach’s Alpha (Winter)

Female

Grade 4

17,399

0.76

 

Split Half Reliability (Winter)

White

Grade 4

21,197

0.77

 

Split Half Reliability (Winter)

American Indian or Alaskan Native

Grade 4

922

0.75

 

Split Half Reliability (Winter)

Asian

Grade 4

1,216

0.73

 

Split Half Reliability (Winter)

Two or more races

Grade 4

2,339

0.76

 

Split Half Reliability (Winter)

Black or African American

Grade 4

4,772

0.75

 

Split Half Reliability (Winter)

Native Hawaiian or Other Pacific Islander

Grade 4

163

0.76

 

Split Half Reliability (Winter)

Unknown race/ethnicity

Grade 4

4,772

0.77

 

Split Half Reliability (Winter)

Male

Grade 4

18,172

0.77

 

Split Half Reliability (Winter)

Female

Grade 4

17,399

0.77

 

Cronbach’s Alpha (Spring)

White

Grade 4

20,331

0.78

 

Cronbach’s Alpha (Spring)

American Indian or Alaskan Native

Grade 4

884

0.77

 

Cronbach’s Alpha (Spring)

Asian

Grade 4

1,155

0.73

 

Cronbach’s Alpha (Spring)

Two or more races

Grade 4

2,373

0.78

 

Cronbach’s Alpha (Spring)

Black or African American

Grade 4

4,492

0.76

 

Cronbach’s Alpha (Spring)

Native Hawaiian or Other Pacific Islander

Grade 4

152

0.77

 

Cronbach’s Alpha (Spring)

Unknown race/ethnicity

Grade 4

264

0.78

 

Cronbach’s Alpha (Spring)

Male

Grade 4

17,416

0.79

 

Cronbach’s Alpha (Spring)

Female

Grade 4

16,846

0.79

 

Split Half Reliability (Spring)

White

Grade 4

20,331

0.8

 

Split Half Reliability (Spring)

American Indian or Alaskan Native

Grade 4

884

0.78

 

Split Half Reliability (Spring)

Asian

Grade 4

1,155

0.76

 

Split Half Reliability (Spring)

Two or more races

Grade 4

2,373

0.79

 

Split Half Reliability (Spring)

Black or African American

Grade 4

4,492

0.76

 

Split Half Reliability (Spring)

Native Hawaiian or Other Pacific Islander

Grade 4

152

0.78

 

Split Half Reliability (Spring)

Unknown race/ethnicity

Grade 4

264

0.79

 

Split Half Reliability (Spring)

Male

Grade 4

17,416

0.79

 

Split Half Reliability (Spring)

Female

Grade 4

16,846

0.8

 

Cronbach’s Alpha (Fall)

White

Grade 5

21,876

0.73

 

Cronbach’s Alpha (Fall)

American Indian or Alaskan Native

Grade 5

1,015

0.74

 

Cronbach’s Alpha (Fall)

Asian

Grade 5

1,347

0.7

 

Cronbach’s Alpha (Fall)

Two or more races

Grade 5

2,498

0.75

 

Cronbach’s Alpha (Fall)

Black or African American

Grade 5

4,998

0.77

 

Cronbach’s Alpha (Fall)

Native Hawaiian or Other Pacific Islander

Grade 5

159

0.73

 

Cronbach’s Alpha (Fall)

Unknown race/ethnicity

Grade 5

5,276

0.73

 

Cronbach’s Alpha (Fall)

Male

Grade 5

19,329

0.75

 

Cronbach’s Alpha (Fall)

Female

Grade 5

18,314

0.73

 

Split Half Reliability (Fall)

White

Grade 5

21,876

0.77

 

Split Half Reliability (Fall)

American Indian or Alaskan Native

Grade 5

1,015

0.76

 

Split Half Reliability (Fall)

Asian

Grade 5

1,347

0.75

 

Split Half Reliability (Fall)

Two or more races

Grade 5

2,498

0.77

 

Split Half Reliability (Fall)

Black or African American

Grade 5

4,998

0.78

 

Split Half Reliability (Fall)

Native Hawaiian or Other Pacific Islander

Grade 5

159

0.77

 

Split Half Reliability (Fall)

Unknown race/ethnicity

Grade 5

5,276

0.76

 

Split Half Reliability (Fall)

Male

Grade 5

19,329

0.77

 

Split Half Reliability (Fall)

Female

Grade 5

18,314

0.77

 

Cronbach’s Alpha (Winter)

White

Grade 5

21,206

0.79

 

Cronbach’s Alpha (Winter)

American Indian or Alaskan Native

Grade 5

1,009

0.78

 

Cronbach’s Alpha (Winter)

Asian

Grade 5

1,262

0.76

 

Cronbach’s Alpha (Winter)

Two or more races

Grade 5

2,243

0.8

 

Cronbach’s Alpha (Winter)

Black or African American

Grade 5

4,666

0.8

 

Cronbach’s Alpha (Winter)

Native Hawaiian or Other Pacific Islander

Grade 5

170

0.79

 

Cronbach’s Alpha (Winter)

Unknown race/ethnicity

Grade 5

5,168

--

 

Cronbach’s Alpha (Winter)

Male

Grade 5

18,572

0.81

 

Cronbach’s Alpha (Winter)

Female

Grade 5

17,517

0.79

 

Split Half Reliability (Winter)

White

Grade 5

21,206

0.8

 

Split Half Reliability (Winter)

American Indian or Alaskan Native

Grade 5

1,009

0.79

 

Split Half Reliability (Winter)

Asian

Grade 5

1,262

0.79

 

Split Half Reliability (Winter)

Two or more races

Grade 5

2,243

0.81

 

Split Half Reliability (Winter)

Black or African American

Grade 5

4,666

0.8

 

Split Half Reliability (Winter)

Native Hawaiian or Other Pacific Islander

Grade 5

170

0.8

 

Split Half Reliability (Winter)

Unknown race/ethnicity

Grade 5

5,168

--

 

Split Half Reliability (Winter)

Male

Grade 5

18,572

0.81

 

Split Half Reliability (Winter)

Female

Grade 5

17,517

0.8

 

Cronbach’s Alpha (Spring)

White

Grade 5

20,152

0.72

 

Cronbach’s Alpha (Spring)

American Indian or Alaskan Native

Grade 5

1,000

0.7

 

Cronbach’s Alpha (Spring)

Asian

Grade 5

1,254

0.68

 

Cronbach’s Alpha (Spring)

Two or more races

Grade 5

2,247

0.74

 

Cronbach’s Alpha (Spring)

Black or African American

Grade 5

4,380

0.77

 

Cronbach’s Alpha (Spring)

Native Hawaiian or Other Pacific Islander

Grade 5

174

0.69

 

Cronbach’s Alpha (Spring)

Unknown race/ethnicity

Grade 5

4,917

0.72

 

Cronbach’s Alpha (Spring)

Male

Grade 5

17,588

0.75

 

Cronbach’s Alpha (Spring)

Female

Grade 5

16,821

0.72

 

Split Half Reliability (Spring)

White

Grade 5

20,152

0.78

 

Split Half Reliability (Spring)

American Indian or Alaskan Native

Grade 5

1,000

0.77

 

Split Half Reliability (Spring)

Asian

Grade 5

1,254

0.74

 

Split Half Reliability (Spring)

Two or more races

Grade 5

2,247

0.78

 

Split Half Reliability (Spring)

Black or African American

Grade 5

4,380

0.8

 

Split Half Reliability (Spring)

Native Hawaiian or Other Pacific Islander

Grade 5

174

0.71

 

Split Half Reliability (Spring)

Unknown race/ethnicity

Grade 5

4,917

0.78

 

Split Half Reliability (Spring)

Male

Grade 5

17,588

0.79

 

Split Half Reliability (Spring)

Female

Grade 5

16,821

0.78

 

Cronbach’s Alpha (Fall)

White

Grade 6

18,215

0.68

 

Cronbach’s Alpha (Fall)

American Indian or Alaskan Native

Grade 6

781

0.72

 

Cronbach’s Alpha (Fall)

Asian

Grade 6

941

0.67

 

Cronbach’s Alpha (Fall)

Two or more races

Grade 6

2,215

0.7

 

Cronbach’s Alpha (Fall)

Black or African American

Grade 6

3,517

0.74

 

Cronbach’s Alpha (Fall)

Native Hawaiian or Other Pacific Islander

Grade 6

136

0.72

 

Cronbach’s Alpha (Fall)

Unknown race/ethnicity

Grade 6

3,373

0.7

 

Cronbach’s Alpha (Fall)

Male

Grade 6

15,383

0.72

 

Cronbach’s Alpha (Fall)

Female

Grade 6

13,905

0.68

 

Split Half Reliability (Fall)

White

Grade 6

18,215

0.71

 

Split Half Reliability (Fall)

American Indian or Alaskan Native

Grade 6

781

0.73

 

Split Half Reliability (Fall)

Asian

Grade 6

941

0.71

 

Split Half Reliability (Fall)

Two or more races

Grade 6

2,215

0.72

 

Split Half Reliability (Fall)

Black or African American

Grade 6

3,517

0.75

 

Split Half Reliability (Fall)

Native Hawaiian or Other Pacific Islander

Grade 6

136

0.73

 

Split Half Reliability (Fall)

Unknown race/ethnicity

Grade 6

3,373

0.72

 

Split Half Reliability (Fall)

Male

Grade 6

15,383

0.73

 

Split Half Reliability (Fall)

Female

Grade 6

13,905

0.72

 

Cronbach’s Alpha (Winter)

White

Grade 6

16,803

0.68

 

Cronbach’s Alpha (Winter)

American Indian or Alaskan Native

Grade 6

795

0.72

 

Cronbach’s Alpha (Winter)

Asian

Grade 6

850

0.67

 

Cronbach’s Alpha (Winter)

Two or more races

Grade 6

1,975

0.7

 

Cronbach’s Alpha (Winter)

Black or African American

Grade 6

3,407

0.74

 

Cronbach’s Alpha (Winter)

Native Hawaiian or Other Pacific Islander

Grade 6

140

0.68

 

Cronbach’s Alpha (Winter)

Unknown race/ethnicity

Grade 6

2,938

0.7

 

Cronbach’s Alpha (Winter)

Male

Grade 6

12,888

0.69

 

Cronbach’s Alpha (Winter)

Female

Grade 6

14,197

0.67

 

Split Half Reliability (Winter)

White

Grade 6

16,803

0.71

 

Split Half Reliability (Winter)

American Indian or Alaskan Native

Grade 6

795

0.73

 

Split Half Reliability (Winter)

Asian

Grade 6

850

0.71

 

Split Half Reliability (Winter)

Two or more races

Grade 6

1,975

0.72

 

Split Half Reliability (Winter)

Black or African American

Grade 6

3,407

0.75

 

Split Half Reliability (Winter)

Native Hawaiian or Other Pacific Islander

Grade 6

140

0.69

 

Split Half Reliability (Winter)

Unknown race/ethnicity

Grade 6

2,938

0.7

 

Split Half Reliability (Winter)

Male

Grade 6

12,888

0.73

 

Split Half Reliability (Winter)

Female

Grade 6

14,197

0.73

 

Cronbach’s Alpha (Spring)

White

Grade 6

16,562

0.73

 

Cronbach’s Alpha (Spring)

American Indian or Alaskan Native

Grade 6

683

0.74

 

Cronbach’s Alpha (Spring)

Asian

Grade 6

822

0.73

 

Cronbach’s Alpha (Spring)

Two or more races

Grade 6

1,821

0.76

 

Cronbach’s Alpha (Spring)

Black or African American

Grade 6

3,371

0.77

 

Cronbach’s Alpha (Spring)

Native Hawaiian or Other Pacific Islander

Grade 6

137

0.75

 

Cronbach’s Alpha (Spring)

Unknown race/ethnicity

Grade 6

2,722

0.72

 

Cronbach’s Alpha (Spring)

Male

Grade 6

13,726

0.76

 

Cronbach’s Alpha (Spring)

Female

Grade 6

12,565

0.73

 

Split Half Reliability (Spring)

White

Grade 6

16,562

0.76

 

Split Half Reliability (Spring)

American Indian or Alaskan Native

Grade 6

683

0.76

 

Split Half Reliability (Spring)

Asian

Grade 6

822

0.76

 

Split Half Reliability (Spring)

Two or more races

Grade 6

1,821

0.78

 

Split Half Reliability (Spring)

Black or African American

Grade 6

3,371

0.77

 

Split Half Reliability (Spring)

Native Hawaiian or Other Pacific Islander

Grade 6

137

0.76

 

Split Half Reliability (Spring)

Unknown race/ethnicity

Grade 6

2,722

0.75

 

Split Half Reliability (Spring)

Male

Grade 6

13,726

0.77

 

Split Half Reliability (Spring)

Female

Grade 6

12,565

0.76

 

Cronbach’s Alpha (Fall)

White

Grade 7

16,915

0.73

 

Cronbach’s Alpha (Fall)

American Indian or Alaskan Native

Grade 7

764

0.71

 

Cronbach’s Alpha (Fall)

Asian

Grade 7

913

0.72

 

Cronbach’s Alpha (Fall)

Two or more races

Grade 7

1,709

0.72

 

Cronbach’s Alpha (Fall)

Black or African American

Grade 7

3,356

0.73

 

Cronbach’s Alpha (Fall)

Native Hawaiian or Other Pacific Islander

Grade 7

112

0.72

 

Cronbach’s Alpha (Fall)

Unknown race/ethnicity

Grade 7

3,326

0.72

 

Cronbach’s Alpha (Fall)

Male

Grade 7

14,182

0.74

 

Cronbach’s Alpha (Fall)

Female

Grade 7

13,081

0.73

 

Split Half Reliability (Fall)

White

Grade 7

16,915

0.75

 

Split Half Reliability (Fall)

American Indian or Alaskan Native

Grade 7

764

0.73

 

Split Half Reliability (Fall)

Asian

Grade 7

913

0.75

 

Split Half Reliability (Fall)

Two or more races

Grade 7

1,709

0.74

 

Split Half Reliability (Fall)

Black or African American

Grade 7

3,356

0.73

 

Split Half Reliability (Fall)

Native Hawaiian or Other Pacific Islander

Grade 7

112

0.74

 

Split Half Reliability (Fall)

Unknown race/ethnicity

Grade 7

3,326

0.75

 

Split Half Reliability (Fall)

Male

Grade 7

14,182

0.76

 

Split Half Reliability (Fall)

Female

Grade 7

13,081

0.75

 

Cronbach’s Alpha (Winter)

White

Grade 7

14,446

0.72

 

Cronbach’s Alpha (Winter)

American Indian or Alaskan Native

Grade 7

737

0.73

 

Cronbach’s Alpha (Winter)

Asian

Grade 7

782

0.67

 

Cronbach’s Alpha (Winter)

Two or more races

Grade 7

1,763

0.74

 

Cronbach’s Alpha (Winter)

Black or African American

Grade 7

3,146

0.74

 

Cronbach’s Alpha (Winter)

Native Hawaiian or Other Pacific Islander

Grade 7

119

0.73

 

Cronbach’s Alpha (Winter)

Unknown race/ethnicity

Grade 7

3,015

0.7

 

Cronbach’s Alpha (Winter)

Male

Grade 7

12,577

0.75

 

Cronbach’s Alpha (Winter)

Female

Grade 7

11,560

0.7

 

Split Half Reliability (Winter)

White

Grade 7

14,446

0.76

 

Split Half Reliability (Winter)

American Indian or Alaskan Native

Grade 7

737

0.76

 

Split Half Reliability (Winter)

Asian

Grade 7

782

0.72

 

Split Half Reliability (Winter)

Two or more races

Grade 7

1,763

0.76

 

Split Half Reliability (Winter)

Black or African American

Grade 7

3,146

0.77

 

Split Half Reliability (Winter)

Native Hawaiian or Other Pacific Islander

Grade 7

119

0.76

 

Split Half Reliability (Winter)

Unknown race/ethnicity

Grade 7

3,015

0.75

 

Split Half Reliability (Winter)

Male

Grade 7

12,577

0.77

 

Split Half Reliability (Winter)

Female

Grade 7

11,560

0.75

 

Cronbach’s Alpha (Spring)

White

Grade 7

16,528

0.63

 

Cronbach’s Alpha (Spring)

American Indian or Alaskan Native

Grade 7

702

0.65

 

Cronbach’s Alpha (Spring)

Asian

Grade 7

926

0.64

 

Cronbach’s Alpha (Spring)

Two or more races

Grade 7

1,680

0.65

 

Cronbach’s Alpha (Spring)

Black or African American

Grade 7

3,287

0.65

 

Cronbach’s Alpha (Spring)

Native Hawaiian or Other Pacific Islander

Grade 7

119

0.63

 

Cronbach’s Alpha (Spring)

Unknown race/ethnicity

Grade 7

2,930

0.64

 

Cronbach’s Alpha (Spring)

Male

Grade 7

12,806

0.65

 

Cronbach’s Alpha (Spring)

Female

Grade 7

13,493

0.63

 

Split Half Reliability (Spring)

White

Grade 7

16,528

0.7

 

Split Half Reliability (Spring)

American Indian or Alaskan Native

Grade 7

702

0.7

 

Split Half Reliability (Spring)

Asian

Grade 7

926

0.69

 

Split Half Reliability (Spring)

Two or more races

Grade 7

1,680

0.71

 

Split Half Reliability (Spring)

Black or African American

Grade 7

3,287

0.7

 

Split Half Reliability (Spring)

Native Hawaiian or Other Pacific Islander

Grade 7

119

0.72

 

Split Half Reliability (Spring)

Unknown race/ethnicity

Grade 7

2,930

0.7

 

Split Half Reliability (Spring)

Male

Grade 7

12,806

0.71

 

Split Half Reliability (Spring)

Female

Grade 7

13,493

0.7

 

Cronbach’s Alpha (Fall)

White

Grade 8

16,174

0.69

 

Cronbach’s Alpha (Fall)

American Indian or Alaskan Native

Grade 8

559

0.69

 

Cronbach’s Alpha (Fall)

Asian

Grade 8

973

0.7

 

Cronbach’s Alpha (Fall)

Two or more races

Grade 8

2,152

0.71

 

Cronbach’s Alpha (Fall)

Black or African American

Grade 8

3,777

0.76

 

Cronbach’s Alpha (Fall)

Native Hawaiian or Other Pacific Islander

Grade 8

124

0.73

 

Cronbach’s Alpha (Fall)

Unknown race/ethnicity

Grade 8

2,472

0.73

 

Cronbach’s Alpha (Fall)

Male

Grade 8

13,580

0.72

 

Cronbach’s Alpha (Fall)

Female

Grade 8

12,672

0.71

 

Split Half Reliability (Fall)

White

Grade 8

16,174

0.73

 

Split Half Reliability (Fall)

American Indian or Alaskan Native

Grade 8

559

0.72

 

Split Half Reliability (Fall)

Asian

Grade 8

973

0.74

 

Split Half Reliability (Fall)

Two or more races

Grade 8

2,152

0.73

 

Split Half Reliability (Fall)

Black or African American

Grade 8

3,777

0.76

 

Split Half Reliability (Fall)

Native Hawaiian or Other Pacific Islander

Grade 8

124

0.76

 

Split Half Reliability (Fall)

Unknown race/ethnicity

Grade 8

2,472

0.76

 

Split Half Reliability (Fall)

Male

Grade 8

13,580

0.72

 

Split Half Reliability (Fall)

Female

Grade 8

12,672

0.71

 

Cronbach’s Alpha (Winter)

White

Grade 8

13,512

0.64

 

Cronbach’s Alpha (Winter)

American Indian or Alaskan Native

Grade 8

521

0.64

 

Cronbach’s Alpha (Winter)

Asian

Grade 8

809

0.64

 

Cronbach’s Alpha (Winter)

Two or more races

Grade 8

2,216

0.64

 

Cronbach’s Alpha (Winter)

Black or African American

Grade 8

3,320

0.69

 

Cronbach’s Alpha (Winter)

Native Hawaiian or Other Pacific Islander

Grade 8

131

0.63

 

Cronbach’s Alpha (Winter)

Unknown race/ethnicity

Grade 8

2,199

0.62

 

Cronbach’s Alpha (Winter)

Male

Grade 8

10,752

0.66

 

Cronbach’s Alpha (Winter)

Female

Grade 8

11,940

0.63

 

Split Half Reliability (Winter)

White

Grade 8

13,512

0.66

 

Split Half Reliability (Winter)

American Indian or Alaskan Native

Grade 8

521

0.68

 

Split Half Reliability (Winter)

Asian

Grade 8

809

0.69

 

Split Half Reliability (Winter)

Two or more races

Grade 8

2,216

0.66

 

Split Half Reliability (Winter)

Black or African American

Grade 8

3,320

0.7

 

Split Half Reliability (Winter)

Native Hawaiian or Other Pacific Islander

Grade 8

131

0.69

 

Split Half Reliability (Winter)

Unknown race/ethnicity

Grade 8

2,199

0.66

 

Split Half Reliability (Winter)

Male

Grade 8

10,752

0.68

 

Split Half Reliability (Winter)

Female

Grade 8

11,940

0.66

 

Cronbach’s Alpha (Spring)

White

Grade 8

14,445

0.67

 

Cronbach’s Alpha (Spring)

American Indian or Alaskan Native

Grade 8

488

0.7

 

Cronbach’s Alpha (Spring)

Asian

Grade 8

882

0.6

 

Cronbach’s Alpha (Spring)

Two or more races

Grade 8

2,040

0.68

 

Cronbach’s Alpha (Spring)

Black or African American

Grade 8

3,409

0.7

 

Cronbach’s Alpha (Spring)

Native Hawaiian or Other Pacific Islander

Grade 8

117

0.65

 

Cronbach’s Alpha (Spring)

Unknown race/ethnicity

Grade 8

1,846

0.66

 

Cronbach’s Alpha (Spring)

Male

Grade 8

11,943

0.68

 

Cronbach’s Alpha (Spring)

Female

Grade 8

11,237

0.65

 

Split Half Reliability (Spring)

White

Grade 8

14,445

0.7

 

Split Half Reliability (Spring)

American Indian or Alaskan Native

Grade 8

488

0.74

 

Split Half Reliability (Spring)

Asian

Grade 8

882

0.65

 

Split Half Reliability (Spring)

Two or more races

Grade 8

2,040

0.71

 

Split Half Reliability (Spring)

Black or African American

Grade 8

3,409

0.73

 

Split Half Reliability (Spring)

Native Hawaiian or Other Pacific Islander

Grade 8

117

0.67

 

Split Half Reliability (Spring)

Unknown race/ethnicity

Grade 8

1,846

0.69

 

Split Half Reliability (Spring)

Male

Grade 8

11,943

0.71

 

Split Half Reliability (Spring)

Female

Grade 8

11,237

0.68

 

 

Validity

Grade345678
RatingHalf-filled bubbledFull bubbledFull bubbledHalf-filled bubbledHalf-filled bubbledHalf-filled bubbled

Describe and justify the criterion measures used to demonstrate validity:

The reading portion of the Washington state summative test was used for all criterion-related validity evidence. Washington is part of the Smarter Balanced Assessment Consortium, and the corresponding state test was used.

 

Describe the sample characteristics for each validity analysis conducted:

Across grades, approximately 40–48% of students were male (7–21% missing data); 8–10% of students received special education services; 23–31% received free or reduced price lunch; and 6–10% received English language services.

 

Describe the analysis procedures for each reported type of validity:

Scores from each measure were correlated with their corresponding Smarter Balanced assessment scores. See pages 34–35 in the article, “The Relation Between easyCBM and Smarter Balanced Reading and Mathematics Assessments,” for plots of the relation.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Predictive criterion validity
(Fall)

Grade 3

Smarter Balanced Assessment Consortium (SBAC)

1239

0.62

0.58-0.67

Predictive criterion validity
(Winter)

Grade 3

SBAC

1285

0.63

0.59-0.67

Concurrent criterion validity (Spring)

Grade 3

SBAC

1288

0.68

0.64-0.72

Predictive criterion validity
(Fall)

Grade 4

SBAC

1443

0.68

0.65-0.72

Predictive criterion validity
(Winter)

Grade 4

SBAC

1483

0.64

0.60-0.68

Concurrent criterion validity
(Spring)

Grade 4

SBAC

1307

0.62

0.58-0.66

Predictive criterion validity
(Fall)

Grade 5

SBAC

1531

0.67

0.63-0.71

Predictive criterion validity
(Winter)

Grade 5

SBAC

1570

0.68

0.64-0.71

Concurrent criterion validity
(Spring)

Grade 5

SBAC

1333

0.64

0.60-0.68

Predictive criterion validity
(Fall)

Grade 6

SBAC

1505

0.60

0.55-0.64

Predictive criterion validity
(Winter)

Grade 6

SBAC

1554

0.54

0.50-0.58

Concurrent criterion validity

(Spring)

Grade 6

SBAC

1297

0.65

0.61-0.69

Predictive criterion validity
(Fall)

Grade 7

SBAC

1433

0.65

0.61-0.69

Predictive criterion validity
(Winter)

Grade 7

SBAC

1476

0.59

0.55-0.63

Concurrent criterion validity (Spring)

Grade 7

SBAC

1200

0.62

0.58-0.67

Predictive criterion validity
(Fall)

Grade 8

SBAC

1475

0.63

0.59-0.67

Predictive criterion validity
(Winter)

Grade 8

SBAC

1535

0.58

0.54-0.63

Concurrent criterion validity (Spring)

Grade 8

SBAC

1250

0.64

0.60-0.68

 

Describe the degree to which the provided data support the validity of the tool:

Across grades, the concurrent and predictive validity evidence is strong, ranging from 0.54 to 0.68. These values correspond to the easyCBM measures accounting for approximately 29–46% of the total variance in the statewide assessment.

 

Disaggregated Validity Data

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

(R2)

Confidence Interval

Concurrent

American Indian/Alaskan Native

Grade 3

Multiple Choice Reading Comprehension (MCRC)

36

0.30

 

Concurrent

Asian/Pacific Islander

Grade 3

MCRC

94

0.34

 

Concurrent

Black

Grade 3

MCRC

46

0.45

 

Concurrent

Hispanic

Grade 3

MCRC

373

0.30

 

Concurrent

White

Grade 3

MCRC

1656

0.35

 

Concurrent

Multiethnic

Grade 3

MCRC

60

0.53

 

Concurrent

American Indian/Alaskan Native

Grade 4

MCRC

45

0.59

 

Concurrent

Asian/Pacific Islander

Grade 4

MCRC

91

0.16

 

Concurrent

Black

Grade 4

MCRC

52

0.44

 

Concurrent

Hispanic

Grade 4

MCRC

384

0.31

 

Concurrent

White

Grade 4

MCRC

1596

0.35

 

Concurrent

Multiethnic

Grade 4

MCRC

96

0.42

 

Concurrent

American Indian/Alaskan Native

Grade 5

MCRC

49

0.38

 

Concurrent

Asian/Pacific Islander

Grade 5

MCRC

104

0.18

 

Concurrent

Black

Grade 5

MCRC

56

0.26

 

Concurrent

Hispanic

Grade 5

MCRC

379

0.27

 

Concurrent

White

Grade 5

MCRC

1668

0.30

 

Concurrent

Multiethnic

Grade 5

MCRC

89

0.20

 

Concurrent

American Indian/Alaskan Native

Grade 6

MCRC

54

0.37

 

Concurrent

Asian/Pacific Islander

Grade 6

MCRC

84

0.27

 

Concurrent

Black

Grade 6

MCRC

53

0.20

 

Concurrent

Hispanic

Grade 6

MCRC

241

0.36

 

Concurrent

White

Grade 6

MCRC

1584

0.29

 

Concurrent

Multiethnic

Grade 6

MCRC

81

0.20

 

Concurrent

American Indian/Alaskan Native

Grade 7

MCRC

33

0.43

 

Concurrent

Asian/Pacific Islander

Grade 7

MCRC

182

0.33

 

Concurrent

Black

Grade 7

MCRC

75

0.26

 

Concurrent

Hispanic

Grade 7

MCRC

649

0.35

 

Concurrent

White

Grade 7

MCRC

2118

0.34

 

Concurrent

Multiethnic

Grade 7

MCRC

120

0.35

 

Concurrent

American Indian/Alaskan Native

Grade 8

MCRC

36

0.20

 

Concurrent

Asian/Pacific Islander

Grade 8

MCRC

161

0.41

 

Concurrent

Black

Grade 8

MCRC

87

0.24

 

Concurrent

Hispanic

Grade 8

MCRC

715

0.37

 

Concurrent

White

Grade 8

MCRC

2132

0.32

 

Concurrent

Multiethnic

Grade 8

MCRC

102

0.29

 

Predictive (Fall)

American Indian/Alaskan Native

Grade 3

MCRC

35

0.38

 

Predictive (Fall)

Asian/Pacific Islander

Grade 3

MCRC

92

0.29

 

Predictive (Fall)

Black

Grade 3

MCRC

43

0.32

 

Predictive (Fall)

Hispanic

Grade 3

MCRC

360

0.25

 

Predictive (Fall)

White

Grade 3

MCRC

1615

0.32

 

Predictive (Fall)

Multiethnic

Grade 3

MCRC

60

0.25

 

Predictive (Fall)

American Indian/Alaskan Native

Grade 4

MCRC

45

0.35

 

Predictive (Fall)

Asian/Pacific Islander

Grade 4

MCRC

84

0.33

 

Predictive (Fall)

Black

Grade 4

MCRC

49

0.40

 

Predictive (Fall)

Hispanic

Grade 4

MCRC

380

0.41

 

Predictive (Fall)

White

Grade 4

MCRC

1555

0.44

 

Predictive (Fall)

Multiethnic

Grade 4

MCRC

93

0.56

 

Predictive (Fall)

American Indian/Alaskan Native

Grade 5

MCRC

47

0.36

 

Predictive (Fall)

Asian/Pacific Islander

Grade 5

MCRC

97

0.43

 

Predictive (Fall)

Black

Grade 5

MCRC

51

0.37

 

Predictive (Fall)

Hispanic

Grade 5

MCRC

422

0.24

 

Predictive (Fall)

White

Grade 5

MCRC

1660

0.30

 

Predictive (Fall)

Multiethnic

Grade 5

MCRC

89

0.30

 

Predictive (Fall)

American Indian/Alaskan Native

Grade 6

MCRC

56

0.38

 

Predictive (Fall)

Asian/Pacific Islander

Grade 6

MCRC

89

0.46

 

Predictive (Fall)

Black

Grade 6

MCRC

53

0.22

 

Predictive (Fall)

Hispanic

Grade 6

MCRC

242

0.42

 

Predictive (Fall)

White

Grade 6

MCRC

1602

0.27

 

Predictive (Fall)

Multiethnic

Grade 6

MCRC

81

0.39

 

Predictive (Fall)

American Indian/Alaskan Native

Grade 7

MCRC

32

0.36

 

Predictive (Fall)

Asian/Pacific Islander

Grade 7

MCRC

182

0.39

 

Predictive (Fall)

Black

Grade 7

MCRC

68

0.56

 

Predictive (Fall)

Hispanic

Grade 7

MCRC

621

0.42

 

Predictive (Fall)

White

Grade 7

MCRC

2116

0.37

 

Predictive (Fall)

Multiethnic

Grade 7

MCRC

120

0.55

 

Predictive (Fall)

American Indian/Alaskan Native

Grade 8

MCRC

37

0.29

 

Predictive (Fall)

Asian/Pacific Islander

Grade 8

MCRC

158

0.40

 

Predictive (Fall)

Black

Grade 8

MCRC

85

0.32

 

Predictive (Fall)

Hispanic

Grade 8

MCRC

699

0.32

 

Predictive (Fall)

White

Grade 8

MCRC

2153

0.30

 

Predictive (Fall)

Multiethnic

Grade 8

MCRC

105

0.35

 

Predictive (Winter)

American Indian/Alaskan Native

Grade 3

MCRC

36

0.31

 

Predictive (Winter)

Asian/Pacific Islander

Grade 3

MCRC

95

0.36

 

Predictive (Winter)

Black

Grade 3

MCRC

48

0.41

 

Predictive (Winter)

Hispanic

Grade 3

MCRC

398

0.21

 

Predictive (Winter)

White

Grade 3

MCRC

1700

0.27

 

Predictive (Winter)

Multiethnic

Grade 3

MCRC

64

0.33

 

Predictive (Winter)

American Indian/Alaskan Native

Grade 4

MCRC

45

0.52

 

Predictive (Winter)

Asian/Pacific Islander

Grade 4

MCRC

86

0.25

 

Predictive (Winter)

Black

Grade 4

MCRC

49

0.41

 

Predictive (Winter)

Hispanic

Grade 4

MCRC

399

0.21

 

Predictive (Winter)

White

Grade 4

MCRC

1582

0.29

 

Predictive (Winter)

Multiethnic

Grade 4

MCRC

98

0.34

 

Predictive (Winter)

American Indian/Alaskan Native

Grade 5

MCRC

48

0.32

 

Predictive (Winter)

Asian/Pacific Islander

Grade 5

MCRC

106

0.22

 

Predictive (Winter)

Black

Grade 5

MCRC

56

0.28

 

Predictive (Winter)

Hispanic

Grade 5

MCRC

400

0.29

 

Predictive (Winter)

White

Grade 5

MCRC

1680

0.26

 

Predictive (Winter)

Multiethnic

Grade 5

MCRC

90

0.26

 

Predictive (Winter)

American Indian/Alaskan Native

Grade 6

MCRC

33

0.48

 

Predictive (Winter)

Asian/Pacific Islander

Grade 6

MCRC

44

0.31

 

Predictive (Winter)

Black

Grade 6

MCRC

33

0.48

 

Predictive (Winter)

Hispanic

Grade 6

MCRC

137

0.32

 

Predictive (Winter)

White

Grade 6

MCRC

802

0.16

 

Predictive (Winter)

Multiethnic

Grade 6

MCRC

48

0.30

 

Predictive (Winter)

American Indian/Alaskan Native

Grade 7

MCRC

16

0.07

 

Predictive (Winter)

Asian/Pacific Islander

Grade 7

MCRC

120

0.34

 

Predictive (Winter)

Black

Grade 7

MCRC

46

0.51

 

Predictive (Winter)

Hispanic

Grade 7

MCRC

532

0.38

 

Predictive (Winter)

White

Grade 7

MCRC

1228

0.31

 

Predictive (Winter)

Multiethnic

Grade 7

MCRC

62

0.43

 

Predictive (Winter)

American Indian/Alaskan Native

Grade 8

MCRC

25

0.48

 

Predictive (Winter)

Asian/Pacific Islander

Grade 8

MCRC

110

0.39

 

Predictive (Winter)

Black

Grade 8

MCRC

59

0.38

 

Predictive (Winter)

Hispanic

Grade 8

MCRC

582

0.35

 

Predictive (Winter)

White

Grade 8

MCRC

1229

0.34

 

Predictive (Winter)

Multiethnic

Grade 8

MCRC

35

0.57

 

 

Bias Analysis Conducted

Grade345678
RatingYesYesYesYesYesYes

Have additional analyses been conducted to establish whether the tool is or is not biased against demographic subgroups (e.g., students who vary by race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)?

Bias Analysis Method:

To investigate Differential Item Functioning (DIF) of the easyCBM reading measures, the Mantel-Haenszel test using a purification process was conducted (Holland & Thayer, 1988; Kamata & Vaughn, 2004) with the R software using the difR package (Magis et al., 2013). When using the Mantel-Haenszel test to investigate DIF, contingency tables are constructed, and the resulting odds for the focal group answering the item correctly are compared to the odds for the reference group.

The contingency table summarizes correct and incorrect responses to each item by respondents’ total raw score by subgroup (Kamata & Vaughn, 2004). If there is no difference in performance for the two groups, the odds ratio of the focal group performance to reference group performance will equal one. An odds ratio greater than one means the focal group is performing better than the reference group, with the opposite being true for odds ratios less than one.

The difR package contains a built-in algorithm to conduct purification automatically. We used criteria outlined by the Educational Testing Service (ETS) for DIF Classification (Holland & Thayer, 1988) to determine whether or not items exhibited DIF, as the difR package reports delta values by default.

 

The Holland and Thayer criteria were used for all Mantel-Haenszel analyses. It was determined a priori that any items that were flagged as “B” level items would be reviewed by the item development team for potential biases and possible removal. All items flagged as “C” level items would be removed from the operational pool. All operational items from 2016-17 were included in the analyses.

 

Subgroups Included:

The easyCBM MCRC DIF analyses compared Male/Female, White/Non-White, and Non-Hispanic-Latino/Hispanic-Latino subgroups. Male, White, and Non-Hispanic-Latino were the reference groups, respectively. Sample sizes ranged from 23,000–35,000 students. Full sample statistics are published in the technical report.

 

Bias Analysis Results:

Overall, easyCBM reading items are functioning in a manner that appears not to exhibit bias based on the demographic characteristics reviewed. Across all MCRC grade levels, no items were assigned to the “C” level, denoting large effect size differences between subgroups. In the Male/Female comparisons, 98% of items were assigned to the “A” level DIF category (352/360). In the White/Non-White comparisons, 99% of items were assigned to the “A” level DIF category (358/360). In the Non-Hispanic-Latino/ Hispanic-Latino comparisons, 100% of items were assigned to the “A” level DIF category (360/360). The “A” level categorizations are labeled as negligible DIF by the ETS delta criteria.

Sensitivity: Reliability of the Slope

Grade345678
Ratingdashddashddashddashddashddashd

Describe the sample used for analyses, including size and characteristics:

No qualifying evidence provided.

 

Describe the frequency of measurement:

No qualifying evidence provided.

 

Describe reliability of the slope analyses conducted with a population of students in need of intensive intervention:

No qualifying evidence provided.

 

Disaggregated Reliability Data

Type of Reliability

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Slope

Multi-Ethnic

Grade 3

Multiple Choice Reading Comprehension (MCRC)

15

0.70

 

Slope

White

Grade 3

MCRC

422

0.63

 

Slope

Hispanic

Grade 3

MCRC

151

0.46

 

Slope

Black

Grade 3

MCRC

11

0.77

 

Slope

Asian

Grade 3

MCRC

14

0.00

 

Slope

American Indian/Alaskan Native

Grade 3

MCRC

10

0.05

 

Slope

Multi-Ethnic

Grade 4

MCRC

28

0.47

 

Slope

White

Grade 4

MCRC

361

0.44

 

Slope

Hispanic

Grade 4

MCRC

155

0.43

 

Slope

Black

Grade 4

MCRC

19

0.02

 

Slope

Asian

Grade 4

MCRC

19

0.30

 

Slope

American Indian/Alaskan Native

Grade 4

MCRC

11

0.50

 

Slope

Multi-Ethnic

Grade 5

MCRC

19

0.22

 

Slope

White

Grade 5

MCRC

387

0.17

 

Slope

Black

Grade 5

MCRC

23

0.03

 

Slope

Asian

Grade 5

MCRC

28

0.14

 

Slope

American Indian/Alaskan Native

Grade 5

MCRC

19

0.15

 

Slope

Multi-Ethnic

Grade 6

MCRC

12

0.28

 

Slope

White

Grade 6

MCRC

175

0.15

 

Slope

Hispanic

Grade 6

MCRC

65

0.24

 

Slope

Black

Grade 6

MCRC

8

0.56

 

Slope

Asian

Grade 6

MCRC

5

0.02

 

Slope

American Indian/Alaskan Native

Grade 6

MCRC

9

0.28

 

Slope

Multi-Ethnic

Grade 7

MCRC

26

0.12

 

Slope

White

Grade 7

MCRC

265

0.08

 

Slope

Hispanic

Grade 7

MCRC

203

0.01

 

Slope

Black

Grade 7

MCRC

17

0.10

 

Slope

Asian

Grade 7

MCRC

30

0.42

 

Slope

American Indian/Alaskan Native

Grade 7

MCRC

10

0.02

 

Slope

Multi-Ethnic

Grade 8

MCRC

6

0.65

 

Slope

White

Grade 8

MCRC

304

0.19

 

Slope

Hispanic

Grade 8

MCRC

265

0.22

 

Slope

Black

Grade 8

MCRC

24

0.00

 

Slope

Asian

Grade 8

MCRC

23

0.15

 

Slope

American Indian/Alaskan Native

Grade 8

MCRC

12

0.22

 

 

Sensitivity: Validity of the Slope

Grade345678
RatingEmpty bubbledFull bubbledFull bubbledFull bubbledFull bubbledFull bubbled

Describe and justify the criterion measures used to demonstrate validity:

The reading portion of the Washington summative state test was used for all criterion-related validity evidence. Washington is part of the Smarter Balanced Assessment Consortium, and the corresponding state test was used.

 

Describe the sample used for analyses, including size and characteristics:

Across grades, approximately 40–48% of students were male (7–21% missing data); 8–10% of students received special education services; 23–31% received free or reduced price lunch; and 6–10% received English language services.

 

Describe predictive validity of the slope of improvement analyses conducted with a population of students in need of intensive intervention:

Data were split into quartiles of normative achievement based on students’ fall benchmark. A two-level growth model was then fit to the data for each quartile within each grade, with measurement occasions nested in students. The random effect estimates for the students’ slope were then extracted and correlated with their corresponding Smarter Balanced Assessment scores. We conducted analyses separately by quartile to ensure students’ growth was referenced to students with similar initial achievement.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Slope

Grade 3

MCRC

1138

0.58

0.53 - 0.63

Slope

Grade 4

MCRC

1090

0.49

0.44 - 0.54

Slope

Grade 5

MCRC

1233

0.59

0.55 - 0.64

Slope

Grade 6

MCRC

1043

0.53

0.48 - 0.58

Slope

Grade 7

MCRC

1037

0.53

0.48 - 0.58

Slope

Grade 8

MCRC

1103

0.62

0.57 - 0.67

Predictive

Grade 2

MCRC

568

0.68

 

Predictive

Grade 3

MCRC

666

0.56

 

Predictive

Grade 4

MCRC

641

0.53

 

Predictive

Grade 5

MCRC

682

0.61

 

Predictive

Grade 6

MCRC

333

0.60

 

Predictive

Grade 7

MCRC

576

0.63

 

Predictive

Grade 8

MCRC

677

0.64

 

 

Describe the degree to which the provided data support the validity of the tool:

Typically, measures of growth correlate far less strongly with criterion measures than do measures of status. When comparing growth estimates within quartiles, and thus to students with similar initial achievement, the correlations were moderate, providing good evidence for the validity of the slope for informing instructional decisions.

Disaggregated Validity Data

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Predictive

Multi-Ethnic

Grade 3

MCRC

15

0.25

 

Predictive

White

Grade 3

MCRC

424

0.57

 

Predictive

Hispanic

Grade 3

MCRC

152

0.53

 

Predictive

Black

Grade 3

MCRC

11

0.74

 

Predictive

Asian

Grade 3

MCRC

14

0.53

 

Predictive

American Indian/Alaskan Native

Grade 3

MCRC

10

0.00

 

Predictive

Multi-Ethnic

Grade 4

MCRC

28

0.46

 

Predictive

White

Grade 4

MCRC

365

0.53

 

Predictive

Hispanic

Grade 4

MCRC

159

0.49

 

Predictive

Black

Grade 4

MCRC

19

0.52

 

Predictive

Asian

Grade 4

MCRC

20

0.46

 

Predictive

American Indian/Alaskan Native

Grade 4

MCRC

11

0.90

 

Predictive

Multi-Ethnic

Grade 5

MCRC

19

0.55

 

Predictive

White

Grade 5

MCRC

387

0.62

 

Predictive

Black

Grade 5

MCRC

24

0.40

 

Predictive

Asian

Grade 5

MCRC

28

0.58

 

Predictive

American Indian/Alaskan Native

Grade 5

MCRC

19

0.74

 

Predictive

Multi-Ethnic

Grade 6

MCRC

12

0.55

 

Predictive

White

Grade 6

MCRC

180

0.62

 

Predictive

Hispanic

Grade 6

MCRC

65

0.65

 

Predictive

Black

Grade 6

MCRC

8

0.23

 

Predictive

Asian

Grade 6

MCRC

5

-0.80

 

Predictive

American Indian/Alaskan Native

Grade 6

MCRC

9

0.71

 

Predictive

Multi-Ethnic

Grade 7

MCRC

26

0.81

 

Predictive

White

Grade 7

MCRC

267

0.62

 

Predictive

Hispanic

Grade 7

MCRC

209

0.60

 

Predictive

Black

Grade 7

MCRC

17

0.74

 

Predictive

Asian

Grade 7

MCRC

30

0.58

 

Predictive

American Indian/Alaskan Native

Grade 7

MCRC

10

0.76

 

 

Alternate Forms

Grade345678
RatingEmpty bubbleEmpty bubbleEmpty bubbleEmpty bubbleEmpty bubbleEmpty bubble

Describe the sample for these analyses, including size and characteristics:

No qualifying evidence provided.

 

Evidence that alternate forms are of equal and controlled difficulty or, if IRT based, evidence of item or ability invariance:

Initially, items were piloted using a common person / common item design to create an item bank with known item parameters (measure, mean square outfit, standard error, etc.) Using this data, the authors then distributed items across the multiple forms (3 screening forms to be administered in the fall, winter, and spring and 17 progress monitoring) to have approximately equal item measure estimates and comparable ranges. The comparability of each of the alternate forms was tested with grade-level students, using repeated measures ANOVA to test for form differences. Results of these studies are reported in the technical reports documenting the development of the measures:                                                                                          

Alonzo, J., Liu, K., & Tindal, G. (2008). Examining the technical adequacy of second-grade reading comprehension measures in a progress monitoring assessment system (Technical Report No. 0808). Eugene, OR: Behavioral Research and Teaching, University of Oregon.
TechRpt_0808

Alonzo, J., Liu, K., & Tindal, G. (2007). Examining the technical adequacy of reading comprehension measures in a progress monitoring assessment system (Technical Report
No. 41). Eugene, OR: Behavioral Research and Teaching, University of Oregon. TechRpt_41

Alonzo, J., & Tindal, G. (2008). Examining the technical adequacy of fifth-grade reading comprehension measures in a progress monitoring assessment system (Technical Report No. 0807). Eugene, OR: Behavioral Research and Teaching, University of Oregon.TechRpt_0807

Alonzo, J., & Tindal, G. (2009). Alternate form and test-retest reliability of easyCBM® reading measures (Technical Report No. 0906). Eugene, OR: Behavioral Research and Teaching, University of Oregon.

 

Number of alternate forms of equal and controlled difficulty:

3 screening forms to be administered in the fall, winter, and spring and 17 progress monitoring forms.

Decision Rules: Setting and Revising Goals

Grade345678
Ratingdashdashdashdashdashdash

Specification of validated decision rules for when goals should be set or revised:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Decision Rules: Changing Instruction

Grade345678
Ratingdashdashdashdashdashdash

Specification of validated decision rules for when changes to instruction should be made:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Administration Format

Grade345678
Data
  • Individual
  • Group
  • Computer-administered
  • Individual
  • Group
  • Computer-administered
  • Individual
  • Group
  • Computer-administered
  • Individual
  • Group
  • Computer-administered
  • Individual
  • Group
  • Computer-administered
  • Individual
  • Group
  • Computer-administered
  • Administration & Scoring Time

    Grade345678
    Data
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • 30 minutes
  • Scoring Format

    Grade345678
    Data
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • ROI & EOY Benchmarks

    Grade345678
    Data
  • EOY Benchmarks Available
  • EOY Benchmarks Available
  • EOY Benchmarks Available
  • EOY Benchmarks Available
  • EOY Benchmarks Available
  • EOY Benchmarks Available
  • Specify the minimum acceptable rate of growth/improvement:

    No qualifying evidence provided.

     

    Specify the benchmarks for minimum acceptable end-of-year performance:

    Spring norms based on representative national sample (see easyCBM Norms, 2014 Edition, available upon request from the Center.)