Children’s Education Services, Inc. (formerly EdCheckup)

Standard Reading Passages

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

Each “set” allows you to assess up to 35 students.

  • 1 – 10 sets: $100
  • 11 – 20 sets: $95
  • 21 – 100 sets: $90
  • 101 – 200 sets: $85
  • 201 + sets: $80

 

Replacement Cost:

No information provided; contact vendor for details.

 

Included in Cost:

No information provided; contact vendor for details.

Technology Requirements:

  • Computer or tablet
  • Internet connection

 

Training Requirements:

  • Less than 1 hour of training

 

Qualified Administrators:

  • Paraprofessionals
  • Professionals

 

Accommodations:

No information provided; contact vendor for details.

Where to Obtain:

Website:

https://www.ces-schools.net/

Address:

12276 San Jose Blvd, Ste 528, Jacksonville, Florida 32223

Phone Number:

904.652.1282

Email:
info@ces-schools.net


Access to Technical Support:

Email, phone, limited on-site.

The Standard Reading Passages system comprises a web-based system for teachers and other school personnel to use in screening to establish student achievement in reading, to set annual goals for those students, to monitor their progress and to evaluate the effectiveness of instruction for the purpose of modifying instruction when it is not leading to desired outcomes. Guides are provided to teachers to assist them in making judgments about the degree of student progress and the need to make program changes when success is not being achieved.

 

The system includes online capabilities for downloading materials, procedures for administration, scoring and data entry. Teachers are enabled to create class lists and use the online system to represent student progress graphically and create both individual and group reports. Standard Reading Passages also enables aggregation of classroom level data to produce summary reports at various administrative levels. The essential procedures are the same as those developed by researchers at the University of Minnesota beginning in 1977 and the years following. The reading passages were initially developed in 1988 and disseminated as the Standard Reading Passages.

Assessment Format:

  • Individual

 

Administration Time:

  • 2 minutes per student

 

Scoring Time:

  • 1 minute per student

 

Scoring Method:

  • Calculated manually
  • Calculated automatically

 

Scores Generated:

  • Raw Score

 

Reliability

Grade123456
RatingHalf-filled bubbleHalf-filled bubbledHalf-filled bubbledHalf-filled bubbledHalf-filled bubbledHalf-filled bubbled

Justify the appropriateness of each type of reliability reported:

No qualifying evidence provided.

 

Describe the sample characteristics for each reliability analysis conducted:

No qualifying evidence provided.

 

Describe the analysis procedures for each reported type of reliability:

No qualifying evidence provided.

 

Type of Reliability

Age or Grade

n

Coefficient

Confidence Interval

Internal Consistency

1

89

0.9446

0.917-0.963

Internal Consistency

2

216

0.9016

0.873-0.924

Internal Consistency

3

227

0.9152

0.891-0.934

Internal Consistency

4

206

0.9204

0.896-0.939

Internal Consistency

5

255

0.886

0.856-0.910

Internal Consistency

6

78

0.9046

0.854-0.938

Alternate Form

2

216

0.902

0.854-0.912

Alternate Form

3

227

0.914

0.890-0.933

Alternate Form

4

206

0.92

0.896-0.939

Alternate Form

5

255

0.886

0.856-0.910

Alternate Form

6

78

0.905

0.855-0.938

 

Disaggregated Reliability Data:

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

Confidence Interval

Internal Consistency

Black

2

19

0.9633

0.905-0.986

Internal Consistency

Black

3

20

0.8173

0.588-0.925

Internal Consistency

Black

4

25

0.8989

0.781-0.955

Internal Consistency

Black

5

25

0.9241

0.833-0.966

Internal Consistency

Black

6

9

0.9825

0.916-0.996

Cronbach’s Alpha

African American

2

19

0.9633

 

Cronbach’s Alpha

African American

3

20

0.8173

 

Cronbach’s Alpha

African American

4

25

0.8989

 

Cronbach’s Alpha

African American

5

25

0.9241

 

Cronbach’s Alpha

African American

6

9

0.9825

 

Alternate Form

African American

2

19

0.963

 

Alternate Form

African American

3

20

0.817

 

Alternate Form

African American

4

25

0.899

 

Alternate Form

African American

5

25

0.924

 

Alternate Form

African American

6

9

0.983

 

 

Validity

Grade123456
RatingFull bubbledFull bubbledFull bubbledFull bubbledFull bubbledHalf-filled bubbled

Describe and justify the criterion measures used to demonstrate validity:

The Measures of Academic Progress (MAP) was used as the outcome measure. Published by the NWEA the MAP is regarded as a highly valid and reliable measure of broad reading ability.  The NWEA website states, “Our tools are trusted by educators in 140 countries and more than half the schools in the US” which indicates it can be considered an excellent outcome measure for classification studies on the Oral Reading measure studied here. See  https://www.nwea.org/normative-data-rit-scores/ for more information. The MAP is an external measure.

A second criterion used in our analysis is the Minnesota Comprehensive Assessment (MCA) and reading. The MCA is the state accountability test for Minnesota and has been established by the Minnesota Department of Education to be a highly reliable and valid measure or reading. See http://education.state.mn.us/MDE/dse/test/mn/Tech/ for more technical information. The MCA is an external measure.

 

Describe the sample characteristics for each validity analysis conducted:

For the Concurrent and Predictive validity analyses for Passage Reading and MAP there was a total sample of 9,410 students (grades 1-6). Demographic data indicated 48.4% of the sample was female, 12.4% was special education, 24.0% received Title services, 7.5% was served in a gifted program, 38.2% received free or reduced lunch, and 5.1% received ELL service. Ethnic percentages for this sample were 1.3% for American Indian, 4.9% for Asian American, 10.3% for Hispanic American, 7.7% for Black American, and 75.8% for White American.

For the Concurrent and Predictive validity analyses for Passage Reading and MCA there was a total sample of 11,094 students (grades 3-6). Demographic data indicated 48.2% of the sample was female, 12.4% was special education, 27.2% received Title services, 9.3% was served in a gifted program, 27.6% received free or reduced lunch, and 7.7% received ELL service. Ethnic percentages for this sample were 2.6% for American Indian, 4.0% for Asian American, 9.4% for Hispanic American, 12.2% for Black American, and 71.8% for White American.

 

Describe the analysis procedures for each reported type of validity:

Two types of validity analysis were conducted; Concurrent and Predictive.  The Concurrent study used Pearson Product Moment Correlational analysis to examine the relationship between Passage Reading (Words Read Correctly) and the MAP and the MCA.  All measures were administered in the Spring of 2017. The Predictive study used Pearson Product Moment Correlational analysis to examine the relationship between Passage Reading (Words Read Correctly) and the MAP and the MCA.  Passage Reading was administered in the Winter of 2017 and the criterion measures of MAP and MCA were administered in the Spring of 2017, approximately three months later.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Concurrent (Spring)

1

MAP (Spring)

1212

0.75

0.72-0.77

Concurrent (Spring)

2

MAP (Spring)

2038

0.80

0.78-0.82

Concurrent (Spring)

3

MAP (Spring)

2108

0.76

0.73-.77

Concurrent (Spring)

4

MAP (Spring)

2004

0.72

0.70-0.74

Concurrent (Spring)

5

MAP (Spring)

1699

0.66

0.63-0.69

Concurrent (Spring)

6

MAP (Spring)

372

0.62

0.55-0.68

Concurrent (Spring)

3

MCA (Spring)

3751

0.77

0.76-0.78

Concurrent (Spring)

4

MCA (Spring)

3709

0.74

0.73-0.75

Concurrent (Spring)

5

MCA (Spring)

3053

0.69

0.67-0.71

Concurrent (Spring)

6

MCA (Spring)

634

0.68

0.64-0.72

Predictive (Winter)

1

MAP (Spring)

1191

0.71

0.68-0.74

Predictive (Winter)

2

MAP (Spring)

2024

0.79

0.77-0.81

Predictive (Winter)

3

MAP (Spring)

1972

0.75

0.73-0.77

Predictive (Winter)

4

MAP (Spring)

2069

0.69

0.67-0.71

Predictive (Winter)

5

MAP (Spring)

1736

0.65

0.62-0.68

Predictive (Winter)

6

MAP (Spring)

487

0.58

0.52-0.64

Predictive (Winter)

3

MCA (Spring)

3638

0.76

0.75-0.77

Predictive (Winter)

4

MCA (Spring)

3770

0.73

0.72-0.75

Predictive (Winter)

5

MCA (Spring)

3195

0.68

0.66-0.70

Predictive (Winter)

6

MCA (Spring)

487

0.58

0.52-0.64

 

Describe the degree to which the provided data support the validity of the tool:

Validity coefficients for both concurrent and predictive validity are typically high. The median correlation coefficient for concurrent validity is .73 and is .70 for predictive validity.

 

Disaggregated Validity Data

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Concurrent (Spring)

Black

1

MAP (Spring)

80

0.72

0.60-0.81

Concurrent (Spring)

Black

2

MAP (Spring)

208

0.78

0.72-0.83

Concurrent (Spring)

Black

3

MAP (Spring)

133

0.7

0.60-0.78

Concurrent (Spring)

Black

4

MAP (Spring)

146

0.75

0.67-0.81

Concurrent (Spring)

Black

5

MAP (Spring)

118

0.62

0.50-0.72

Concurrent (Spring)

Black

3

MCA (Spring)

452

0.79

0.75-0.82

Concurrent (Spring)

Black

4

MCA (Spring)

497

0.74

0.70-0.78

Concurrent (Spring)

Black

5

MCA (Spring)

361

0.70

0.64-0.75

Predictive (Winter)

Black

1

MAP (Spring)

78

0.72

0.59-0.81

Predictive (Winter)

Black

2

MAP (Spring)

205

0.80

0.75-0.85

Predictive (Winter)

Black

3

MAP (Spring)

127

0.67

0.56-0.76

Predictive (Winter)

Black

4

MAP (Spring)

155

0.70

0.61-0.77

Predictive (Winter)

Black

5

MAP (Spring)

113

0.59

0.46-0.70

Predictive (Winter)

Black

3

MCA (Spring)

436

0.77

0.73-0.81

Predictive (Winter)

Black

4

MCA (Spring)

498

0.71

0.66-0.75

Predictive (Winter)

Black

5

MCA (Spring)

364

0.69

0.63-0.74

Concurrent (Spring)

Hispanic

1

MAP (Spring)

134

0.76

0.68-0.82

Concurrent (Spring)

Hispanic

2

MAP (Spring)

225

0.81

0.76-0.85

Concurrent (Spring)

Hispanic

3

MAP (Spring)

180

0.80

0.74-0.85

Concurrent (Spring)

Hispanic

4

MAP (Spring)

176

0.72

0.64-0.79

Concurrent (Spring)

Hispanic

5

MAP (Spring)

173

0.66

0.57-0.74

Concurrent (Spring)

Hispanic

6

MAP (Spring)

62

0.58

0.39-0.73

Concurrent (Spring)

Hispanic

3

MCA (Spring)

329

0.83

0.79-0.86

Concurrent (Spring)

Hispanic

4

MCA (Spring)

321

0.78

0.73-0.82

Concurrent (Spring)

Hispanic

5

MCA (Spring)

306

0.66

0.59-0.72

Concurrent (Spring)

Hispanic

6

MCA (Spring)

77

0.64

0.49-0.76

Predictive (Winter)

Hispanic

1

MAP (Spring)

134

0.72

0.63-0.79

Predictive (Winter)

Hispanic

2

MAP (Spring)

226

0.79

0.74-0.83

Predictive (Winter)

Hispanic

3

MAP (Spring)

172

0.82

0.76-0.86

Predictive (Winter)

Hispanic

4

MAP (Spring)

175

0.68

0.59-0.75

Predictive (Winter)

Hispanic

5

MAP (Spring)

170

0.65

0.55-0.73

Predictive (Winter)

Hispanic

3

MCA (Spring)

329

0.84

0.81-0.87

Predictive (Winter)

Hispanic

4

MCA (Spring)

319

0.76

0.71-0.80

Predictive (Winter)

Hispanic

5

MCA (Spring)

306

0.66

0.59-0.72

Predictive (Winter)

Hispanic

6

MCA (Spring)

77

0.65

0.50-0.76

 

Bias Analysis Conducted

Grade123456
RatingYesYesYesYesYesYes

Have additional analyses been conducted to establish whether the tool is or is not biased against demographic subgroups (e.g., students who vary by race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)?

Bias Analysis Methods:

The authors first examined bias through the lens of test bias (measurement invariance) analyses using invariance testing in multiple-group confirmatory factor models.

1. Model fit indices for each ethnic group:

Ethnic group (n)

Chi-square (df)

CFI

TLI

RMSEA (90% CI)

White (648)

96.541 (27)

0.993

0.991

0.063 (.050-.077)

African American (646)

99.024 (27)

0.993

0.990

0.064 (.051-.078)

Asian (192)

39.131 (27)

0.996

0.995

0.048 (.000-.080)

Hispanic (50)

38.578 (27)

0.986

0.981

0.093 (.000-.154)

American Indian (53)

42.681 (27)

0.981

0.975

0.105 (.035-.162)

 

 

Overall, every ethnic group’s model fit well. All relative fit indices (CFI, TLI) were above .98, which indicates very good model fit, given that a value over .90 is suggested to be a reasonably good fit for CFI and TLI (Hu & Bentler, 1998). In addition, the 90 percent confidence interval for RMSEA indicates very good model fit, given that a value below or at .05 is considered a very close approximate fit (Browne & Cudeck, 1993). These results show that all CBM-R assessments appeared to function equally well as one factor for all ethnic groups. Below are results for the measurement invariance (test bias) among different ethnic groups.

 

2. Model fit statistics for tests of structural/measurement invariance between White and other groups

 

(a) White versus African American

Model

Chi-square

df

Delta Chi-square

Delta df

Critical value (α= .05)

CFI

RMSEA

(90% CI)

Configural model

(equal structure; baseline)

195.565

54

-

-

-

.993

.064

(.054-.073)

Full metric model

(equal loadings)

212.670

62

17.105

8

15.507

.992

.061

(.052-.070)

Partial metric model* (equal loadings but one loading)

204.222

61

8.657

7

14.067

.993

.060

(.051-.069)

Full scalar model

(equal intercepts)

222.125

70

17.903

9

16.919

.992

.058

(.049-.067)

Partial scalar model

(equal intercepts but one intercept)

212.284

68

8.062

7

14.067

.993

.057

(.049-.066)

* Partial metric/scalar models were applied based on the model modification indices.

 

(b) White versus Asian

Model

Chi-square

df

Delta Chi-square

Delta df

Critical value

CFI

RMSEA

(90% CI)

Configural model

(equal structure; baseline)

135.672

54

-

-

-

.994

.060

(.047-.073)

Full metric model

(equal loadings)

151.275

62

15.603

8

15.507

.993

.059

(.047-.070)

Partial metric model* (equal loadings but one loading)

144.606

61

8.934

7

14.067

.994

.057

(.045-.069)

Full scalar model

(equal intercepts)

165.097

70

20.491

9

16.919

.993

.057

(.046-.068)

Partial scalar model

(equal intercepts but one intercept)

155.250

68

10.644

7

14.067

.993

.055

(.044-.067)

 

(c) White versus Hispanic

Model

Chi-square

df

Delta Chi-square

Delta df

Critical value

CFI

RMSEA

(90% CI)

Configural model

(equal structure; baseline)

135.119

54

-

-

-

.992

.066

(.052-.080)

Full metric model

(equal loadings)

154.734

62

19.615

8

15.507

.991

.065

(.053-.078)

Partial metric model* (equal loadings but one loading)

137.729

60

2.61

6

12.592

.993

.061

(.048-.074)

Full scalar model

(equal intercepts)

165.019

70

27.29

10

18.307

.991

.062

(.050-.075)

Partial scalar model

(equal intercepts but one intercept)

145.049

66

7.32

6

12.592

.993

.059

(.046-.072)

 

(d) White versus American Indian

Model

Chi-square

df

Delta Chi-square

Delta df

Critical value

CFI

RMSEA

(90% CI)

Configural model

(equal structure; baseline)

139.222

54

-

-

-

.992

.067

(.053-.081)

Full metric model

(equal loadings)

146.153

62

6.931

8

15.507

.992

.062

(.049-.075)

Partial metric model* (equal loadings but one loading)

N/A

N/A

N/A

N/A

N/A

N/A

N/A

Full scalar model

(equal intercepts)

155.698

70

9.545

8

15.507

.992

.059

(.047-.072)

Partial scalar model

(equal intercepts but one intercept)

N/A

N/A

N/A

N/A

N/A

N/A

N/A

 

For the first three comparisons (White vs. African American/Asian/Hispanic), the configural (equal structure) model had a good fit, and thus a series of model constraints were then applied in successive models to examine potential decrease in model fit. Two nested models were compared in model fit based on chi-square and degree of freedom differences at α= .05. The full metric invariance model fit well; but resulted in a marginally significant decrease in fit (chi-square difference) relative to the configural model. The partial metric invariance model (freeing just one loading), however, did not result in a significant decrease in fit, which indicates that CBM-R measures seem to be related to the factor equivalently between White and other groups. The full scalar model fit well; but also resulted in a marginally significant decrease in fit relative to the partial metric model. However, the partial scalar model (freeing just one intercept) did not result in a significant decrease in fit, which means that both ethnic groups (with the same level of reading) are expected to perform on CBM-R assessments in a very similar way. In case of the comparison between White and American Indian, both the full metric invariance model and the full scalar model did not result in a significant decrease in fit, which indicates that the same latent factor was being measured for the two groups and they have exactly the same expected performance on CBM-R if their levels are the same.

 

The authors also examined bias by fitting several regression models in which the spring benchmark progress monitoring score was regressed on a score on an external standardized measure of broad reading ability, The Measures of Academic Progress (MAP). In addition to the spring MAP assessment score, the authors included various demographic variables in the model to estimate main effects and potential interaction effects between the demographic variables and the MAP score. The results from these models are presented below by grade and demographic variables.

 

Grade

Term

Estimate

SE

Statistic

p Value

Model

1

(Intercept)

69.64

8.44

8.26

0.00

Ethnicity

1

center_scale(map.spring)

2.96

0.84

3.54

0.00

Ethnicity

1

ethnicAsian

-5.99

9.76

-0.61

0.54

Ethnicity

1

ethnicHispanic

-5.09

9.02

-0.56

0.57

Ethnicity

1

ethnicBlack

0.77

9.45

0.08

0.93

Ethnicity

1

ethnicWhite

-5.43

8.51

-0.64

0.52

Ethnicity

1

center_scale(map.spring):ethnicAsian

-1.49

0.89

-1.67

0.09

Ethnicity

1

center_scale(map.spring):ethnicHispanic

-0.91

0.86

-1.05

0.29

Ethnicity

1

center_scale(map.spring):ethnicBlack

-1.04

0.88

-1.18

0.24

Ethnicity

1

center_scale(map.spring):ethnicWhite

 

-1.02

0.84

-1.21

0.23

Ethnicity

2

(Intercept)

82.97

8.51

9.76

0.00

Ethnicity

2

center_scale(map.spring)

1.75

0.55

3.17

0.00

Ethnicity

2

ethnicAsian

18.56

9.71

1.91

0.06

Ethnicity

2

ethnicHispanic

5.09

8.84

0.58

0.57

Ethnicity

2

ethnicBlack

12.88

9.10

1.41

0.16

Ethnicity

2

ethnicWhite

9.78

8.59

1.14

0.26

Ethnicity

2

center_scale(map.spring):ethnicAsian

0.08

0.63

0.13

0.89

Ethnicity

2

center_scale(map.spring):ethnicHispanic

0.18

0.57

0.31

0.76

Ethnicity

2

center_scale(map.spring):ethnicBlack

-0.13

0.58

-0.23

0.82

Ethnicity

2

center_scale(map.spring):ethnicWhite

 

0.02

0.56

0.04

0.97

Ethnicity

3

(Intercept)

107.93

11.68

9.24

0.00

Ethnicity

3

center_scale(map.spring)

1.64

0.76

2.17

0.03

Ethnicity

3

ethnicAsian

1.29

13.56

0.10

0.92

Ethnicity

3

ethnicHispanic

4.16

12.11

0.34

0.73

Ethnicity

3

ethnicBlack

-0.23

12.34

-0.02

0.99

Ethnicity

3

ethnicWhite

2.51

11.79

0.21

0.83

Ethnicity

3

center_scale(map.spring):ethnicAsian

0.27

0.92

0.30

0.77

Ethnicity

3

center_scale(map.spring):ethnicHispanic

0.47

0.78

0.61

0.54

Ethnicity

3

center_scale(map.spring):ethnicBlack

-0.17

0.79

-0.21

0.83

Ethnicity

3

center_scale(map.spring):ethnicWhite

 

0.27

0.76

0.36

0.72

Ethnicity

4

(Intercept)

107.78

12.29

8.77

0.00

Ethnicity

4

center_scale(map.spring)

1.90

1.29

1.46

0.14

Ethnicity

4

ethnicAsian

25.43

14.03

1.81

0.07

Ethnicity

4

ethnicHispanic

18.86

12.64

1.49

0.14

Ethnicity

4

ethnicBlack

14.45

12.89

1.12

0.26

Ethnicity

4

ethnicWhite

16.62

12.37

1.34

0.18

Ethnicity

4

center_scale(map.spring):ethnicAsian

0.05

1.40

0.04

0.97

Ethnicity

4

center_scale(map.spring):ethnicHispanic

0.34

1.31

0.26

0.79

Ethnicity

4

center_scale(map.spring):ethnicBlack

0.10

1.32

0.07

0.94

Ethnicity

4

center_scale(map.spring):ethnicWhite

 

-0.04

1.30

-0.03

0.98

Ethnicity

5

(Intercept)

143.26

11.58

12.37

0.00

Ethnicity

5

center_scale(map.spring)

2.97

0.93

3.19

0.00

Ethnicity

5

ethnicAsian

8.53

14.14

0.60

0.55

Ethnicity

5

ethnicHispanic

1.19

12.10

0.10

0.92

Ethnicity

5

ethnicBlack

10.47

12.27

0.85

0.39

Ethnicity

5

ethnicWhite

-0.83

11.71

-0.07

0.94

Ethnicity

5

center_scale(map.spring):ethnicAsian

-0.59

1.05

-0.57

0.57

Ethnicity

5

center_scale(map.spring):ethnicHispanic

-1.03

0.96

-1.08

0.28

Ethnicity

5

center_scale(map.spring):ethnicBlack

-0.89

0.97

-0.92

0.36

Ethnicity

5

center_scale(map.spring):ethnicWhite

 

-1.18

0.94

-1.25

0.21

Ethnicity

6

(Intercept)

151.52

30.36

4.99

0.00

Ethnicity

6

center_scale(map.spring)

1.36

0.18

7.55

0.00

Ethnicity

6

ethnicAsian

8.18

32.43

0.25

0.80

Ethnicity

6

ethnicHispanic

6.48

30.69

0.21

0.83

Ethnicity

6

ethnicBlack

8.48

31.04

0.27

0.78

Ethnicity

6

ethnicWhite

-3.50

30.44

-0.11

0.91

Ethnicity

6

center_scale(map.spring):ethnicAsian

-0.48

0.66

-0.73

0.47

Ethnicity

6

center_scale(map.spring):ethnicHispanic

0.49

0.38

1.28

0.20

Ethnicity

6

center_scale(map.spring):ethnicBlack

-1.08

0.51

-2.11

0.04

Ethnicity

 

Grade

Term

Estimate

SE

Statistic

p Value

Model

1

(Intercept)

63.72

1.23

51.95

0.00

Free and Reduced Lunch

1

center_scale(map.spring)

2.05

0.09

23.54

0.00

Free and Reduced Lunch

1

frlY

0.53

2.25

0.24

0.81

Free and Reduced Lunch

1

center_scale(map.spring):frlY

 

-0.30

0.15

-2.03

0.04

Free and Reduced Lunch

2

(Intercept)

94.13

1.29

72.93

0.00

Free and Reduced Lunch

2

center_scale(map.spring)

1.76

0.08

21.78

0.00

Free and Reduced Lunch

2

frlY

-4.21

2.04

-2.06

0.04

Free and Reduced Lunch

2

center_scale(map.spring):frlY

 

-0.03

0.12

-0.23

0.82

Free and Reduced Lunch

3

(Intercept)

110.47

1.69

65.47

0.00

Free and Reduced Lunch

3

center_scale(map.spring)

1.91

0.11

16.89

0.00

Free and Reduced Lunch

3

frlY

-0.39

2.69

-0.14

0.89

Free and Reduced Lunch

3

center_scale(map.spring):frlY

 

-0.03

0.17

-0.19

0.85

Free and Reduced Lunch

4

(Intercept)

124.16

1.54

80.83

0.00

Free and Reduced Lunch

4

center_scale(map.spring)

1.87

0.11

16.68

0.00

Free and Reduced Lunch

4

frlY

1.12

2.48

0.45

0.65

Free and Reduced Lunch

4

center_scale(map.spring):frlY

 

0.21

0.18

1.20

0.23

Free and Reduced Lunch

5

(Intercept)

143.93

1.91

75.55

0.00

Free and Reduced Lunch

5

center_scale(map.spring)

1.91

0.16

12.19

0.00

Free and Reduced Lunch

5

frlY

0.62

2.91

0.21

0.83

Free and Reduced Lunch

5

center_scale(map.spring):frlY

 

-0.04

0.21

-0.21

0.83

Free and Reduced Lunch

6

(Intercept)

150.03

2.58

58.05

0.00

Free and Reduced Lunch

6

center_scale(map.spring)

1.25

0.19

6.68

0.00

Free and Reduced Lunch

6

frlY

3.11

4.04

0.77

0.44

Free and Reduced Lunch

6

center_scale(map.spring):frlY

0.11

0.31

0.37

0.71

Free and Reduced Lunch

 

Grade

Term

Estimate

SE

Statistic

p Value

Model

1

(Intercept)

64.52

1.00

64.35

0.00

English Language Learner

1

center_scale(map.spring)

1.95

0.07

28.39

0.00

English Language Learner

1

ellY

-4.49

6.17

-0.73

0.47

English Language Learner

1

center_scale(map.spring):ellY

 

-0.46

0.36

-1.27

0.20

English Language Learner

2

(Intercept)

92.70

1.04

89.45

0.00

English Language Learner

2

center_scale(map.spring)

1.77

0.06

27.88

0.00

English Language Learner

2

ellY

-3.24

3.92

-0.83

0.41

English Language Learner

2

center_scale(map.spring):ellY

 

-0.07

0.24

-0.31

0.76

English Language Learner

3

(Intercept)

110.18

1.34

82.53

0.00

English Language Learner

3

center_scale(map.spring)

1.93

0.09

22.07

0.00

English Language Learner

3

ellY

-0.96

5.11

-0.19

0.85

English Language Learner

3

center_scale(map.spring):ellY

 

-0.30

0.29

-1.03

0.30

English Language Learner

4

(Intercept)

124.70

1.22

101.88

0.00

English Language Learner

4

center_scale(map.spring)

1.90

0.09

21.49

0.00

English Language Learner

4

ellY

1.30

7.25

0.18

0.86

English Language Learner

4

center_scale(map.spring):ellY

 

0.68

0.45

1.52

0.13

English Language Learner

5

(Intercept)

144.69

1.46

98.84

0.00

English Language Learner

5

center_scale(map.spring)

1.87

0.11

16.70

0.00

English Language Learner

5

ellY

-13.88

9.28

-1.49

0.14

English Language Learner

5

center_scale(map.spring):ellY

 

-0.39

0.44

-0.90

0.37

English Language Learner

6

(Intercept)

151.10

2.01

75.06

0.00

English Language Learner

6

center_scale(map.spring)

1.27

0.15

8.52

0.00

English Language Learner

6

ellY

14.10

14.90

0.95

0.34

English Language Learner

6

center_scale(map.spring):ellY

1.24

1.26

0.98

0.33

English Language Learner

 

Subgroups Included:

Ethnic subgroups included White students, African American students, Asian students, Hispanic students, and American Indian students. Other demographic variables included free and reduced lunch and English Language Learner (ELL) status.

 

Bias Analysis Results:

The results from the measurement invariance analysis comparing performance among different ethnic groups suggest that the same structure (configural invariance) and the same latent factor (full/partial metric invariance) were being measured in each ethnic group. In addition, the fact that the partial or full scalar invariance held indicates that White and other ethnic groups (with the same level of reading) have almost the same expected performance on CBM-R. In conclusion, there was no evidence on structural/measurement invariance (i.e., test bias) between White and other ethnic groups, meaning that CBM-R appears to be an unbiased measure among different ethnic groups.

In the regression analyses, the authors interpreted any main effects associated with the demographic variables, or interaction effects between the demographic variables and the MAP assessment score as evidence of bias in passages (analogous to differential item functioning). Most P-values exceed the .001 level of significance. The results from these models are presented below by grade and demographic variables.

Sensitivity: Reliability of the Slope

Grade123456
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble

Describe the sample used for analyses, including size and characteristics:

Total number of students in subject sample was 2029. The percentage of students that were American Indian was 5.3%, for African American was 29.7%, for Asian American was 9.9%, for Hispanic American was 14.4%, for White American was 40.1%. 47% of the sample was female. 18.8% of the sample were students with disabilities.

 

Describe the frequency of measurement:

Assessments were administered weekly with nine testing timepoints total.

 

Describe reliability of the slope analyses conducted with a population of students in need of intensive intervention:

In our reliability of slope analysis we used Latent Growth Modeling (LGM) to estimate the reliability of longitudinal weekly growth data as demonstrated by Yeo, Kim, Branum-Martin, Wayman, and Espin (2011). These researchers point out, "The feature of LGM that treats time as an independent variable at each occasion makes it possible to estimate the reliability of longititudinal data such as CBM data" (p. 276). An important conclusion of their study was, “ a crucial advantage of LGM is that reliability estimated by LGM takes into account the developmental trajectories at the individual level of analysis (Tisak & Tisak, 2000)” (p. 287).

The reliabilities for 9 weekly assessments at each grade level, grades 1-6, are presented in the table below. For each grade level (and the entire sample) we report sample sizes, means, model-implied observed score variance, measurement error variance, observed variance - error variance, SEM, reliability for each assessment, and the median reliability for each grade level.  The range of median reliability coefficients for weekly progress monitoring probes across the six grades levels was .73 to .89 with a median of .81.

Type of Reliability

Grade

Assessment

n

Mean

Model-implied Observed Score Variance

Measure-ment Error Variance

Observed Variance – Error Variance

SEM

Coefficient

Latent Growth Modeling

1

Assess1

267

23.98

302.22

72.28

229.94

8.5

0.76

Latent Growth Modeling

1

Assess2

267

25.19

325.45

69.83

255.62

8.36

0.79

Latent Growth Modeling

1

Assess3

267

27.26

356.97

72.74

284.23

8.53

0.8

Latent Growth Modeling

1

Assess4

267

29.66

416.99

72.34

344.65

8.51

0.83

Latent Growth Modeling

1

Assess5

267

31.5

442.08

87.55

354.53

9.36

0.8

Latent Growth Modeling

1

Assess6

267

33.23

459.59

71.49

388.1

8.46

0.84

Latent Growth Modeling

1

Assess7

267

35.24

497.72

81.67

416.05

9.04

0.84

Latent Growth Modeling

1

Assess8

267

36.63

525.61

99.11

426.5

9.96

0.81

Latent Growth Modeling

1

Assess9

267

39.96

674.19

113.48

560.71

10.65

0.83

Latent Growth Modeling (Median)

1

All Timepoints 

267

 

 

 

 

 

0.81

Latent Growth Modeling

2

Assess1

412

41.77

818.61

215.78

602.83

14.69

0.74

Latent Growth Modeling

2

Assess2

412

43.77

809.76

126.18

683.58

11.23

0.84

Latent Growth Modeling

2

Assess3

412

46.2

836.64

130.46

706.17

11.42

0.84

Latent Growth Modeling

2

Assess4

412

47.94

752.42

127.84

624.58

11.31

0.83

Latent Growth Modeling

2

Assess5

412

49.13

846.01

129.77

716.24

11.39

0.85

Latent Growth Modeling

2

Assess6

412

51.47

813.24

123.38

689.87

11.11

0.85

Latent Growth Modeling

2

Assess7

412

53.57

906.92

101.97

804.95

10.1

0.89

Latent Growth Modeling

2

Assess8

412

55.7

1004.91

202.09

802.83

14.22

0.8

Latent Growth Modeling

2

Assess9

412

56.52

896.76

144.52

752.25

12.02

0.84

Latent Growth Modeling (Median)

2

All Timepoints 

412 

 

 

 

 

 

0.84

Latent Growth Modeling

3

Assess1

491

77.5

1202.54

138.87

1063.67

11.78

0.88

Latent Growth Modeling

3

Assess2

491

80.83

1188.67

133.77

1054.9

11.57

0.89

Latent Growth Modeling

3

Assess3

491

83.36

1324.91

224.67

1100.24

14.99

0.83

Latent Growth Modeling

3

Assess4

491

86.53

1383.4

311.23

1072.18

17.64

0.78

Latent Growth Modeling

3

Assess5

491

87.33

1204.38

141.96

1062.43

11.91

0.88

Latent Growth Modeling

3

Assess6

491

88.9

1270.41

133.28

1137.13

11.54

0.9

Latent Growth Modeling

3

Assess7

491

91.46

1459.82

224.46

1235.35

14.98

0.85

Latent Growth Modeling

3

Assess8

491

92.66

1273.58

137.24

1136.34

11.72

0.89

Latent Growth Modeling

3

Assess9

491

93.27

1261.91

162.22

1099.7

12.74

0.87

Latent Growth Modeling (Median)

3

All Timepoints

491 

 

 

 

 

 

0.88

Latent Growth Modeling

4

Assess1

352

98.46

1365.94

481.32

884.63

21.94

0.65

Latent Growth Modeling

4

Assess2

352

98.76

1087.14

273.45

813.69

16.54

0.75

Latent Growth Modeling

4

Assess3

352

100.56

1056.35

257.96

798.39

16.06

0.76

Latent Growth Modeling

4

Assess4

352

100.13

1087.95

198.58

889.37

14.09

0.82

Latent Growth Modeling

4

Assess5

352

103.21

1022.3

207.64

814.66

14.41

0.8

Latent Growth Modeling

4

Assess6

352

105.81

1205.94

276.01

929.93

16.61

0.77

Latent Growth Modeling

4

Assess7

352

106.2

1161.25

165.32

995.93

12.86

0.86

Latent Growth Modeling

4

Assess8

352

107.14

1091.65

232.15

859.5

15.24

0.79

Latent Growth Modeling

4

Assess9

352

107.82

1207.99

207.29

1000.7

14.4

0.83

Latent Growth Modeling (Median)

4

All Timepoints

352 

 

 

 

 

 

0.79

Latent Growth Modeling

5

Assess1

329

103.12

1227.87

301.66

926.21

17.37

0.75

Latent Growth Modeling

5

Assess2

329

104.56

1165.03

305.33

859.71

17.47

0.74

Latent Growth Modeling

5

Assess3

329

107.04

1301.86

451.24

850.62

21.24

0.65

Latent Growth Modeling

5

Assess4

329

106.05

1279.4

300.96

978.44

17.35

0.76

Latent Growth Modeling

5

Assess5

329

106.95

1153.05

285.32

867.73

16.89

0.75

Latent Growth Modeling

5

Assess6

329

110.37

1179.15

319.27

859.88

17.87

0.73

Latent Growth Modeling

5

Assess7

329

113.45

1177.76

318.17

859.59

17.84

0.73

Latent Growth Modeling

5

Assess8

329

111.49

1227.17

350.06

877.11

18.71

0.71

Latent Growth Modeling

5

Assess9

329

113.66

1177.98

326.42

851.56

18.07

0.72

Latent Growth Modeling (Median)

5

All Timepoints

329

 

 

 

 

 

0.73

Latent Growth Modeling

6

Assess1

178

116.9

1507.86

344.73

1163.12

18.57

0.77

Latent Growth Modeling

6

Assess2

178

116.55

1292.55

338.5

954.06

18.4

0.74

Latent Growth Modeling

6

Assess3

178

115.99

1342.37

280.82

1061.55

16.76

0.79

Latent Growth Modeling

6

Assess4

178

120.14

1317.25

242.36

1074.89

15.57

0.82

Latent Growth Modeling

6

Assess5

178

123.16

1317.93

279.8

1038.14

16.73

0.79

Latent Growth Modeling

6

Assess6

178

121.14

1327.09

454.85

872.24

21.33

0.66

Latent Growth Modeling

6

Assess7

178

124.65

1318.15

314.48

1003.67

17.73

0.76

Latent Growth Modeling

6

Assess8

178

125.46

1376.98

251.33

1125.65

15.85

0.82

Latent Growth Modeling

6

Assess9

178

127.28

1327.66

355.22

972.44

18.85

0.73

Latent Growth Modeling (Median)

6

All Timepoints

178

 

 

 

 

 

0.77

 

Sensitivity: Validity of the Slope

Grade123456
RatingEmpty bubbleEmpty bubbleEmpty bubbledashdashdash

Describe and justify the criterion measures used to demonstrate validity:

No qualifying evidence provided. 

 

Describe the sample used for analyses, including size and characteristics:

The sample of students used in the validity of slope analysis are students in need of intensive intervention. All 229 students scored below the 20th percentile on MAP Reading in the fall of the school year. The demographic characteristics of the sample were:

Free and Reduced Lunch: 38%

Special Education: 21%

Title I Service: 50%

Female: 42%  

Male: 58%

English Language Learners: 9%

American Indian students: 4%

Asian American students: 4%

Hispanic American students: 14%

Black American students: 7%

White American students: 71%

 

Describe predictive validity of the slope of improvement analyses conducted with a population of students in need of intensive intervention:

Three types of analysis were used to validate the weekly slope of Words Read Correctly.

1.     Pearson Correlation: Correlating Weekly slope of Words Read Correctly with performance on MAP Reading in the Spring.

2.     Pearson Correlation: Correlating Weekly slope of Words Read Correctly with the increase from fall to spring on the MAP Reading RIT score.

3.     Partial Correlation: Correlating Weekly slope of Words Read Correctly with performance on MAP Reading in the Spring and controlling for initial level of MAP Reading performance in the fall.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Pearson Correlation

1

MAP Spring

59

.269 (p=.039)

.014-.491

Pearson Correlation

1

MAP Gain F to S

59

.498 (p<.001)

.278-.669

Partial Correlation

1

MAP Spring

56

.433 (p=.001)

.192-.625

Pearson Correlation

2

MAP Spring

108

.374 (p=.000)

.199-.526

Pearson Correlation

2

MAP Gain F to S

108

.431 (p<.001)

.263-.573

Partial Correlation

2

MAP Spring

105

.432 (p<.001)

.262-.576

Pearson Correlation

3

MAP Spring

62

.322 (p=.011)

.079-.529

Pearson Correlation

3

MAP Gain F to S

62

.347 (p<.001)

.106-.549

Partial Correlation

3

MAP Spring

59

.345 (p=.007)

.098-.552

 

Describe the degree to which the provided data support the validity of the tool:

All correlation coefficients reported were statistically significant. Data indicates that weekly slope for Words Read Correctly over the school year is significantly related to the external criterion measure of MAP Reading, which was measured in the fall and spring of the same school year.

Alternate Forms

Grade123456
RatingFull bubbleEmpty bubbleFull bubbleEmpty bubbleEmpty bubbleFull bubble

Describe the sample for these analyses, including size and characteristics:

The analysis was conducted at each grade level on a sample of students meeting the criteria for needing intensive intervention (below the 20th percentile).

 

Evidence that alternate forms are of equal and controlled difficulty or, if IRT based, evidence of item or ability invariance:

Evidence of controlled difficulty for the passages used for progress monitoring at each grade was determined through use of the Flesch-Kinkaid Grade Level readability formula and the Lexile Framework for Reading (Metametrics, 2002). Both analyses indicate the passages were placed at appropriate grade levels. 

At grade 1 the passages had an average Flesch readability of 1.6 and Lexile of 272. 

At grade 2 the average Flesch readability was 2.4 and Lexile of 397. 

At grade 3 the average Flesch score was 3.3 and Flesch was 511. 

At grade 4 the average readability was 4.2 and Lexile was 656. 

At grade 5 Flesch readability was 5.8 and Lexile score was 759. 

At grade 6 Flesch readability was 6.9 and Lexile was 850.

To explore alternative forms of the progress monitoring tool, the authors fit an analysis of variance by grade in which words read correctly (for students’ first passage) was regressed on the passage id. An ANOVA model comparing mean words read correctly by passage id, was compared against a null model with an intercept only. The authors used an adjusted critical p-value (for fitting the analyses by grade for grades 1 through 6) of 0.008 (0.05/6). The results indicate no significant differences among the alternate forms of the passages at each grade level and are presented below.

Grade

Time

n

Residual Sum of Squares

Degrees of Freedom

Sum of Squares

Statistic

p Value

1

1

59

122.4

14

27.78

0.73

0.73

2

1

1049

54947.6

19

1969.78

1.94

0.01

3

1

915

154835

20

3978.55

1.15

0.29

4

1

913

298397

19

10234.2

1.61

0.05

5

1

831

284776

19

13077

1.96

0.01

6

1

534

209160

19

9366.12

1.21

0.24

 

Number of alternate forms of equal and controlled difficulty:

Twenty reading passages are available for progress monitoring at each grade level (1-6).

Decision Rules: Setting and Revising Goals

Grade123456
Ratingdashdashdashdashdashdash

Specification of validated decision rules for when goals should be set or revised:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Decision Rules: Changing Instruction

Grade123456
Ratingdashdashdashdashdashdash

Specification of validated decision rules for when changes to instruction should be made:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Administration Format

Grade123456
Data
  • Individual
  • Individual
  • Individual
  • Individual
  • Individual
  • Individual
  • Administration & Scoring Time

    Grade123456
    Data
  • 3 minutes
  • 3 minutes
  • 3 minutes
  • 3 minutes
  • 3 minutes
  • 3 minutes
  • Scoring Format

    Grade123456
    Data
  • Manually-scored
  • Manually-scored
  • Manually-scored
  • Manually-scored
  • Manually-scored
  • Manually-scored
  • ROI & EOY Benchmarks

    Grade123456
    Data
  • ROI Available
  • ROI Available
  • ROI Available
  • ROI Available
  • ROI Available
  • ROI Available
  • Specify the minimum acceptable rate of growth/improvement:

    The Standard Reading Passages growth standards are derived from a sample of 2007 students enrolled in grades 1 to 6 who were progress monitored on a weekly basis during the school year. The sample included 228 students in grade 1, 418 students in grade 2, 493 students in grades 3, 353 students in grade 4, 332 students in grade 5, and 183 students in grade 6. The average number of progress monitoring passages administered to these students was 16.5 (SD=6.3).

    To specify "rates of improvement" for Words Read Correctly, each student’s slope was calculated. The developers then determined percentiles for Words Read Correctly for each grade level. In the table below the 25th, 50, and 75th percentiles at each grade are provided for grades 1 through 6 (see table below). Slopes associated with the 75th percentile are 1.6 at grade 1, 1.6 at grade 2, 1.4 at grade 3, 1.2 at grade 4, 1.4 at grade 5, and 1.3 at grade 6.

     

    Specify the benchmarks for minimum acceptable end-of-year performance:

    No qualifying evidence provided.