i-Ready Diagnostic and Growth Monitoring

Reading / English Language Arts

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

$6.00 per student

 

Replacement Cost:

$6.00 per student per year. Annual license renewal fee subject to change.

 

Included in Cost:

The license fee includes online student access to assessment, plus staff access to management and reporting suite, downloadable lesson plans, and user resources including the i-Ready Central® support website; account set-up and secure hosting; all program maintenance/ updates/enhancements during the active license term; and unlimited user access to U.S.-based service and support via toll-free phone and email during business hours. The license fee also includes hosting, data storage, and data security.

 

Via the i-Ready teacher and administrator dashboards and i-Ready Central support website, educators may access comprehensive user guides and downloadable lesson plans, as well as implementation tips, best practices, video tutorials, and more to supplement onsite, fee-based professional development. These online resources are self-paced and available 24/7.

 

Professional development is required and available at an additional cost ($2,000/session up to six hours).

 

Technology Requirements:

  • Computer or tablet
  • Internet connection

 

Training Requirements:

  • 4-8 hours of training

 

Qualified Administrators:

  • Paraprofessionals
  • Professionals

 

Accommodations:

Curriculum Associates engaged an independent consultant to thoroughly evaluate i‑Ready Diagnostic’s accessibility and provide some recommendations regarding how best to support the broadest possible range of student learners. Overall, the report found that i-Ready “materials included significant functionality that indirectly supports… students with disabilities.” The report also indicated ways to support these groups of students more directly, which we are in the process of prioritizing for future development. We are committed to meaningful ongoing enhancement and expansion of the program’s accessibility.
 

Diverse student groups experience success with the program largely due to its adaptive nature and program design. All items in i-Ready Diagnostic are designed to be accessible for most students. In a majority of cases, students who require accommodations (e.g., large print, extra time) will not require additional help during administration.

 

To address the elements of Universal Design as they apply to large-scale assessment (http://www.cehd.umn.edu/nceo/onlinepubs/Synthesis44.html), in developing i-Ready Curriculum Associates considered several issues related to accommodations. Most may be grouped into the following general categories that i‑Ready addresses:

 

Timing and Flexible Scheduling—The Growth Monitoring assessment may be stopped and started as needed to allow students needing extra time to finish. Growth Monitoring is untimed and can be administered in multiple test sessions.

 

Accommodated Presentation of Material—All i-Ready items are presented in a large, easily legible format specifically chosen for its readability. i‑Ready currently offers the ability to change the screen size. There is only one item on the screen at a time. As appropriate to the skill(s) being assessed, some grade levels K–2 reading items also offer optional audio support.

 

Setting—Students may need to complete the task in a quiet room to minimize distraction. This can easily be done, as i-Ready is available on any computer with internet access that meets the technical requirements.

 

Response Accommodation—Students should be able to control a mouse. They only need to be able to move a cursor with the mouse and be able to point, click, and drag.

Where to Obtain:

Website: www.curriculumassociates.com

Address:
153 Rangeway Road, N. Billerica MA 01862

Phone Number:
800-225-0248

Email: info@cainc.com


Access to Technical Support:

Dedicated account manager plus unlimited access to in-house technical support during business hours.

i-Ready Growth Monitoring is a brief, computer-delivered, periodic adaptive assessment in reading/English language arts (ELA) for students in grades K–8, assessing Phonological Awareness, Phonics, High-Frequency Words, Vocabulary, Comprehension of Informational Text, and Comprehension of Literature. Growth Monitoring is part of the i-Ready Diagnostic & Instruction suite and is designed to be used jointly with i-Ready Diagnostic to allow for progress monitoring throughout the year and determine whether students are on track for appropriate growth. Growth Monitoring is designed to be administered monthly but may be administered as frequently as every week in which the
i-Ready Diagnostic assessment is not administered.

 

Curriculum Associates designed and developed i‑Ready specifically to assess student mastery of state and Common Core State Standards (CCSS). Growth Monitoring assessment takes approximately 15 minutes and may be conducted with all students or with specific groups of students who have been identified as at risk of academic failure.

Assessment Format:

  • Individual
  • Computer-administered

 

Administration Time:

  • 15 minutes per student

 

Scoring Time:

  • Scoring is automatic

 

Scoring Method:

  • Calculated automatically

 

Scores Generated:

  • Percentile Score
  • IRT-Based Score
  • Developmental Benchmarks
  • Lexile Score
  • On-grade achievement level placements

 

 

Reliability

Grade12345678K
RatingFull bubbledFull bubbledFull bubbledFull bubbledFull bubbledFull bubbledFull bubbledFull bubbledFull bubbled

Justify the appropriateness of each type of reliability reported:

For the i-Ready Diagnostic, Curriculum Associates prepares the IRT-based marginal reliability, as well as the standard error of measurement (SEM).

Given that the i-Ready Diagnostic is a computer-adaptive assessment that does not have a fixed form, some traditional reliability estimates such as Cronbach’s alpha are inappropriate for quantifying consistency of student scores. The IRT analogue to classical reliability is called marginal reliability, and operates on the variance of the theta scores (i.e., proficiency) and the average of the expected error variance. The marginal reliability uses the classical definition of reliability as proportion of variance in the total observed score due to true score under an IRT model (the i-Ready Diagnostic uses a Rasch model to be specific).

In addition to marginal reliability, SEMs are also important for quantifying the precision of scores. In an IRT model, SEMs are affected by factors such as how well the data fit the underlying model, student response consistency, student location on the ability continuum, match of items to student ability, and test length.  Given the adaptive nature of i-Ready and the wide difficulty range in the item bank, standard errors are expected to be low and very close to the theoretical minimum for tests of similar length.

The theoretical minimum would be reached if each interim estimate of student ability is assessed by an item with difficulty matching perfectly to the student’s ability estimated from previous items. Theoretical minimums are restricted by the number of items served in the assessment—the more items that are served up, the lower the SEM could potentially be. For ELA, the minimum SEM for overall scores is 8.90.

In addition to providing the mean SEM by subject and grade, graphical representations of the conditional standard errors of measurement (CSEM) provide additional evidence of the precision with which i-Ready measures student ability across the operational score scale. In the context of model-based reliability analyses for computer adaptive tests, such as i-Ready, CSEM plots permit test users to judge the relative precision of the estimate. These figures are available from the Center upon request.

 

Describe the sample characteristics for each reliability analysis conducted:

Data for obtaining the marginal reliability and SEM was from the August and September administrations of the i-Ready Diagnostic from 2016 (reported in Table 4.4 of the i-Ready Diagnostic Technical Manual). All students tested within the timeframe were included and this time period was selected because it coincides with most districts’ first administration of the i-Ready Diagnostic.

 

Describe the analysis procedures for each reported type of reliability:

This marginal reliability uses the classical definition of reliability as the proportion of variance in the total observed score due to true score. The true score variance is computed as the observed score variance minus the error variance. Similar to a classical reliability coefficient, the marginal reliability estimate increases as the standard error decreases; it approaches 1 when the standard error approaches 0.

The observed score variance, the error variance, and SEM (the square root of the error variance) are obtained through WINSTEPS calibrations. One separate calibration was conducted for each grade.

Type of Reliability

Age or Grade

n

Coefficient

Confidence Interval

Marginal

Kindergarten

184,261

0.91

 

Marginal

Grade 1

287,593

0.95

 

Marginal

Grade 2

323,280

0.96

 

Marginal

Grade 3

343,103

0.97

 

Marginal

Grade 4

337,854

0.97

 

Marginal

Grade 5

341,292

0.97

 

Marginal

Grade 6

249,454

0.97

 

Marginal

Grade 7

224,530

0.97

 

Marginal

Grade 8

222,503

0.97

 

SEM

Kindergarten

184,261

9.30

 

SEM

Grade 1

287,593

9.33

 

SEM

Grade 2

323,280

10.38

 

SEM

Grade 3

343,103

10.11

 

SEM

Grade 4

337,854

10.14

 

SEM

Grade 5

341,292

10.35

 

SEM

Grade 6

249,454

10.51

 

SEM

Grade 7

224,530

10.61

 

SEM

Grade 8

222,503

10.71

 

 

Disaggregated Reliability Data:

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

Confidence Interval

Split-half

Asian

Grade 1

531

0.80

n/a*

Split-half

African American

Grade 1

2,665

0.75

n/a*

Split-half

Hispanic

Grade 1

2,246

0.77

n/a*

Split-half

Asian

Grade 2

549

0.86

n/a*

Split-half

African American

Grade 2

2,990

0.81

n/a*

Split-half

Hispanic

Grade 2

2,289

0.79

n/a*

Split-half

Asian

Grade 3

468

0.83

n/a*

Split-half

African American

Grade 3

2,881

0.80

n/a*

Split-half

Hispanic

Grade 3

2,269

0.80

n/a*

Split-half

Asian

Grade 4

439

0.80

n/a*

Split-half

African American

Grade 4

1,977

0.77

n/a*

Split-half

Hispanic

Grade 4

1,577

0.76

n/a*

Split-half

Asian

Grade 5

370

0.79

n/a*

Split-half

African American

Grade 5

1,612

0.78

n/a*

Split-half

Hispanic

Grade 5

1,249

0.79

n/a*

Split-half

Asian

Grade 6

247

0.83

n/a*

Split-half

African American

Grade 6

515

0.78

n/a*

Split-half

Hispanic

Grade 6

639

0.74

n/a*

Split-half

African American

Grade 7

254

0.76

n/a*

Split-half

Hispanic

Grade 7

278

0.81

n/a*

Split-half

African American

Grade 8

234

0.88

n/a*

Split-half

Hispanic

Grade 8

198

0.83

n/a*

* n/a: Confidence intervals are not applicable to split-half reliability estimates due to how they are calculated for computer-adaptive assessments.  Although some modeling approaches exist that yield confidence intervals for adaptive tests, the psychometric field does not currently have an agreed-upon approach and instead favors the reporting of reliability point estimates for adaptive assessments.

Validity

Grade12345678K
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble

Describe and justify the criterion measures used to demonstrate validity:

The Dynamic Indicators of Basic Early Literacy Skills (DIBELS) are a set of procedures and measures for assessing the acquisition of early literacy skills from kindergarten through sixth grade.  The Lexile® Framework for Reading is an educational tool that uses a measure called a Lexile to match readers with books, articles, and other leveled reading resources. Readers and books are assigned a score on the Lexile scale, in which lower scores reflect easier readability for books and lower reading ability for readers.

The North Carolina End-of-Grade (NC EOG) English Language Arts/Reading tests measure student performance on the grade-level competencies specified by North Carolina Public Schools. Ohio’s State Tests (OST) in English Language Arts measure the knowledge and skills specified by Ohio’s Learning Standards. The Mississippi Academic Assessment Program (MAAP) measures student achievement in relation to the Mississippi College and Career Readiness Standards for English Language Arts. The Florida Standards Assessments (FSA) in English Language Arts measure student achievement in relation to the education standards outlined by the Florida Department of Education. These criterions are appropriate because they measure the knowledge and skills specified by the educational standards of four different states.

 

Describe the sample characteristics for each validity analysis conducted:

The K–2 samples described in this section were selected such that at least three U.S. geographic regions are represented, and were then vetted to ensure the samples were consistent with the population of students who take i-Ready in grades K–2. The DIBELS measure consisted of data from the 2016–2017 school year from five districts and one charter organization across three states, Colorado, Ohio, and North Carolina. 

The Lexile data come from the Lexile / i-Ready linking study that was collaboratively conducted by MetaMetrics and Curriculum Associates.  A total of 35 schools in 27 districts representing 10 states participated in the study.

The samples for grades 3–8 described in this section were selected specifically to be representative of the states in terms of urbanicity; district size; proportion of English language learners, and students with disabilities; and proportion of students eligible for free- and reduced-priced lunch. The North Carolina sample consisted of 38,695 students from 12 school districts and 202 schools across the state of North Carolina. The Ohio sample consisted of 13,551 students from 10 school districts and 62 schools across the state of Ohio. The Mississippi sample consisted of 19,618 students from 13 school districts and 78 schools across the state of Mississippi. The Florida sample consisted of 230,705 students from 13 school districts and 816 schools across the state of Florida.

 

Describe the analysis procedures for each reported type of validity:

For the DIBELS analysis, the correlations were calculated between the spring administrations of both tests, allowing for concurrent validity inferences. For the Lexile analysis, the correlations were calculated between the fall administration of the Lexile process and the spring administration of i-Ready, yielding predictive validity inferences.

For the North Carolina and Ohio studies, correlations were calculated between the given state assessment (administered in spring of 2016) and last i-Ready Diagnostic administration in spring of 2016. The state assessments were administered within 1–3 months of the i-Ready Diagnostic. For the Mississippi and Florida studies, correlations were calculated between the given state assessment (administered in spring of 2017) and the first i-Ready Diagnostic administration in fall of 2016. The state assessments were administered 4–10 months after the i-Ready Diagnostic. Fisher’s r to z transformation was used to obtain the 95% confidence interval for the correlation coefficients of all studies.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Concurrent

Kindergarten

DIBELS NEXT

3305

0.64

[0.62, 0.66]

Concurrent

Grade 1

DIBELS NEXT

4108

0.84

[0.83, 0.85]

Concurrent

Grade 2

DIBELS NEXT

4628

0.80

[0.79, 0.81]

Predictive

Kindergarten

Lexile*

840

0.88

[0.86, 0.89]

Predictive

Grade 1

Lexile*

840

0.88

[0.86, 0.89]

Predictive

Grade 2

Lexile*

840

0.88

[0.86, 0.89]

Concurrent/Construct

Grade 3

2016 North Carolina End-of-Grade Tests

7,603

0.83

[0.82, 0.83]

Concurrent/Construct

Grade 4

2016 NC EOG Tests

7,415

0.83

[0.82, 0.84]

Concurrent/Construct

Grade 5

2016 NC EOG Tests

7,505

0.82

[0.81, 0.83]

Concurrent/Construct

Grade 6

2016 NC EOG Tests

5,205

0.82

[0.81, 0.83]

Concurrent/Construct

Grade 7

2016 NC EOG Tests

5,685

0.81

[0.80, 0.82]

Concurrent/Construct

Grade 8

2016 NC EOG Tests

5,282

0.78

[0.77, 0.79]

Concurrent/Construct

Grade 3

2016 Out-of-School Time

3,039

0.74

[0.73, 0.76]

Concurrent/Construct

Grade 4

2016 OST

2,702

0.76

[0.75, 0.78]

Concurrent/Construct

Grade 5

2016 OST

2,702

0.76

[0.74, 0.78]

Concurrent/Construct

Grade 6

2016 OST

1,872

0.77

[0.75, 0.79]

Concurrent/Construct

Grade 7

2016 OST

1,614

0.76

[0.74, 0.78]

Concurrent/Construct

Grade 8

2016 OST

1,622

0.69

[0.67, 0.72]

Predictive

Grade 3

2017 Mississippi Academic Assessment

3,244

0.77

[0.76, 0.79]

Predictive

Grade 4

2017 MAAP

3,879

0.74

[0.73, 0.76]

Predictive

Grade 5

2017 MAAP

3,671

0.77

[0.76, 0.79]

Predictive

Grade 6

2017 MAAP

3,571

0.80

[0.79, 0.81]

Predictive

Grade 7

2017 MAAP

2,701

0.82

[0.81, 0.83]

Predictive

Grade 8

2017 MAAP

2,552

0.80

[0.79, 0.82]

Predictive

Grade 3

2017 Florida State Assessments

54,668

0.80

[0.80, 0.80]

Predictive

Grade 4

2017 FSA

47,982

0.81

[0.81, 0.81]

Predictive

Grade 5

2017 FSA

50,532

0.83

[0.82, 0.83]

Predictive

Grade 6

2017 FSA

25,977

0.83

[0.83, 0.84]

Predictive

Grade 7

2017 FSA

25,537

0.82

[0.81, 0.82]

Predictive

Grade 8

2017 FSA

26,009

0.82

[0.82, 0.83]

* For the purposes of the Lexile study referenced above, grade-banded results are featured, rather than grade-specific results. The i-Ready Diagnostic reading scale scores are created on a vertical scale, which makes the scale scores comparable across grades. Thus, for efficiency purposes, the linking sample for the Lexile study includes only students from every other grade (i.e., grades 1, 3, 5, and 7), but results are generalized across grades in various grade bands (e.g., K–2). Additional information on the Lexile study, which was conducted in concert with MetaMetrics, is available upon request.

 

Describe the degree to which the provided data support the validity of the tool:

The data show that the i-Ready Diagnostic is highly correlated with both near-term and future state assessment scores. The inclusion of four different state assessments shows that i-Ready is a general measure of students’ knowledge and skills in English Language Arts standards across states.

Bias Analysis Conducted

Grade12345678K
RatingYesYesYesYesYesYesYesYesYes

Have additional analyses been conducted to establish whether the tool is or is not biased against demographic subgroups (e.g., students who vary by race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)?

Bias Analysis Method:

Differential Item Function (DIF) was investigated using WINSTEPS® (Version 3.92) by comparing item difficulty for pairs of demographic subgroups through a combined calibration analysis. This methodology evaluates the interaction of the person-level subgroups with each item, while fixing all other item and person measures to those from the combined calibration. The method used to detect DIF is based on the Mantel-Haenszel procedure (MH), and the work of Linacre & Wright (1989) and Linacre (2012). Typically, the groups of test takers are referred to as “reference” and “focal” groups. For example, for analysis of gender bias, Female test takers are the focal group, and Male test takers are the reference group. More information is provided in section 3.4 of the i‑Ready Technical Manual.

 

Subgroups Included:

The latest large-scale DIF analysis included a random sample (20%) of students from the 2015–2016 i-Ready operational data. Given the large size of the 2015–2016 i-Ready student population, it is practical to carry out the calibration analysis with a random sample. The following demographic categories were compared: Female vs. Male; African American and Hispanic vs. Caucasian; English Learner vs. non–English Learner; Special Ed vs. General Ed; Economically Disadvantaged vs. Not Economically Disadvantaged.

 

Bias Analysis Results:

All active items in the current item pool for the 2015–2016 school year are included in the DIF analysis. The total numbers of items are 3,649 for reading. WINSTEPS was used to conduct the calibration for DIF analysis by grade. To help interpret the results, the Educational Testing Service (ETS) criteria using the delta method was used to categorize DIF (Zwick, Thayer, & Lewis, 1999) and is presented in the table below:

ETS DIF Category

Criterion

A (negligible)

|DIF| < 0.43

B (moderate)

|DIF| ≥ 0.43 and |DIF| < 0.64

C (large)

|DIF| ≥ 0.64

B- or C- suggests DIF against focal group

B+ or C+ suggests DIF against reference group

 

The numbers and percentages of items exhibiting DIF for each of the demographic categories are reported in the table below. The majority of reading items show negligible DIF (at least 90 percent), and for very few categories do more than 3 percent of items show large DIF (level C) by grade.

Grade

ETS DIF

Category

Gender

Ethnicity

ELL

Special

Education

Economically

Disadvantaged

N

Percent

N

Percent

N

Percent

N

Percent

N

Percent

K

 

 

 

 

 

A

1,315

97.4

1,227

96.1

1,106

96.9

408

96.0

1,160

98.3

B+

9

0.7

12

0.9

10

0.9

5

1.2

5

0.4

B-

11

0.8

31

2.4

19

1.7

10

2.4

13

1.1

C+

4

0.3

2

0.2

2

0.2

1

0.2

0

0.0

C-

11

0.8

5

0.4

4

0.4

1

0.2

2

0.2

Total

1,350

100.0

1,277

100.0

1,141

100.0

425

100.0

1,180

100.0

1

 

 

 

 

 

A

1,741

96.5

1,686

95.8

1,435

95.1

967

94.7

1,562

97.4

B+

15

0.8

35

2.0

22

1.5

23

2.3

13

0.8

B-

40

2.2

27

1.5

29

1.9

20

2.0

18

1.1

C+

4

0.2

7

0.4

16

1.1

6

0.6

4

0.2

C-

5

0.3

5

0.3

7

0.5

5

0.5

6

0.4

Total

1,805

100.0

1,760

100.0

1,509

100.0

1,021

100.0

1,603

100.0

2

 

 

 

 

 

A

1,886

95.3

1,766

95.2

1,668

93.1

1,094

93.0

1,868

96.4

B+

35

1.8

49

2.6

44

2.5

35

3.0

28

1.4

B-

48

2.4

30

1.6

46

2.6

26

2.2

26

1.3

C+

5

0.3

7

0.4

21

1.2

16

1.4

11

0.6

C-

4

0.2

4

0.2

12

0.7

5

0.4

5

0.3

Total

1,978

100.0

1,856

100.0

1,791

100.0

1,176

100.0

1,938

100.0

3

 

 

 

 

 

A

2,337

94.7

2,047

95.1

1,718

91.2

1,251

89.7

2,122

95.4

B+

44

1.8

52

2.4

54

2.9

54

3.9

38

1.7

B-

63

2.6

38

1.8

69

3.7

50

3.6

39

1.8

C+

14

0.6

9

0.4

15

0.8

22

1.6

17

0.8

C-

9

0.4

6

0.3

28

1.5

18

1.3

9

0.4

Total

2,467

100.0

2,152

100.0

1,884

100.0

1,395

100.0

2,225

100.0

4

 

 

 

 

 

A

2,386

95.3

2,000

96.3

1,863

89.7

1,552

91.8

2,208

96.4

B+

58

2.3

39

1.9

63

3.0

36

2.1

30

1.3

B-

29

1.2

25

1.2

80

3.8

54

3.2

25

1.1

C+

20

0.8

10

0.5

26

1.3

26

1.5

14

0.6

C-

11

0.4

2

0.1

46

2.2

23

1.4

14

0.6

Total

2,504

100.0

2,076

100.0

2,078

100.0

1,691

100.0

2,291

100.0

5

 

 

 

 

 

A

2,280

95.0

2,130

96.1

1,907

89.3

1,551

90.8

2,246

97.0

B+

41

1.7

43

1.9

79

3.7

50

2.9

29

1.3

B-

51

2.1

29

1.3

77

3.6

71

4.2

27

1.2

C+

18

0.8

12

0.5

30

1.4

18

1.1

9

0.4

C-

9

0.4

2

0.1

42

2.0

18

1.1

4

0.2

Total

2,399

100.0

2,216

100.0

2,135

100.0

1,708

100.0

2,315

100.0

6

 

 

 

 

 

A

2,135

92.6

1,921

94.1

1,561

86.2

1,520

90.1

2,120

95.5

B+

54

2.3

62

3.0

80

4.4

64

3.8

39

1.8

B-

81

3.5

43

2.1

96

5.3

69

4.1

41

1.8

C+

25

1.1

10

0.5

39

2.2

14

0.8

7

0.3

C-

10

0.4

5

0.2

34

1.9

20

1.2

14

0.6

Total

2,305

100.0

2,041

100.0

1,810

100.0

1,687

100.0

2,221

100.0

7

 

 

 

 

 

A

2,307

91.8

1,970

92.5

1,476

82.1

1,582

87.6

2,227

94.7

B+

76

3.0

66

3.1

109

6.1

83

4.6

54

2.3

B-

90

3.6

63

3.0

100

5.6

93

5.1

48

2.0

C+

26

1.0

26

1.2

58

3.2

22

1.2

8

0.3

C-

15

0.6

5

0.2

54

3.0

26

1.4

14

0.6

Total

2,514

100.0

2,130

100.0

1,797

100.0

1,806

100.0

2,351

100.0

8

 

 

 

 

 

A

2,280

89.1

1,930

93.2

1,412

78.5

1,599

87.0

2,209

94.4

B+

95

3.7

50

2.4

120

6.7

76

4.1

54

2.3

B-

127

5.0

51

2.5

114

6.3

95

5.2

53

2.3

C+

33

1.3

24

1.2

81

4.5

27

1.5

12

0.5

C-

25

1.0

15

0.7

71

3.9

40

2.2

11

0.5

Total

2,560

100.0

2,070

100.0

1,798

100.0

1,837

100.0

2,339

100.0

 

Sensitivity: Reliability of the Slope

Grade12345678K
Ratingdashdashdashdashdashdashdashdashdash

Describe the sample used for analyses, including size and characteristics:

No qualifying evidence provided.

 

Describe the frequency of measurement:

No qualifying evidence provided.

 

Describe reliability of the slope analyses conducted with a population of students in need of intensive intervention:

No qualifying evidence provided.

Sensitivity: Validity of the Slope

Grade12345678K
Ratingdashdashdashdashdashdashdashdashdash

Describe and justify the criterion measures used to demonstrate validity:

No qualifying evidence provided.

 

Describe the sample used for analyses, including size and characteristics:

No qualifying evidence provided.

 

Describe predictive validity of the slope of improvement analyses conducted with a population of students in need of intensive intervention:

No qualifying evidence provided.

 

Describe the degree to which the provided data support the validity of the tool:

No qualifying evidence provided.

Alternate Forms

Grade12345678K
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble

Describe the sample for these analyses, including size and characteristics:

The i-Ready assessment forms are assembled on the fly by Curriculum Associates’ computer-adaptive testing (CAT) algorithm, subject to objective content and other constraints described in Section 2.1.3 in Chapter 2 of the attached i-Ready Technical Manual. As such, the sample size per form which would be applicable to linear (i.e., non-adaptive) assessments does not directly apply to Curriculum Associates’ i Ready Diagnostic assessment. Note that many analyses that Curriculum Associates conducts (e.g., to estimate growth targets) are based on normative samples, which for the 2015–2016 school year included 3.9 million i-Ready Diagnostic assessments taken by more than one million students from over 4,000 schools. The demographics of the normative sample at each grade closely match that of the national student population. Tables 7.3 and 7.4 of the Technical Manual present the sample sizes for each normative sample and the demographics of the samples compared with the latest population target, as reported by the National Center for Education Statistics.

 

Evidence that alternate forms are of equal and controlled difficulty or, if IRT based, evidence of item or ability invariance:

Section 2.1.3 in Chapter 2 of the i-Ready Technical Manual describes the adaptive nature of the tests and how the item selection process works. The i-Ready Growth Monitoring assessments are a general outcome measure of student ability and measure a subset of skills that are tested on the Diagnostic. Items on Growth Monitoring are from the same domain item pool for the Diagnostic. Test items are served based on the same IRT ability estimate and item selection logic.

Often, test developers want to show that the items in their measure are invariant, meaning the items are measuring both groups similarly. To illustrate the property of item invariance across the groups of i-Ready test takers in need of intensive intervention (i.e., below the national norming sample’s 30th percentile rank in terms of overall reading scale score) and those without such need (i.e., at or above the 30th percentile rank), a special set of item calibrations were prepared. Correlations between independent item calibrations for subgroups of students below and at-or-above the 30th percentile rank were computed to demonstrate the extent that i-Ready parameter estimates are appropriate for use with both groups.

To demonstrate comparable item parameter estimates, correlations between the below and at-or-above the 30th percentile item difficulty parameter estimates and their corresponding confidence intervals—constructed using Fisher’s r-to-z transformation (Fisher, R. A. 1915. Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4), 507-521)—were provided. Correlations and corresponding confidence intervals can serve as a measure of the consistency between the item difficulty estimates.

Student response data used for item invariance analyses were from the August and September 2017 administrations of the i-Ready Diagnostic. Students tested within this timeframe were subjected to the same inclusion rules that Curriculum Associates uses for new item calibration (i.e., embedded field test). This administration window was selected because it coincides with most districts’ first administration of the i-Ready Diagnostic. In order to ensure appropriately precise item parameter estimates, the sample was restricted to those items to which there were at least 300 students from each group (those below and those at-or-above the 30th percentile rank). Subgroup sample sizes and the counts of items included by grade for reading are presented in the table below.

 

Type of Analysis

Age or Grade

Students by 30th Percentile Rank

# Items

Correlation Coefficient

Confidence Interval

Below

At or Above

Item Invariance

K

83,949

133,559

417

0.893

[0.871, 0.911]

Item Invariance

1

125,087

248,046

787

0.840

[0.818, 0.859]

Item Invariance

2

151,681

261,591

559

0.856

[0.832, 0.877]

Item Invariance

3

177,285

294,692

690

0.799

[0.770, 0.824]

Item Invariance

4

147,429

320,484

819

0.793

[0.766, 0.817]

Item Invariance

5

141,917

311,892

860

0.733

[0.700, 0.762]

Item Invariance

6

124,035

228,330

793

0.736

[0.702, 0.766]

Item Invariance

7

105,505

190,601

750

0.706

[0.668, 0.740]

Item Invariance

8

99,419

196,220

791

0.705

[0.668, 0.738]

Note: Counts of students include all measurement occasions and hence may include the same unique student tested more than once.
 

The i-Ready Diagnostic and Growth Monitoring tests are computer adaptive, meaning the items presented to each student vary depending upon how the student has responded to the previous items. Upon completion of an item randomly selected from a set of five items around a predetermined starting difficulty level, interim ability estimates are updated, and the next item is chosen relative to the new interim ability estimate. Thus, the items can better target the estimated student ability, and more information is obtained from each item presented.

 

Number of alternate forms of equal and controlled difficulty:

Virtually infinite. As a computer-adaptive test, in i-Ready all administrations are equivalent forms. However, each student is presented with an individualized testing experience where he or she is served test items based on answer choices to previous questions. In essence, this scenario provides a virtually infinite number of test forms, because individual student testing experiences are largely unique. For grades 1–8, typical item pool sizes are 1670, 1864, 2087, 2311, 2554, 2665, 2794, and 2913, respectively. Students who perform at an extremely high level will be served with items from grade levels higher than the grade level restriction.

Decision Rules: Setting and Revising Goals

Grade12345678K
Ratingdashdashdashdashdashdashdashdashdash

Specification of validated decision rules for when goals should be set or revised:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Decision Rules: Changing Instruction

Grade12345678K
Ratingdashdashdashdashdashdashdashdashdash

Specification of validated decision rules for when changes to instruction should be made:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Administration Format

Grade12345678K
Data
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Individual
  • Computer-administered
  • Administration & Scoring Time

    Grade12345678K
    Data
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • 15 minutes
  • Scoring Format

    Grade12345678K
    Data
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • ROI & EOY Benchmarks

    Grade12345678K
    Data
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • Specify the minimum acceptable rate of growth/improvement:

    For grades K–8, the tool’s reading growth targets over a 30-week period are 46, 47, 35, 25, 19, 17, 12, 10, and 9.

     

    Specify the benchmarks for minimum acceptable end-of-year performance:

    This information is provided directly to districts and schools as part of i-Ready’s support process.