Reading Plus

InSight

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

The Reading Plus program is offered on a subscription basis. There are many available options for license duration and cost. Please contact Reading Plus at https://www.readingplus.com/contact/learn-more/ to learn more.

 

Replacement Cost:

No information provided; contact vendor for details.

 

Included in Cost:

Customer support, including training and start-up assistance, continuing education, and offline learning materials, are all included in the subscription cost.

 

Technology Requirements*:

  • Computer or tablet
  • Internet connection

*Reading Plus does not have any additional technology requirements, but does integrate with Clever and ClassLink if districts are using those solutions.

 

Training Requirements:

  • Training not required*

* The assessment is fully automated including orientation videos for students. Teacher training is not required, but Reading Plus has optional trainings and online resources that provide teachers with an overview of the assessment as well as guidance with scheduling and monitoring and the interpretation of results.

 

Qualified Administrators:

  • No minimum qualifications specified

 

Accommodations:

The assessment is based on an IRT framework, and the difficulty of items presented to students is adjusted dynamically based on ongoing student performance. Adjustments to startup parameters (content level and other parameters) are implemented if a student is identified as having special needs. Students can complete the assessment on computers or tablets using a keyboard, mouse, or touch screen, thereby providing several options to accommodate students with limited motor skills.

 

Close captioning for the hearing impaired is provided for all instructional videos.  Students may complete the assessment at their own pace, taking as much time and as many breaks as needed. Flexible scheduling is possible if students need multiple days to complete the assessment. If students log out before completing the assessment, they are bookmarked so that they can later continue from where they left off. For students with limited vision, the assessment is compatible with the in-browser zooming feature, allowing users to magnify all assessment screens.

Where to Obtain:

Website: https://www.readingplus.com/

Address: 110 West Canal Street, Winooski, VT 05404

Phone number: 800-732-3758

Email: sales@readingplus.com


Access to Technical Support:

A comprehensive system of support is available for educators who use Reading Plus. The Reading Plus Learn Site is the main portal for online video and webinar training, written resources, and additional online and offline teaching tools. Reading Plus Support is available via live chat, email, and phone to assist with any additional questions educators may have.

 

Reading Plus provides a web-based computer-adaptive assessment that measures three dimensions essential to successful independent silent reading: capacity, fluency, and motivation. In addition to assessing comprehension and vocabulary (capacity), it includes measures of comprehension-based silent reading rate (fluency) and motivation for reading. The comprehension and vocabulary components of the assessment are based on an IRT framework. The difficulty of items presented to students during the assessment is adjusted dynamically based on ongoing student performance.

 

The assessment may be used with all students or with specific groups of students who have been identified as at risk of academic failure and may be individually or group administered. The initial administration of the assessment provides baseline performance measures, while subsequent administrations (mid-term and end-of-term) provide progress benchmarking.

 

The assessment automatically scores and reports each student’s academic performance in terms of comprehension level, vocabulary level, and comprehension-based silent reading rate (in words per minute). These sub-scores are combined to provide the student’s overall reading Proficiency Index. Comprehension and Vocabulary Levels as well as the Reading Proficiency Index are reported on a grade-level scale. The assessment also measures students’ self-reported motivation for reading across several Motivation domains. Summary reports become available as soon as the student completes the assessment. If the Reading Plus instructional program has also been purchased, the assessment governs initial placement and an individualized instructional path within the various program components.

Assessment Format:

  • Performance measure
  • Questionnaire
  • Direct: Computerized

 

Administration Time:

  • 30-45 minutes per student
  • 30-45 minutes per group

 

Scoring Time:

  • Scoring is automatic

 

Scoring Method:

  • Calculated automatically

 

Scores Generated:

  • Text Lexile Range
  • Instructional Reading Level
  • Instructional Vocabulary Level
  • Visual Skills Instructional Level
  • Percentile score*
  • IRT-based score**
  • Composite scores***

* Percentile score available for comprehension-based silent reading rate.

**Grade-level scale score based on IRT scoring.

*** Sub-scores are available in the following areas: Comprehension Grade Level, Vocabulary Grade Level, Comprehension-based Silent Reading Rate (in words per minute), and Reading Motivation (e.g., self-improvement belief, confidence, interest).

 

 

Classification Accuracy

Grade345678
Criterion 1 FallHalf-filled bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
Criterion 1 Winterdashdashdashdashdashdash
Criterion 1 SpringFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
Criterion 2 Falldashdashdashdashdashdash
Criterion 2 Winterdashdashdashdashdashdash
Criterion 2 Springdashdashdashdashdashdash

Primary Sample

 

Criterion 1, Fall

Grade

3

4

5

6

7

8

Criterion

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

20th percentile

20th percentile

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

0.5

1.0

2.0

3.0

3.5

4.0

Base rate in the sample for children requiring intensive intervention

0.24

0.21

0.20

0.22

0.22

0.21

False Positive Rate

0.09

0.12

0.13

0.16

0.14

0.16

False Negative Rate

0.25

0.27

0.19

0.13

0.22

0.27

Sensitivity

0.75

0.73

0.81

0.87

0.78

0.73

Specificity

0.91

0.88

0.87

0.84

0.86

0.84

Positive Predictive Power

0.71

0.62

0.60

0.60

0.62

0.55

Negative Predictive Power

0.92

0.92

0.95

0.96

0.93

0.92

Overall Classification Rate

0.87

0.85

0.86

0.84

0.84

0.82

Area Under the Curve (AUC)

0.81

0.85

0.91

0.92

0.91

0.88

AUC 95% Confidence Interval Lower Bound

0.79

0.84

0.89

0.91

0.89

0.86

AUC 95% Confidence Interval Upper Bound

0.83

0.87

0.92

0.94

0.93

0.90

 

Criterion 1, Spring

Grade

3

4

5

6

7

8

Criterion

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Smarter Balanced ELA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

20th percentile

20th percentile

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1.0

2.5

3.0

3.5

4.0

5.0

Base rate in the sample for children requiring intensive intervention

0.19

0.18

0.16

0.19

0.19

0.18

False Positive Rate

0.14

0.11

0.09

0.10

0.14

0.18

False Negative Rate

0.14

0.20

0.24

0.23

0.22

0.26

Sensitivity

0.86

0.80

0.76

0.77

0.78

0.74

Specificity

0.86

0.89

0.91

0.90

0.86

0.82

Positive Predictive Power

0.59

0.61

0.62

0.64

0.57

0.47

Negative Predictive Power

0.96

0.95

0.95

0.94

0.94

0.94

Overall Classification Rate

0.86

0.87

0.89

0.87

0.85

0.81

Area Under the Curve (AUC)

0.91

0.93

0.92

0.92

0.91

0.87

AUC 95% Confidence Interval Lower Bound

0.89

0.91

0.90

0.90

0.89

0.84

AUC 95% Confidence Interval Upper Bound

0.93

0.95

0.94

0.93

0.93

0.89

 

Reliability

Grade345678
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
  1. Justification for each type of reliability reported, given the type and purpose of the tool:

a. Marginal Reliability:

The Reading Plus assessment includes a large bank of test items designed to evaluate a student’s vocabulary knowledge and reading comprehension level relative to their grade level. In an IRT-based model, marginal reliability provides an estimate of how well an assessment classifies students in relation to what is measured. Like traditional reliability coefficients, marginal reliability estimates are reported on a scale of 0 to 1. Higher values for marginal reliability indicate that an instrument is more sensitive to individual differences in the ability being measured; e.g., higher-performing students are more likely to achieve higher scores, and vice versa.

b. Test-Retest Reliability:

The Reading Plus assessment can be administered up to three times per school year and yields several sub scores (vocabulary, comprehension, comprehension-based reading rate, and motivation measures) as well as an overall reading proficiency composite score. Unlike the classic alternate form testing scenario, however, this assessment is an adaptive test with large content pools and with logic designed to avoid the repetition of content on successive administrations. As such, the specific content of each administration differs, as might the difficulty of the items in cases where a student has advanced. Under these circumstances, the calculated reliability coefficients reflect what some have called a “stratified, randomly parallel form reliability” (Green, Bock, Humphreys, Linn, & Reckase, 1984, p. 353).

The interval between assessments can vary according to the goals of the administrator; e.g., to evaluate short-term versus full year gains. Under these circumstances, test-retest reliability coefficients are useful as estimates of score stability over these time spans. With this in mind, two test-retest reliability coefficients were calculated for each grade:

i. One based on all students who completed two assessments with fewer than 60 days between administrations (actual mean test-retest intervals across grades ranged from 31 to 41 days).

ii. Another based on a different sample consisting of all students who completed two assessments separated by at least 180 days (actual mean test-retest intervals across grades ranged from 226 to 251 days).

 

  1. Description of the sample(s), including size and characteristics, for each reliability analysis conducted: Calculations of marginal and test-retest reliability were calculated using all participating students in 48 states and the District of Columbia who fit the inclusion criteria; i.e., for whom valid scores were available on at least two Reading Plus assessments (a) administered within 60 days of each other or (b) administered more than 180 days apart, during the 2016-2017 school year.

 

  1. Description of the analysis procedures for each reported type of reliability: Marginal reliability statistics were calculated separately for both the vocabulary and comprehension components of the assessment using the Winsteps Rasch-Model program (Linacre, 2006). Marginal reliability statistics were calculated separately for students at each grade level. Confidence intervals were also calculated by the Winsteps program (lower bound is the “real” person reliability; upper bound is the “model” person reliability).

To evaluate test-retest reliability, Pearson correlation coefficients were calculated separately at each grade level for two independent samples (students with two assessments within 60 days, and others with two assessments separated by at least 180 days) using data collected in the Fall of 2016 and Spring of 2017.

The 95% confidence intervals around each test-retest correlation coefficient was calculated by (a) using Fisher's z' transformation to convert the coefficients to z' scores, (b) calculating the associated standard errors and 95% confidence intervals for z’, and then (c) converting the z’ score confidence intervals back into regression coefficients.

 

  1. Reliability of performance level score (e.g., model-based, internal consistency, inter-rater reliability).

Type of Reliability

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Marginal (comprehension)

3

6,563

0.92

0.92

0.93

Marginal (comprehension)

4

19,873

0.93

0.93

0.94

Marginal (comprehension)

5

26,699

0.95

0.95

0.95

Marginal (comprehension)

6

29,790

0.95

0.95

0.95

Marginal (comprehension)

7

35,576

0.95

0.95

0.95

Marginal (comprehension)

8

35,261

0.95

0.95

0.95

Marginal (comprehension)

11

18,339

0.94

0.94

0.95

Marginal (vocabulary)

3

6,557

0.96

0.96

0.96

Marginal (vocabulary)

4

19,901

0.96

0.96

0.96

Marginal (vocabulary)

5

26,621

0.96

0.96

0.96

Marginal (vocabulary)

6

29,796

0.96

0.96

0.96

Marginal (vocabulary)

7

35,458

0.96

0.96

0.96

Marginal (vocabulary)

8

35,193

0.96

0.96

0.96

Marginal (vocabulary)

11

18,288

0.96

0.96

0.96

Test-retest (< 2 months)

3

291

0.70

0.64

0.76

Test-retest (< 2 months)

4

368

0.78

0.74

0.82

Test-retest (< 2 months)

5

459

0.78

0.75

0.82

Test-retest (< 2 months)

6

903

0.79

0.77

0.81

Test-retest (< 2 months)

7

1,090

0.80

0.78

0.82

Test-retest (< 2 months)

8

1,051

0.78

0.75

0.80

Test-retest (< 2 months)

11

670

0.76

0.73

0.79

Test-retest (> 6 months)

3

2,976

0.67

0.65

0.69

Test-retest (> 6 months)

4

5,798

0.73

0.72

0.74

Test-retest (> 6 months)

5

8,435

0.76

0.75

0.77

Test-retest (> 6 months)

6

25,223

0.80

0.79

0.80

Test-retest (> 6 months)

7

25,556

0.81

0.81

0.81

Test-retest (> 6 months)

8

24,565

0.81

0.81

0.82

Test-retest (> 6 months)

11

6,256

0.76

0.75

0.77

 

Disaggregated Reliability

The following disaggregated reliability data are provided for context and did not factor into the Reliability rating.

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

Validity

Grade345678
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubbleFull bubble
  1. Description of each criterion measure used and explanation as to why each measure is appropriate, given the type and purpose of the tool: The criterion measures used in this analysis were English-Language Arts (ELA) scores from the spring 2017 administration of the Smarter Balanced assessment (SBAC; Smarter Balanced Assessment Consortium, 2016). This criterion measure is appropriate because:
    1. SBAC is among the most widely administered assessments in the United States (administered in 13 states plus the US Virgin Islands and the Bureau of Indian Education).
    2. The ELA portion of the SBAC was developed for the explicit purpose of providing a valid and reliable measure of literacy achievement as outlined in the Common Core State Standards (2010).
    3. Evidence supporting the validity of the SBAC derives from numerous sources, including evaluations of test content, internal structure, test bias, scoring reliability, and alignment with other measures.

 

  1. Description of the sample(s), including size and characteristics, for each validity analysis conducted: The sample included all students in grades 3-8 who completed both the SBAC and a valid Reading Plus assessment during the 2016-2017 school year, and who attended school districts that shared SBAC data. This comprised a fall sample of 9,688 students and a spring sample of 6,179 students in school districts located in the Northeastern, Midwestern, and Western regions of the United States.

 

  1. Description of the analysis procedures for each reported type of validity: Two types of validity were evaluated:
  1. Concurrent validity was evaluated by calculating Pearson correlation coefficients comparing proficiency scores on the Spring 2017 administration of the Reading Plus InSight assessment with Spring 2017 ELA scaled scores on the SBAC.
  2. Predictive validity was evaluated by calculating Pearson correlation coefficients comparing proficiency scores on the Fall 2016 administration of the Reading Plus InSight assessment with ELA scaled scores on the Spring 2017 administrations of SBAC.
  3. The 95% confidence intervals around each correlation coefficient was calculated by (a) using Fisher's z' transformation to convert the coefficients to z' scores, (b) calculating the associated standard errors and 95% confidence intervals for z’, and then (c) converting the z’ score confidence intervals back into regression coefficients.

 

  1. Validity for the performance level score (e.g., concurrent, predictive, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Concurrent (spring / spring)

3

SBAC

673

0. 80

0.77

0.82

Concurrent (spring / spring)

4

SBAC

763

0.82

0.79

0.84

Concurrent (spring / spring)

5

SBAC

768

0.80

0.78

0.83

Concurrent (spring / spring)

6

SBAC

1,620

0.81

0.79

0.82

Concurrent (spring / spring)

7

SBAC

1,300

0.81

0.79

0.83

Concurrent (spring / spring)

8

SBAC

1,055

0.78

0.76

0.81

Predictive (fall/spring)

3

SBAC

1,444

0.72

0.69

0.74

Predictive (fall/spring)

4

SBAC

1,817

0.75

0.73

0.77

Predictive (fall/spring)

5

SBAC

1,688

0.79

0.77

0.8

Predictive (fall/spring)

6

SBAC

1,923

0.79

0.76

0.81

Predictive (fall/spring)

7

SBAC

1,591

0.80

0.78

0.82

Predictive (fall/spring)

8

SBAC

1,225

0.78

0.76

0.84

 

  1. Results for other forms of validity (e.g. factor analysis) not conducive to the table format: Not Provided

 

  1. Describe the degree to which the provided data support the validity of the tool: Not Provided  

 

 

Disaggregated Validity

The following disaggregated validity data are provided for context and did not factor into the Validity rating.

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

None

 

 

 

 

 

 

 

 

Results for other forms of disaggregated validity (e.g. factor analysis) not conducive to the table format: Not Provided

 

Sample Representativeness

Grade345678
Data
  • National without Cross-Validation
  • National without Cross-Validation
  • National without Cross-Validation
  • Regional without Cross-Validation
  • Regional without Cross-Validation
  • Regional without Cross-Validation
  • Primary Classification Accuracy Sample

     

    Criterion 1, Fall

    Grade

    3

    4

    5

    6

    7

    8

    Criterion

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    National/Local Representation

    New England, West North Central,
    Pacific

    New England, West North Central,
    Pacific

    New England, West North Central,
    Pacific

    West North Central,
    Pacific

    West North Central,
    Pacific

    West North Central,
    Pacific

    Date

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    Sample Size

    1,444

    1,817

    1,688

    1,923

    1,591

    1,225

    Male

    52%

    52%

    50%

    50%

    52%

    48%

    Female

    48%

    48%

    50%

    50%

    48%

    52%

    Gender Unknown

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Free or Reduced-price Lunch Eligible

    29%

    39%

    43%

    53%

    66%

    85%

    White, Non-Hispanic

    47%

    41%

    40%

    33%

    22%

    12%

    Black, Non-Hispanic

    3%

    4%

    3%

    4%

    4%

    6%

    Hispanic

    27%

    38%

    41%

    51%

    62%

    76%

    American Indian/Alaska Native

    14%

    10%

    9%

    7%

    6%

    2%

    Other

    9%

    7%

    7%

    5%

    6%

    4%

    Race/Ethnicity Unknown

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Disability Classification

    12%

    10%

    10%

    8%

    11%

    8%

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    12%

    17%

    13%

    14%

    13%

    16%

     

    Criterion 1, Spring

    Grade

    3

    4

    5

    6

    7

    8

    Criterion

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    Smarter Balanced

    National/Local Representation

    New England, West North Central,
    Pacific

    New England, West North Central,
    Pacific

    New England, West North Central,
    Pacific

    West North Central,
    Pacific

    West North Central,
    Pacific

    West North Central,
    Pacific

    Date

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    SBAC – April 2017

    Sample Size

    673

    763

    768

    1,620

    1,300

    1,055

    Male

    53%

    51%

    49%

    49%

    50%

    52%

    Female

    47%

    49%

    51%

    51%

    50%

    48%

    Gender Unknown

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Free or Reduced-price Lunch Eligible

    29%

    39%

    43%

    56%

    74%

    86%

    White, Non-Hispanic

    72%

    69%

    68%

    31%

    17%

    12%

    Black, Non-Hispanic

    2%

    2%

    2%

    3%

    4%

    6%

    Hispanic

    6%

    7%

    9%

    54%

    69%

    76%

    American Indian/Alaska Native

    11%

    15%

    12%

    6%

    4%

    1%

    Other

    9%

    7%

    9%

    6%

    6%

    5%

    Race/Ethnicity Unknown

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Disability Classification

    11%

    11%

    11%

    8%

    9%

    6%

    First Language

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Not Provided

    Language Proficiency Status

    2%

    2%

    3%

    15%

    14%

    15%

     

    Bias Analysis Conducted

    Grade345678
    RatingYesYesYesYesYesYes
    1. Description of the method used to determine the presence or absence of bias: The Winsteps® program was used to investigate Differential Item Functioning (DIF) (Linacre, 2006).

     

    1. Description of the subgroups for which bias analyses were conducted: DIF analyses were conducted using males as a reference group and females as a focal group. Other DIF analyses could not be completed due to limited information identifying students in other demographic categories.

     

    1. Description of the results of the bias analyses conducted, including data and interpretative statements: The Educational Testing Service (ETS) delta method of categorizing DIF was used to differentiate items with negligible DIF (<.43 logits) from those with moderate DIF (>.43 to <.64 logits) and those with large DIF (>.64 logits) (Zwick, Thayer, & Lewis, 1999).

    Well over 90% of the items showed negligible DIF. Less than 1% of the comprehension items and 2.2% of the vocabulary items showed large DIF and have been pulled from the pool for revision.

     

    Administration Format

    Grade345678
    Data
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Individual
  • Group
  • Administration & Scoring Time

    Grade345678
    Data
  • 30-45 minutes
  • 30-45 minutes
  • 30-45 minutes
  • 30-45 minutes
  • 30-45 minutes
  • 30-45 minutes
  • Scoring Format

    Grade345678
    Data
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Automatic
  • Types of Decision Rules

    Grade345678
    Data
  • None
  • None
  • None
  • None
  • None
  • None
  • Evidence Available for Multiple Decision Rules

    Grade345678
    Data
  • No
  • No
  • No
  • No
  • No
  • No