iSTEEP

Oral Reading Fluency

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

No information provided; contact vendor for details.

 

Replacement Cost:

No information provided; contact vendor for details.

 

Included in Cost:

Progress monitoring tools are included within the iSTEEP Universal Screening and Progress Monitoring system. Student (self-report) forms can be downloaded for student use and are reusable. Progress monitoring forms for assessors use online scoring which allows assessors to mark errors online.  A web-based data management system is also available for storing, managing and reporting on student progress. Progress monitoring is one component of the iSTEEP assessment and data management system that includes screening, intervention, and progress monitoring.

Technology Requirements*:

  • Computer or tablet
  • Internet connection

*For computerized site license

 

Training Requirements:

  • 1 – 4 hours of training

 

Qualified Administrators:

  • Paraprofessionals
  • Professionals

 

Accommodations:

Includes standard accommodations for students with disabilities.

Where to Obtain:

Website:

www.isteep.com

Address:
iSTEEP 1302 Waugh, Suite #623, Houston, TX, 77019.

Phone:
800-881-9142

Email:
support@isteep.com


Access to Technical Support:

Support is provided through email, telephone and web.

The iSTEEP Oral Reading Fluency tool is designed for progress monitoring in the area of Oral Reading Fluency. It consists of 50 forms of equivalent difficulty at each grade 1-5.

Assessment Format:

  • Individual
  • Computer-administered*

*With computerized site license. Manual administration is also possible.

 

Administration Time:

  • 1 minute per student*

*With computerized site license. Manual administration is also possible.

 

Scoring Time:

  • Scoring is automatic*

*With computerized site license. Manual administration is also possible.

 

Scoring Method:

  • Calculated automatically*
  • Calculated manually

*With a computerized site license, each of the three equivalent forms is scored automatically and saved. At the end of the assessment, the median score for the three forms is saved as well as the progress monitoring score for the session.

 

Scores Generated:

  • Percentile Score
  • Raw Score

 

 

Reliability

Grade12345
RatingFull bubbleFull bubbleFull bubbleFull bubbleFull bubble

Justify the appropriateness of each type of reliability reported:

Two types of reliability are reported. Alternate form is justifiable given that PM with this assessment relies on equivalence and stability from form to form so this indicator is relevant. Inter-Rater reliability helps to estimate the degree to which two assessors score each student protocol in the same way.

 

Describe the sample characteristics for each reliability analysis conducted:

The students in this study were from medium sized suburban schools in Louisiana and Texas.

 

Describe the analysis procedures for each reported type of reliability:

Alternate form: Three alternate forms were administered in a single setting. Single Forms 1 and 3 were used within a correlational analysis.

Inter-rater: Audio recordings were made of students reading a single ORF passage. Two different experienced assessors then independently scored each recording. The two scoring protocols were examined for agreement on a word-by-word basis. The analysis of agreement consisted of dividing the total number of agreements by the number of agreements plus disagreements.

Type of Reliability

Age or Grade

n

Coefficient

Confidence Interval

Alternate Form

Grade 1

227

0.98

0.96-0.99

Alternate Form

Grade 2

204

0.95

0.89-0.98

Alternate Form

Grade 3

182

0.97

0.93-0.98

Alternate Form

Grade 4

241

0.94

0.85-0.98

Alternate Form

Grade 5

201

0.97

0.89-0.99

Inter-rater

Grade 1

35

0.98

0.71-0.99

Inter-rater

Grade 2

35

0.99

0.79-0.99

Inter-rater

Grade 3

35

0.98

0.71-0.99

Inter-rater

Grade 4

35

0.97

0.70-0.99

Inter-rater

Grade 5

35

0.97

0.70-0.99

 

 

Validity

Grade12345
RatingFull bubbleFull bubbleFull bubbleHalf-filled bubbleHalf-filled bubble

Describe and justify the criterion measures used to demonstrate validity:

Predictive: The iSTEEP Maze Assessment was used. Maze is a multiple-choice cloze assessment where, after the first sentence, every 7th word is replaced with three words inside parentheses. One of these words is correct. Given the theoretical and empirical linkage between fluency and comprehension, showing a relationship between fluency and the subsequent development of basic comprehension skills would support the validity of the ORF assessment. The vendor acknowledges that the use of a criterion that is not external to the iSTEEP system may cause concern. However maze and ORF are different ways of assessing reading which may help to assuage concerns about method variance. Also, item overlap is minimal in that the two assessments don’t rely on the same items types. The ORF assessment was administered within the second and third week of January and the Maze assessment was administered within the first two weeks of May.

Concurrent: For Grades 1-4, the Word Identification subtest from the Woodcock Reading Mastery Test was used. For Grade 5 the Louisiana State Accountability Test, ILEAP) was used.

 

Describe the sample characteristics for each validity analysis conducted:

The sample consisted of a diverse group of students. Approximately 25% of the sample was taken from each of four states: Arizona, Georgia, Indiana, and Texas. Students were from suburban and rural districts.

Predictive: The sample consisted of a diverse group of students. Approximately 25% of the sample was taken from each of four states: Arizona, Georgia, Indiana, and Texas. Students were from suburban and rural districts. Demographic breakdown was as follows:

Ethnicity:

African American 23% Asian 2%

Hispanic 18%

White, non-Hispanic 57%

Concurrent: The sample consisted of a diverse group of students from the State of Louisiana. Students were from suburban and rural districts. Demographic breakdown was as follows:

African American 53%

Asian 2%

Hispanic 2%

White, non-Hispanic 43%

 

Describe the analysis procedures for each reported type of validity:

Predictive: Bivariate correlation between the two measures was used to derive the validity coefficients.

Concurrent: Bivariate correlation between the two measures was used to derive the validity coefficients.

 

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

Confidence Interval

Predictive

Grade 1

iSTEEP Reading Maze

208

0.74

0.67-0.80

Predictive

Grade 2

iSTEEP Reading Maze

221

0.79

0.73-0.83

Predictive

Grade 3

iSTEEP Reading Maze

215

0.75

0.68-0.80

Predictive

Grade 4

iSTEEP Reading Maze

213

0.80

0.74-0.84

Predictive

Grade 5

iSTEEP Reading Maze

240

0.74

0.68-0.79

Concurrent

Grade 1

Woodcock Reading Mastery Tests (WRMT)

50

0.78

0.64-0.87

Concurrent

Grade 2

WRMT

50

0.79

0.66-0.88

Concurrent

Grade 3

WRMT

50

0.76

0.61-0.86

Concurrent

Grade 4

WRMT

50

0.73

0.57-0.84

Concurrent

Grade 5

WRMT

112

0.69

0.58-0.78

 

Describe the degree to which the provided data support the validity of the tool:

Validity coefficients are within an acceptable range given the expected relationship with the criterion measures.

Bias Analysis Conducted

Grade12345
RatingNoNoNoNoNo

Have additional analyses been conducted to establish whether the tool is or is not biased against demographic subgroups (e.g., students who vary by race/ethnicity, gender, socioeconomic status, students with disabilities, English language learners)?

Bias Analysis Method: No qualifying evidence provided.

 

Subgroups Included: No qualifying evidence provided.

 

Bias Analysis Results: No qualifying evidence provided.

Sensitivity: Reliability of the Slope

Grade12345
Ratingdashdashdashdashdash

Describe the sample used for analyses, including size and characteristics:

No qualifying evidence provided.

 

Describe the frequency of measurement:

No qualifying evidence provided.

 

Describe reliability of the slope analyses conducted with a population of students in need of intensive intervention:

No qualifying evidence provided.

Sensitivity: Validity of the Slope

Grade12345
Ratingdashdashdashdashdash

Describe and justify the criterion measures used to demonstrate validity:

No qualifying evidence provided.

 

Describe the sample used for analyses, including size and characteristics:

No qualifying evidence provided.

 

Describe predictive validity of the slope of improvement analyses conducted with a population of students in need of intensive intervention:

No qualifying evidence provided.

 

Describe the degree to which the provided data support the validity of the tool:

No qualifying evidence provided.

Alternate Forms

Grade12345
RatingEmpty bubbleEmpty bubbleEmpty bubbleEmpty bubbleEmpty bubble

Describe the sample for these analyses, including size and characteristics:

Sample Characteristics. The students in this study were from medium sized suburban schools in Louisiana and Texas. Demographics were:

Gender: F 51%; M 49%

Ethnicity: African American 39%; Asian 1%; Hispanic 5%; White, non-Hispanic 55%

 

Evidence that alternate forms are of equal and controlled difficulty or, if IRT based, evidence of item or ability invariance:

Alternate form reliability provides an indication of the consistency of a student’s score at two different points in time. It also provides an indicator of the consistency of response to different passages which is partially dependent of the equivalence of passages. Three alternate forms were administered in a single setting. Single Forms 1 and 3 were used within a correlational analysis.

Type of Reliability

Age or Grade

n

Coefficient

Confidence Interval

Alternate form

Grade 1

227

0.98

0.96-0.99

0Alternate form

Grade 2

204

0.95

0.89-0.98

Alternate form

Grade 3

182

0.97

0.93-0.98

Alternate form

Grade 4

241

0.94

0.85-0.98

Alternate form

Grade 5

201

0.97

0.89-0.99

 

Number of alternate forms of equal and controlled difficulty:

iSTEEP has 50 alternate forms for each grade 1-5.

Decision Rules: Setting and Revising Goals

Grade12345
Ratingdashdashdashdashdash

Specification of validated decision rules for when goals should be set or revised:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Decision Rules: Changing Instruction

Grade12345
Ratingdashdashdashdashdash

Specification of validated decision rules for when changes to instruction should be made:

No qualifying evidence provided.

 

Evidentiary basis for these rules:

No qualifying evidence provided.

Administration Format

Grade12345
Data
  • Individual
  • Individual
  • Individual
  • Individual
  • Individual
  • Administration Format:

    Individual

    Computer-administered

    Administration & Scoring Time

    Grade12345
    Data
  • 1 minute
  • 1 minute
  • 1 minute
  • 1 minute
  • 1 minute
  • Administration Time:

    1 minute

    Scoring Time:

    Not applicable

    Scoring Format

    Grade12345
    Data
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Computer-scored
  • Scoring Format:

    Computer-scored*

    Manually-scored

    *With computerized site license. Manual administration is also possible.

    ROI & EOY Benchmarks

    Grade12345
    Data
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • ROI & EOY Benchmarks Available
  • Specify the minimum acceptable rate of growth/improvement:

    iSTEEP suggests defining the minimum acceptable rate as an increase of 1-1.5 words per week. However, iSTEEP cautions against a using a single indicator and provides the means to also compute local growth norms and to chart progress toward end of year percentile goals.

     

    Specify the benchmarks for minimum acceptable end-of-year performance:

    Minimum performance:

    Grade 1: 43

    Grade 2: 74

    Grade 3: 91

    Grade 4: 108

    Grade 5: 118