Istation’s Indicators of Progress (ISIP)

Early Math

 

Cost

Technology, Human Resources, and Accommodations for Special Needs

Service and Support

Purpose and Other Implementation Information

Usage and Reporting

Initial Cost:

$5.95 per student

 

Replacement Cost:

$5.95 per student per year.


Annual license renewal fee subject to change.

 

Included in Cost:

ISIP Early Math is purchased as a yearly subscription. ISIP Early Math assessment packages includes online assessment, data hosting, reporting, teacher resources, online training center, user and manuals. In-person training conducted by a professional development specialist is available at additional cost ($2800 per specialist per day). Computers and/or tablets are needed to implement this assessment, as well as internet access. ISIP Early Math can be used on many different technology platforms including desktops, laptops, and tablets.

 

Technology Requirements:

  • Computer or tablet
  • Internet connection

 

Training Requirements:

  • 1-4 hours of training

 

Qualified Administrators:

  • Paraprofessionals
  • Professionals

 

Accommodations:

Appropriate accommodations are provided during ISIP assessments for students who are receiving support services, including those who have an Individual Education or 504 Plan, or who qualify as English Language learners. These accommodations support students’ access to the content of the assessment by reducing or eliminating the effects of the disability or limitation but do not change the content of the assessment. ISIP assessments provide people with disabilities access that is comparable to access for non-impaired people — with the exception of a totally blind or totally deaf disabled person. Administrators with manager accounts can assign accommodations to students in the Istation report and Management Portal.

ISIP supports or is compatible with the following types of accommodations:

  • Scribe
  • Touch screen overlay
  • ZoomText software
  • Extended time (Untimed Assessment feature)
  • Adjustable volume and/or headphones for students with hearing difficulties

Where to Obtain:

Website: www.istation.com

Address: 8150 North Central Expressway, Suite 2000, Dallas, TX, 75206

Phone number: (866) 883-READ

Email: info@istation.com

 

Access to Technical Support:

By email and phone (M-F 7am-6:30pm, CST).

 

ISIP Early Math is a web-based computer adaptive formative assessment intended for students in Pre-Kindergarten through Grade 1. The assessment is intended to provide teachers and administrators with student test results to answer two questions: (a) whether students are at risk of failure, and (b) the degree of intensity of instructional support students need to be successful. ISIP Early Math can be administered individually or in group settings. The assessment is administered in English. The assessment is untimed; however, most students complete the assessment within 30 minutes. There is no additional scoring time required for the assessment. Teachers can be trained on ISIP Math through either a webinar or an in-person training session. Training takes between 1 and 4 hours. All training materials are online and are created by Istation. Reports are available for both individual and groups of students indicating single administration results and comparisons of results over time. All reports include student scaled scores and tier levels based on student percentiles.

Assessment Format:

  • Performance measure
  • Direct: Computerized
  • One-to-one

 

Administration Time:

  • 20-30 minutes per student
  • 20-30 minutes per group

 

Scoring Time:

  • Scoring is automatic

 

Scoring Method:

  • Calculated automatically

 

Scores Generated:

  • Raw score
  • Percentile score
  • IRT-based score
  • Composite scores
  • MetaMetrics Quantile Score

 

Classification Accuracy

GradeK1
Criterion 1 FallHalf-filled bubbleEmpty bubble
Criterion 1 WinterHalf-filled bubbleHalf-filled bubble
Criterion 1 SpringHalf-filled bubbleEmpty bubble
Criterion 2 Falldashdash
Criterion 2 Winterdashdash
Criterion 2 Springdashdash

Primary Sample

 

Criterion 1, Fall

Grade

K

1

Criterion

TEMA

TEMA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1793

1762

Base rate in the sample for children requiring intensive intervention

0.12

0.22

False Positive Rate

0.29

0.39

False Negative Rate

0.29

0.12

Sensitivity

0.71

0.88

Specificity

0.71

0.61

Positive Predictive Power

0.96

0.91

Negative Predictive Power

0.20

0.52

Overall Classification Rate

0.89

0.87

Area Under the Curve (AUC)

0.71

0.74

AUC 95% Confidence Interval Lower Bound

0.50

0.66

AUC 95% Confidence Interval Upper Bound

0.74

0.81

 

Criterion 1, Winter

Grade

K

1

Criterion

TEMA

TEMA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1883

1871

Base rate in the sample for children requiring intensive intervention

0.12

0.22

False Positive Rate

0.14

0.28

False Negative Rate

0.34

0.17

Sensitivity

0.66

0.83

Specificity

0.86

0.72

Positive Predictive Power

0.98

0.92

Negative Predictive Power

0.20

0.51

Overall Classification Rate

0.86

0.88

Area Under the Curve (AUC)

0.76

0.77

AUC 95% Confidence Interval Lower Bound

0.58

0.69

AUC 95% Confidence Interval Upper Bound

0.81

0.84

 

Criterion 1, Spring

Grade

K

1

Criterion

TEMA

TEMA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1984

1965

Base rate in the sample for children requiring intensive intervention

0.12

0.22

False Positive Rate

0.21

0.39

False Negative Rate

0.20

0.13

Sensitivity

0.80

0.87

Specificity

0.79

0.61

Positive Predictive Power

0.97

0.90

Negative Predictive Power

0.29

0.53

Overall Classification Rate

0.80

0.82

Area Under the Curve (AUC)

0.80

0.74

AUC 95% Confidence Interval Lower Bound

0.68

0.66

AUC 95% Confidence Interval Upper Bound

0.91

0.82

 

Additional Classification Accuracy

The following are provided for context and did not factor into the Classification Accuracy ratings.

 

Cross-Validation Sample

 

Criterion 1, Fall

Grade

K

1

Criterion

TEMA

TEMA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1793

1762

Base rate in the sample for children requiring intensive intervention

0.12

0.22

False Positive Rate

0.29

0.39

False Negative Rate

0.29

0.12

Sensitivity

0.71

0.88

Specificity

0.71

0.61

Positive Predictive Power

0.96

0.91

Negative Predictive Power

0.20

0.52

Overall Classification Rate

0.89

0.87

Area Under the Curve (AUC)

0.71

0.74

AUC 95% Confidence Interval Lower Bound

0.50

0.66

AUC 95% Confidence Interval Upper Bound

0.74

0.81

 

Criterion 1, Winter

Grade

K

1

Criterion

TEMA

TEMA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1883

1871

Base rate in the sample for children requiring intensive intervention

0.12

0.22

False Positive Rate

0.14

0.28

False Negative Rate

0.34

0.17

Sensitivity

0.66

0.83

Specificity

0.86

0.72

Positive Predictive Power

0.98

0.92

Negative Predictive Power

0.20

0.51

Overall Classification Rate

0.86

0.88

Area Under the Curve (AUC)

0.76

0.77

AUC 95% Confidence Interval Lower Bound

0.58

0.69

AUC 95% Confidence Interval Upper Bound

0.81

0.84

 

Criterion 1, Spring

Grade

K

1

Criterion

TEMA

TEMA

Cut points: Percentile rank on criterion measure

20th percentile

20th percentile

Cut points: Performance score (numeric) on criterion measure

Not Provided

Not Provided

Cut points: Corresponding performance score (numeric) on screener measure

1984

1965

Base rate in the sample for children requiring intensive intervention

0.12

0.22

False Positive Rate

0.21

0.39

False Negative Rate

0.20

0.13

Sensitivity

0.80

0.87

Specificity

0.79

0.61

Positive Predictive Power

0.97

0.90

Negative Predictive Power

0.29

0.53

Overall Classification Rate

0.80

0.82

Area Under the Curve (AUC)

0.80

0.74

AUC 95% Confidence Interval Lower Bound

0.68

0.66

AUC 95% Confidence Interval Upper Bound

0.91

0.82

 

Reliability

GradeK1
RatingFull bubbledFull bubbled
  1. Justification for each type of reliability reported, given the type and purpose of the tool: Cronbach’s (1951) coefficient alpha is typically used as an indicator of reliability across test items within a testing instance. However, Cronboch’s Alpha is not appropriate for any IRT based measure because alpha assumes that all students in the testing instance respond to a common set of items. Due to its very nature, students taking a CAT-based assessment, such as ISIP Math, will receive a custom set of items based on their initial estimates of ability and response patterns. Thus, students do not respond to a common set of items. The IRT analogue to classical internal consistency is marginal reliability (Bock & Mislevy, 1982) and thus applied to ISIP Math.

 

  1. Description of the sample(s), including size and characteristics, for each reliability analysis conducted: Samples were obtained from three school districts in Texas during the 2015-2016 school year. There were 171 kindergartners for beginning of the year assessment (BOY/Fall), 170 kindergartners for middle of the year assessment (MOY/Winter), 163 kindergartners for end of the year assessment (EOY/Spring), 217 first graders for beginning of the year assessment (BOY), 230 first graders for middle of the year assessment (MOY), and 221 first graders for end of the year assessment (EOY).

 

  1. Description of the analysis procedures for each reported type of reliability: IRT based reliability is derived from Classical Test Theory to Item Response Theory. IRT based reliability is computed at each administration. 

 

  1. Reliability of performance level score (e.g., model-based, internal consistency, inter-rater reliability).

Type of Reliability

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

IRT-based reliability

ISIP Math BOY

K

171

0.89

0.85

IRT-based reliability

ISIP Math MOY

K

170

0.92

0.89

IRT-based reliability

ISIP Math EOY

K

163

0.95

0.93

IRT-based reliability

ISIP Math BOY

1

217

0.92

0.89

IRT-based reliability

ISIP Math MOY

1

230

0.94

0.92

IRT-based reliability

ISIP Math EOY

1

221

0.94

0.92

 

Disaggregated Reliability

The following disaggregated reliability data are provided for context and did not factor into the Reliability rating.

Type of Reliability

Subgroup

Age or Grade

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

IRT-based reliability

Gender: Male

K

83

0.86

0.79

0.90

IRT-based reliability

Gender: Female

K

86

0.91

0.86

0.94

IRT-based reliability

Race: Black

K

16

0.91

0.75

0.96

IRT-based reliability

Race: Hispanic

K

67

0.88

0.81

0.92

IRT-based reliability

Race: White

K

79

0.90

0.84

0.93

IRT-based reliability

Eco-Dis: No

K

87

0.89

0.83

0.92

IRT-based reliability

Eco-Dis: Yes

K

82

0.89

0.83

0.92

IRT-based reliability

Gender: Male

K

78

0.90

0.84

0.93

IRT-based reliability

Gender: Female

K

87

0.94

0.90

0.96

IRT-based reliability

Race: Black

K

16

0.92

0.78

0.97

IRT-based reliability

Race: Hispanic

K

66

0.92

0.87

0.95

IRT-based reliability

Race: White

K

77

0.92

0.87

0.94

IRT-based reliability

Eco-Dis: No

K

83

0.91

0.86

0.94

IRT-based reliability

Eco-Dis: Yes

K

82

0.93

0.89

0.95

IRT-based reliability

Gender: Male

K

76

0.93

0.89

0.95

IRT-based reliability

Gender: Female

K

83

0.96

0.93

0.97

IRT-based reliability

Race: Black

K

16

0.94

0.83

0.97

IRT-based reliability

Race: Hispanic

K

61

0.95

0.91

0.97

IRT-based reliability

Race: White

K

77

0.94

0.90

0.96

IRT-based reliability

Eco-Dis: No

K

81

 

0.95

0.92

0.96

IRT-based reliability

Eco-Dis: Yes

K

 

78

 

0.95

0.92

0.96

IRT-based reliability

Gender: Male

1

101

 

0.92

0.88

0.94

IRT-based reliability

Gender: Female

1

115

0.92

0.88

0.94

IRT-based reliability

Race: Black

1

17

0.94

0.83

0.97

IRT-based reliability

Race: Hispanic

1

67

0.93

0.88

0.95

IRT-based reliability

Race: White

1

117

0.91

0.87

0.93

IRT-based reliability

Eco-Dis: No

1

109

0.90

0.85

0.93

IRT-based reliability

Eco-Dis: Yes

1

107

0.93

0.89

0.95

IRT-based reliability

Gender: Male

1

101

0.94

0.91

0.95

IRT-based reliability

Gender: Female

1

128

0.94

0.91

0.95

IRT-based reliability

Race: Black

1

17

0.92

0.78

0.97

IRT-based reliability

Race: Hispanic

1

77

0.95

0.92

0.96

IRT-based reliability

Race: White

1

120

0.94

0.91

0.95

IRT-based reliability

Eco-Dis: No

1

110

0.94

0.91

0.95

IRT-based reliability

Eco-Dis: Yes

1

119

0.94

0.91

0.95

IRT-based reliability

Gender: Male

1

98

0.95

0.92

0.96

IRT-based reliability

Gender: Female

1

122

0.94

0.91

0.95

IRT-based reliability

Race: Black

1

16

0.95

0.85

0.98

IRT-based reliability

Race: Hispanic

1

75

0.95

0.92

0.96

IRT-based reliability

Race: White

1

114

0.93

0.90

0.95

IRT-based reliability

Eco-Dis: No

1

110

0.93

0.89

0.95

IRT-based reliability

Eco-Dis: Yes

1

110

0.95

0.92

0.96

 

Validity

GradeK1
RatingEmpty bubbledFull bubbled
  1. Description of each criterion measure used and explanation as to why each measure is appropriate, given the type and purpose of the tool: Predictive validity examines the relation between performance on the screener and a criterion of similar content that is administered at some time in the future. On the other hand, Concurrent validity examines the relation between performance on the screener and a criterion of similar content that is administered at the same point in time.

Renaissance Learning’s STAR Math is a computerized adaptive test intended for students in Grades 1 through 8. STAR Math is designed to provide teachers and administrators with data that can be used for multiple purposes such as screening, placement, planning instruction, benchmarking, and outcomes measurement. It also provides educators with estimates of students’ instructional math levels relative to national norms. Because STAR Math assesses a similar construct and has a similar use, STAR Math was used to provide concurrent validity for ISIP Math.

The Test of Early Mathematics Ability –Third Edition (TEMA-3) was selected to provide concurrent validity for ISIP Math because it assesses a similar construct. TEMA-3 is intended to identify children who are significantly behind or ahead of their peers in the development of mathematical thinking (Ginsburg & Baroody, 2003). The TEMA-3 is intended to be administered at the beginning of the school year as an early indicator of students’ abilities, but can also be administered later in the school year to assess student progress. Mathematical concepts and skills assessed in the TEMA-3 include relative magnitude, counting, calculation, convention, number facts, base 10 concepts, non-verbal production, non-verbal addition and subtraction, part whole concepts, equal partitioning, symbolic additive commutativity, number comparisons, and mental addition and subtraction.

 

  1. Description of the sample(s), including size and characteristics, for each validity analysis conducted: Samples are derived from urban school districts in the northeast area of the state of Texas approximately 200 students per each assessment point (Fall/Winter/Spring).

 

  1. Description of the analysis procedures for each reported type of validity: Data for this study was obtained from three school districts in Texas during the 2015-2016 school year. There were 178 kindergartners and 239 first graders. The validity was calculated by determining the correlation between the scaled scores of the ISIP Math and the scaled scores of the TEMA-3, and the STAR Math individually, by grade level.

 

  1. Validity for the performance level score (e.g., concurrent, predictive, evidence based on response processes, evidence based on internal structure, evidence based on relations to other variables, and/or evidence based on consequences of testing), and the criterion measures.

Type of Validity

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Concurrent Validity

K

TEMA-3

152

0.48

0.35

0.59

Concurrent Validity

1

TEMA-3

210

0.66

0.58

0.73

Concurrent Validity

1

STAR Math

208

0.66

0.58

0.73

Concurrent Validity

1

STAR Math

212

0.77

0.71

0.82

Concurrent Validity

1

STAR Math

213

0.72

0.65

0.78

Predictive Validity

K

TEMA-3

154

0.51

0.38

0.62

Predictive Validity

1

TEMA-3

199

0.65

0.56

0.72

Predictive Validity

K

TEMA-3

153

0.44

0.30

0.56

Predictive Validity

1

TEMA-3

215

0.74

0.67

0.80

 

 

  1. Describe the degree to which the provided data support the validity of the tool: The results suggest moderate to strong relationships between ISIP Math and STAR Math and TEMA-3. Our findings also show very convincing evidence across all 3 data points (Fall: beginning of the year (BOY), Winter: middle of the year (MOY), and Spring: end of the year (EOY) both K and Grade 1.

 

Disaggregated Validity

The following disaggregated validity data are provided for context and did not factor into the Validity rating.

Type of Validity

Subgroup

Age or Grade

Test or Criterion

n

Coefficient

95% Confidence Interval: Lower Bound

95% Confidence Interval: Upper Bound

Concurrent Validity

Gender: Male

1

STAR Math

108

0.66

0.54

0.75

Concurrent Validity

Gender: Female

1

STAR Math

104

0.67

0.55

0.76

Concurrent Validity

Race: Black

1

STAR Math

15

0.23

-0.32

0.66

Concurrent Validity

Race: Hispanic

1

STAR Math

64

0.70

0.55

0.81

Concurrent Validity

Race: White

1

STAR Math

113

0.71

0.60

0.79

Concurrent Validity

Eco-Dis: No

1

STAR Math

104

0.69

0.57

0.78

Concurrent Validity

Eco-Dis: Yes

1

STAR Math

103

0.63

0.50

0.73

Concurrent Validity

Gender: Male

1

STAR Math

115

0.77

0.68

0.84

Concurrent Validity

Gender: Female

1

STAR Math

96

0.78

0.69

0.85

Concurrent Validity

Race: Black

1

STAR Math

15

0.77

0.43

0.92

Concurrent Validity

Race: Hispanic

1

STAR Math

67

0.78

0.66

0.86

Concurrent Validity

Race: White

1

STAR Math

115

0.77

0.68

0.84

Concurrent Validity

Eco-Dis: No

1

STAR Math

105

0.79

0.71

0.85

Concurrent Validity

Eco-Dis: Yes

1

STAR Math

106

0.74

0.64

0.82

Concurrent Validity

Gender: Male

1

STAR Math

110

0.69

0.58

0.78

Concurrent Validity

Gender: Female

1

STAR Math

96

0.75

0.65

0.83

Concurrent Validity

Race: Black

1

STAR Math

16

0.80

0.50

0.93

Concurrent Validity

Race: Hispanic

1

STAR Math

71

0.66

0.50

0.77

Concurrent Validity

Race: White

1

STAR Math

111

0.73

0.63

0.81

Concurrent Validity

Eco-Dis: No

1

STAR Math

106

0.71

0.60

0.79

Concurrent Validity

Eco-Dis: Yes

1

STAR Math

106

0.71

0.60

0.79

Concurrent Validity

Gender: Male

K

TEMA-3

78

0.55

0.37

0.69

Concurrent Validity

Gender: Female

K

TEMA-3

73

0.43

0.22

0.60

Concurrent Validity

Race: Black

K

TEMA-3

16

0.17

-0.36

0.61

Concurrent Validity

Race: Hispanic

K

TEMA-3

57

0.43

0.19

0.62

Concurrent Validity

Race: White

K

TEMA-3

73

0.66

0.51

0.77

Concurrent Validity

Eco-Dis: No

K

TEMA-3

78

0.45

0.25

0.61

Concurrent Validity

Eco-Dis: Yes

K

TEMA-3

73

0.44

0.23

0.61

Concurrent Validity

Gender: Male

1

TEMA-3

115

0.66

0.54

0.75

Concurrent Validity

Gender: Female

1

TEMA-3

94

0.67

0.54

0.77

Concurrent Validity

Race: Black

1

TEMA-3

14

0.62

0.13

0.87

Concurrent Validity

Race: Hispanic

1

TEMA-3

71

0.64

0.48

0.76

Concurrent Validity

Race: White

1

TEMA-3

110

0.68

0.56

0.77

Concurrent Validity

Eco-Dis: No

1

TEMA-3

105

0.64

0.51

0.74

Concurrent Validity

Eco-Dis: Yes

1

TEMA-3

104

0.64

0.51

0.74

Predictive Validity

Gender: Male

K

TEMA-3

79

0.60

0.44

0.72

Predictive Validity

Gender: Female

K

TEMA-3

75

0.39

0.18

0.57

Predictive Validity

Race: Black

K

TEMA-3

16

0.30

-0.23

0.69

Predictive Validity

Race: Hispanic

K

TEMA-3

60

0.41

0.17

0.60

Predictive Validity

Race: White

K

TEMA-3

73

0.65

0.49

0.77

Predictive Validity

Eco-Dis: No

K

TEMA-3

80

0.52

0.34

0.66

Predictive Validity

Eco-Dis: Yes

K

TEMA-3

74

0.48

0.28

0.64

Predictive Validity

Gender: Male

1

TEMA-3

105

0.61

0.47

0.72

Predictive Validity

Gender: Female

1

TEMA-3

93

0.71

0.59

0.80

Predictive Validity

Race: Black

1

TEMA-3

14

0.37

-0.20

0.75

Predictive Validity

Race: Hispanic

1

TEMA-3

63

0.72

0.57

0.82

Predictive Validity

Race: White

1

TEMA-3

107

0.65

0.52

0.75

Predictive Validity

Eco-Dis: No

1

TEMA-3

101

0.65

0.52

0.75

Predictive Validity

Eco-Dis: Yes

1

TEMA-3

97

0.63

0.49

0.74

Predictive Validity

Gender: Male

K

TEMA-3

79

0.51

0.33

0.66

Predictive Validity

Gender: Female

K

TEMA-3

73

0.39

0.18

0.57

Predictive Validity

Race: Black

K

TEMA-3

16

0.23

-0.30

0.65

Predictive Validity

Race: Hispanic

K

TEMA-3

60

0.34

0.09

0.55

Predictive Validity

Race: White

K

TEMA-3

71

0.62

0.45

0.75

Predictive Validity

Eco-Dis: No

K

TEMA-3

78

0.43

0.23

0.60

Predictive Validity

Eco-Dis: Yes

K

TEMA-3

74

0.46

0.26

0.62

Predictive Validity

Gender: Male

1

TEMA-3

119

0.75

0.66

0.82

Predictive Validity

Gender: Female

1

TEMA-3

95

0.72

0.61

0.80

Predictive Validity

Race: Black

1

TEMA-3

14

0.70

0.27

0.90

Predictive Validity

Race: Hispanic

1

TEMA-3

74

0.73

0.60

0.82

Predictive Validity

Race: White

1

TEMA-3

112

0.73

0.63

0.81

Predictive Validity

Eco-Dis: No

1

TEMA-3

105

0.74

0.64

0.82

Predictive Validity

Eco-Dis: Yes

1

TEMA-3

109

0.71

0.60

0.79

 

Results for other forms of disaggregated validity (e.g. factor analysis) not conducive to the table format: Not Provided

 

Sample Representativeness

GradeK1
Data
  • Local with Cross-Validation
  • Local with Cross-Validation
  • Primary Classification Accuracy Sample

     

    Criterion 1, Fall

    Grade

    K

    1

    Criterion

    TEMA

    TEMA

    National/Local Representation

    All samples were from Texas

    All samples were from Texas

    Date

    September – October 2015

    September – October 2015

    Sample Size

    171

    217

    Male

    86

    115

    Female

    83

    101

    Gender Unknown

    2

    1

    Free or Reduced-price Lunch Eligible

    82

    107

    White, Non-Hispanic

    77

    117

    Black, Non-Hispanic

    17

    17

    Hispanic

    67

    67

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    10

    16

    Disability Classification

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

     

    Criterion 1, Winter

    Grade

    K

    1

    Criterion

    TEMA

    TEMA

    National/Local Representation

    All samples were from Texas

    All samples were from Texas

    Date

    January – February 2016

    January – February 2016

    Sample Size

    170

    230

    Male

    87

    128

    Female

    78

    101

    Gender Unknown

    5

    1

    Free or Reduced-price Lunch Eligible

    82

    119

    White, Non-Hispanic

    77

    120

    Black, Non-Hispanic

    16

    17

    Hispanic

    66

    77

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    11

    16

    Disability Classification

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

     

    Criterion 1, Spring

    Grade

    K

    1

    Criterion

    TEMA

    TEMA

    National/Local Representation

    All samples were from Texas

    All samples were from Texas

    Date

    May – June 2016

    May – June 2016

    Sample Size

    163

    221

    Male

    83

    122

    Female

    76

    98

    Gender Unknown

    4

    1

    Free or Reduced-price Lunch Eligible

    78

    110

    White, Non-Hispanic

    77

    114

    Black, Non-Hispanic

    16

    16

    Hispanic

    61

    75

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    9

    16

    Disability Classification

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

     

     

    Cross Validation Sample

    Criterion 1, Fall

    Grade

    K

    1

    Criterion

    TEMA-3

    TEMA-3

    National/Local Representation

    All samples were from Texas

    All samples were from Texas

    Date

    September – October 2015

    September – October 2015

    Sample Size

    171

    217

    Male

    86

    115

    Female

    83

    101

    Gender Unknown

    2

    1

    Free or Reduced-price Lunch Eligible

    82

    107

    White, Non-Hispanic

    77

    117

    Black, Non-Hispanic

    17

    17

    Hispanic

    67

    67

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    10

    16

    Disability Classification

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

     

    Criterion 1, Winter

    Grade

    K

    1

    Criterion

    TEMA-3

    TEMA-3

    National/Local Representation

    All samples were from Texas

    All samples were from Texas

    Date

    January – February 2016

    January – February 2016

    Sample Size

    170

    230

    Male

    87

    128

    Female

    78

    101

    Gender Unknown

    5

    1

    Free or Reduced-price Lunch Eligible

    82

    119

    White, Non-Hispanic

    77

    120

    Black, Non-Hispanic

    16

    17

    Hispanic

    66

    77

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    11

    16

    Disability Classification

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

     

    Criterion 1, Spring

    Grade

    K

    1

    Criterion

    TEMA-3

    TEMA-3

    National/Local Representation

    All samples were from Texas

    All samples were from Texas

    Date

    May – June 2016

    May – June 2016

    Sample Size

    163

    221

    Male

    83

    122

    Female

    76

    98

    Gender Unknown

    4

    1

    Free or Reduced-price Lunch Eligible

    78

    110

    White, Non-Hispanic

    77

    114

    Black, Non-Hispanic

    16

    16

    Hispanic

    61

    75

    American Indian/Alaska Native

    Not Provided

    Not Provided

    Other

    Not Provided

    Not Provided

    Race/Ethnicity Unknown

    9

    16

    Disability Classification

    Not Provided

    Not Provided

    First Language

    Not Provided

    Not Provided

    Language Proficiency Status

    Not Provided

    Not Provided

     

    Bias Analysis Conducted

    GradeK1
    RatingYesYes
    1. Description of the method used to determine the presence or absence of bias: Differential Item Functioning (DIF) analysis was conducted by grade level (K - 1) using logistic regression DIF detection analysis by difR package in R software.

     

    1. Description of the subgroups for which bias analyses were conducted: Four DIF factors were investigated: socioeconomic status, gender, race/ethnicity, and special education students.

     

    1. Description of the results of the bias analyses conducted, including data and interpretative statements: Using Zumbo & Thomas (ZT) DIF criterion, results showed 97% displayed as A item (negligible or non-significant DIF effect), 2% displayed as B item (slightly to moderate DIF effect), and only 1% displayed as C item (moderate to large DIF effect) across these four DIF factors.

     

    Administration Format

    GradeK1
    Data
  • Individual
  • Group
  • Individual
  • Group
  • Administration & Scoring Time

    GradeK1
    Data
  • 20-30 minutes
  • 20-30 minutes
  • Scoring Format

    GradeK1
    Data
  • Automatic
  • Automatic
  • Types of Decision Rules

    GradeK1
    Data
  • None
  • None
  • Evidence Available for Multiple Decision Rules

    GradeK1
    Data
  • No
  • No