Achieve Intervention
Study: Tracey & Young (2004)

Describe how students were selected to participate in the study:

Students from five schools and 11 classrooms participated in the study. Four classrooms received differentiated instruction with KidBiz3000; three classrooms received non-differentiated instruction with KidBiz3000; and four classrooms participated in the control condition.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

N/A

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Four classrooms received differentiated instruction with KidBiz3000.

Specify which condition is the control condition:

Four classrooms participated in the control condition, i.e., business as usual.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Three classrooms received non-differentiated instruction with KidBiz3000.

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Systematic

Please describe the assignment method or the process for defining treatment/comparison groups.

The researchers designed a project that tested the use of KidBiz3000 in two conditions--differentiated and non-differentiated. The differentiated condition provided access to KidBiz3000 using its full capability, that is, it provided students with reading materials and follow-up activities at their individual level of reading ability. The non-differentiated condition provided access to the same KidBiz3000 reading material and follow-up activities, but all materials were presented at the students’ grade of difficulty. Additionally, the study included a control condition in which students did not receive any exposure to the KidBiz3000 learning system. The KidBiz3000 program was made available to seven classrooms; four received differentiated instruction (n=84) and three received non-differentiated instruction (n=51). Four additional classrooms participated in the control condition (n=84).

What was the unit of assignment?

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

KidBiz3000’s standard, “On-Site Teacher Training Model” professional development program was provided to the participating district’s teachers prior to program implementation. All of the teachers in the differentiated and non-differentiated classrooms received direct professional development from a KidBiz3000 trainer. The professional development program consisted of two, large group, two-and-a-half hour sessions, and one small group one-hour session. Topics covered in the professional development sessions included: a program overview, suggestions for implementation, trouble-shooting recommendations, the sharing of successes and concerns, and ideas for using the program to strengthen students’ comprehension skills. Additionally, the trainer made several classrooms visits in response to individual requests by the teachers.

Describe when and how fidelity of treatment information was obtained.

Fidelity of implementation data is collected automatically within the Achieve3000 system as students use the program. Specifically, the Achieve3000 system collects data related to the number of sessions completed as well as students’ scores on formative assessments that provide a reliable indicator of quality of program use.

What were the results on the fidelity-of-treatment implementation measure?

Not applicable.

Was the fidelity measure also used in control classrooms?

No.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

1)The SouthEast and Islands Regional Technology in Education Consortium Teacher Survey: Since only three teachers completed both the pre-test and post-test administration of this survey, no analyses were conducted. Given the small sample size, any significant results would have been highly unlikely and highly suspect upon interpretation. 2)The SouthEast and Islands Regional Technology in Education Consortium Student Survey & 3)Elementary Reading Attitude Survey: The data requested in the “Results” section of Study Form A was not included in the report.

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

Comparisons of the treatment and control groups using a one-tail t-test.