Headsprout
Study: Huffstetter et al. (2010)

Describe how students were selected to participate in the study:

Children from two Head Start centers participated in the study. Both centers were randomly selected from the five Head Start centers located in a mid-sized city on Florida’s east coast. Sixty-two children who were enrolled in the two selected centers at the time of the study were randomly assigned to groups, with 31 children assigned to each the experimental and the control groups (no parental objections to participation or attrition occurred). This sample of 62 children represented 22% of the total population of four- and five-year-old children in Head Start programs in this city.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

All children qualified for free or reduced-price lunch and came from families meeting the poverty index guidelines for the state of Florida, based on the most recent data available at that time (United States Department of Health and Human Services, 2004). English was the second language of 32 of the children.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Headsprout Early Reading

Specify which condition is the control condition:

Millie’s Math House

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

N/A

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Children from two Head Start centers participated in the study. Both centers were randomly selected from the five Head Start centers located in a mid-sized city on Florida’s east coast. Sixty-two children who were enrolled in the two selected centers at the time of the study were randomly assigned to groups, with 31 children assigned to each the experimental and the control groups (no parental objections to participation or attrition occurred). This sample of 62 children represented 22% of the total population of four- and five-year-old children in Head Start programs in this city.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Teacher and Teacher Assistant Demographic Data: Teacher Gender: Female=5, Male=0 Teacher Assistant Gender: Female=5, Male=0 Teacher Ethnicity: African American= 5, Caucasian=0 Teacher Assistant Ethnicity: African American=4, Caucasian= 1 Teacher Classroom Experience: 0-2 years=0, 3-5 years=1, 6 or more years=4 Teacher Assistant Classroom Experience: 0-2 years=1, 3-5 years=1, 6 or more years=3 Teacher Highest Degree Received: General education diploma=0, high school diploma=0, associate degree=4, bachelor’s degree=1 Teacher Assistant Highest Degree Received: General education diploma=2, high school diploma=3, associate degree=0, bachelor’s degree=0

Describe when and how fidelity of treatment information was obtained.

Teachers’ implementation was assessed using one procedural checklist for each the experimental and control programs. The checklists were developed by the first author after identifying necessary tasks for successful implementation of the programs, which was done through observation of the programs and discussion with teachers. During each observation period, the first author observed teachers while they implemented the control or experimental intervention. Each item from the applicable implementation integrity checklist was marked as being present (if the teacher implemented the item) or absent (if the teacher failed to implement the item). A teacher with experience in using both the experimental and the control interventions served as a second observer during ten randomly selected observation sessions.

What were the results on the fidelity-of-treatment implementation measure?

Implementation integrity for each observed session was calculated by applying the following formula to the implementation integrity checklists: (p=[p þ a] Â 100), where p represents the number of items present and a represents the number of items absent. Implementation integrity percentages for Headsprout Early Reading ranged from 60% to 90%, with a mean of 77%. Implementation integrity percentages for Millie’s Math House ranged from 60% to 100%, with a mean of 78%.

Was the fidelity measure also used in control classrooms?

Yes

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes: