Stepping Stones to Literacy
Study: Nelson et al. (2010)

Describe how students were selected to participate in the study:

Eight Head Start classrooms from rural Midwest communities agreed to participate in the study, All children whose teachers believed the interventions were appropriate were recruited to participate. The consent procedures allowed teachers to exclude those children they believed would not benefit from the interventions. All children were recruited to participate with the exception of those experiencing significant learning and/or behavioral difficulties (n=4). Parental informed consent was obtained in all cases.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Participation in Head Start classrooms. Children in the study were participants in Head Start. Head Start provides services for students from disadvantaged backgrounds (low income or eligible for public assistance) who are at risk for being unprepared for school. Additional information from the lead author, Nelson, indicates the TOPEL Phonological Awareness subtest is directly linked to the intervention. The treatment group (entire sample) mean pre-test is equal to the 16th percentile (standard score = 85).

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The treatment (PA-focused intervention) received Stepping Stones to Literacy.

Specify which condition is the control condition:

Children in the control group (vocabulary-focused intervention) received instruction in a modified Interactive Book Reading condition. The program is based in part on Interactive Book Reading in that storybooks linked to themes and the same three-step instructional sequence is used. The difference is that pictures and guiding prompts were used in the control condition to introduce, engage and motivate, encourage explanations and support independent use of word meanings (instead of a Prop Box as in Interactive Book Reading).

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Parent consent was then obtained prior to pretest. Within classrooms, participating children were randomly assigned to small groups (there were 2 to 4 small groups per classroom), and then small groups comprising 2 to 6 children each (with an average of 3 preschoolers per small group) were randomly assigned to receive supplemental small-group instruction in either the treatment or the control program. After attrition of 11 children (5 treatment and 6 controls due to moving out of the program), there were 41 treatment children (across 13 small groups) and 47 controls (across 14 small groups).

What was the unit of assignment?

Other

If other, please specify:

Students and small groups

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:
Students and small groups

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Stepping Stones instruction was provided by trained paraprofessional level tutors. All tutors were recruited from respective communities served by the Head Start centers. All tutors had completed high school and had no prior teaching experience. Stepping Stones to Literacy is user friendly with an instructional layout that is predictable and clear for ease of implementation by a wide array of educators, including classroom teachers, resource teachers, and literacy specialists. Further, such instructional layouts have allowed paraeducators to use it effectively, which will increase the instructional potential of the increasing number of paraeducators in U.S. schools (National Center for Education Statistics, 2009) to positively affect early literacy achievement. The training and ongoing support of the instructions included the following steps. First, tutors were provided the theory and rationale for Stepping Stones. Second, research staff modeled and practiced the instructional activities with tutors. Third, tutors then practiced each of the instructional activities in simulated practice conditions. Tutors were provided corrective feedback while they were practicing the instructional activities. Fourth, tutors were observed delivering the first two lessons during the project and provided corrective feedback to ensure treatment fidelity. Tutors were then observed by project staff three additional times to assess treatment fidelity; If necessary, following these observations project staff provided corrective feedback.

Describe when and how fidelity of treatment information was obtained.

Each tutor in both treatment and control conditions was observed during an intervention session on equidistant occasions (once per week). To measure fidelity, tutors were rated on a 5 point behavior frequency scale on each relevant instructional practice (8 practices for treatment and 9 practices for control). Across each instructional practice category, means were computed and then the means were averaged across the three observations to obtain a fidelity score which was applied to each small group.

What were the results on the fidelity-of-treatment implementation measure?

The effects of treatment on posttest measures were examined when statistically controlling for treatment fidelity. Multilevel modeling was used to analyze child outcomes. To determine whether treatment had unique impacts on posttests after controlling for pretest, small-group treatment fidelity, and classroom and home literacy environments, the authors added the respective grand-mean centered covariates to their three-level direct effects models. The unique effect of treatment fidelity on posttest measures did not reach significance. Thus, the reviewers did not believe it was necessary to include raw data given the outcomes. The obtained fidelity means and standard deviations for the experimental and control conditions were 30.39 (SD=1.23) (maximum score =32) and 34.56 (SD=1.82) (maximum score = 36).

Was the fidelity measure also used in control classrooms?

Yes, The obtained fidelity mean and standard deviation for the control condition was 34.56 (sd=1.82) (maximum score = 36).

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

The following is from the Analytic Methods section of the article on pages 14-15: Because of the complex nature of the child outcome data, we used multilevel modeling to analyze intervention condition differences. Because children received intervention in small groups within classrooms, two inherent nesting structures were present: Children within a given small group are likely to have more similar instructional experiences than children in other small groups, and small groups were more likely to be more similar to one another if they were drawn from the same classroom compared to other classrooms. Consequently, scores from children and small groups were not treated as independent. Specifically, we used two models. First, we tested for direct effects using a simple three-level model in which children’s scores (Level 1) are nested in small groups (Level 2), which are in turn nested in classrooms (Level 3). In our second set of models (to test for unique treatment effects), we simultaneously added four covariates, all grand-mean centered, including respective child pretest, child’s home literacy environment (mean across 12 items on home literacy survey), small-group treatment fidelity, and classroom literacy environment (teacher report of total daily minutes afforded to literacy instruction).