Sound Partners Kindergarten
Study: Vadasy & Sanders (2008)

Describe how students were selected to participate in the study:

In December of the school year, full-day kindergarten teachers in 13 urban public elementary schools were asked to refer students who they felt would benefit from intensive additional reading instruction. Participating schools had student populations averaging 61% minority, 50% free-reduced lunch, 12% English Language Learner, and 16% special education. Parent consents for student participation were obtained by the schools prior to screening and testing. Of the referred students with parent consent, 99 met eligibility screening criteria that included: (a) producing <15 letter sounds per min when presented with a sheet of randomly ordered upper case letters (Fuchs et al., 2001); and (b) scoring in the deficit range (<15 letter names per min) on the Letter Name Fluency subtest of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002). (Criteria for eligibility in our earlier study included students < 27 letter names per min). Students receiving special education, Title I, or English Language Learner services were not excluded from the study. Eligible students were quasi-randomly assigned within classrooms to one of three conditions using an MS Excel random number generator in tandem with constraints designed to ensure each classroom was represented in the control group (at study onset) as well as to ensure a larger dyad-tutored total group size relative to the individual-tutored group size (in consideration of power and dependence among dyads). These specific a priori constraints were that: 1. students from classrooms with only one eligible student were removed from study participation; 2. students from classrooms with only two eligible students were randomly assigned to either no tutoring or individual tutoring; 3. students from classrooms with three eligible students were assigned to either no tutoring or dyad tutoring; 4. students from classrooms with four eligible students were assigned to either no tutoring, individual tutoring, or dyad tutoring, with two of the four students assigned to dyad tutoring; and 5. students from classrooms with larger numbers of eligible students were assigned using the four-student procedure in conjunction with randomly alternating extra students into one of the three conditions. After the removal of one student due to our first assignment constraint and attrition of 12 students who moved (two from the control condition, six from the individual tutoring condition, and four from the dyad tutoring condition), the final sample comprised 32 non-tutored students, 22 individually tutored students, and 32 students tutored in dyads.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Of the referred students with parent consent, 99 met eligibility screening criteria that included: (a) producing <15 letter sounds per min when presented with a sheet of randomly ordered upper case letters (Fuchs et al., 2001); and (b) scoring in the deficit range (<15 letter names per min) on the Letter Name Fluency subtest of the Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002). Students in the sample were at-risk because the grand mean of the sample was below 25th percentile on the CTOPP Phonological Awareness measure. (The grand mean across the sample is 87.24, which is < 20th percentile).

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Both treatment groups in this study received Sound Partners Kindergarten instruction.

Specify which condition is the control condition:

Students in the no-tutoring control condition received regular classroom reading instruction only “business as usual.”

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Systematic

Please describe the assignment method or the process for defining treatment/comparison groups.

Eligible students were quasi-randomly assigned within classrooms to one of three conditions using an MS Excel random number generator in tandem with constraints designed to ensure each classroom was represented in the control group (at study onset) as well as to ensure a larger dyad-tutored total group size relative to the individual-tutored group size (in consideration of power and dependence among dyads). These specific a priori constraints were that: students from classrooms with only one eligible student were removed from study participation; students from classrooms with only two eligible students were randomly assigned to either no tutoring or individual tutoring; students from classrooms with three eligible students were assigned to either no tutoring or dyad tutoring; students from classrooms with four eligible students were assigned to either no tutoring, individual tutoring, or dyad tutoring, with two of the four students assigned to dyad tutoring; and students from classrooms with larger numbers of eligible students were assigned using the four-student procedure in conjunction with randomly alternating extra students into one of the three conditions.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

All paraeducator tutors (paraeducators) were recruited from their school communities based on their interest in working with children, prior tutoring and school volunteer experience, and scheduling flexibility. The 21 participating paraeducators were mostly non-minority (86%) and female (95%), and varied in their age, educational levels, general tutoring experience, and experience working with kindergartners. Tutors averaged M = 46.81 years old (SD = 13.31), and ranged in educational level from high school to master’s degree, with a modal level of some college or vocational education (43%). (The average education level of paraeducators in this study is similar to that recommended in new NCLB requirements.) Prior to the study, the range of paraeducator tutoring experience ranged from 0 (38%) to 10 or more years (14%), with an average of 3.33 years (SD = 4.12). Eight paraeducators had some previous experience working with kindergarteners, although most had no experience with this age group, M = 0.81 years (SD = 1.29, Range = 0 to 4 years). All paraeducators were hired as district employees and paid by the schools with funds provided by the research grant.

Describe when and how fidelity of treatment information was obtained.

Seven research staff members were hired and trained to conduct on-site fidelity observations of paraeducators with their assigned students. Fidelity observations involved a 5-point rating scale ranging from 1 (Never Implements Correctly) to 5 (Always Implements Correctly) for each of the instructional components. After training but prior to field observations, researchers viewed seven videotaped tutoring sessions of paraeducators implementing instruction with single students or pairs of students; students in the videotaped sessions were selected to represent a range of reading skills we expected to see in the field, with ages ranging from 4- to 7-years-old. To determine interrater reliability, we calculated the internal consistency of the observers’ mean implementation ratings for the seven videotaped sessions (using observers’ ratings as items and each videotape as subjects): Cronbach’s alpha was 0.81.

What were the results on the fidelity-of-treatment implementation measure?

After establishing reliability, researchers conducted a total of 217 observations for the 21 paraeducators over the course of the intervention, averaging 5.71 observations per student or student pair (126 observations for the 22 individually tutored students and 91 for the 16 dyads), with all students or pairs observed at least twice. Average fidelity ratings were M = 4.58 (SD = 0.28) for the individually tutored group and M = 4.70 (SD = 0.24) for dyads; there was no significant difference between individually tutored students and dyads on their mean fidelity ratings, t(36) = -0.15, p > 0.05.

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

We used multilevel modeling to account for classroom and dyad effects in all posttest analyses. A full description of our analysis approach is on p. 943-944 of the publication.