Sound Partners (1-3)
Study: Vadasy et al. (2005)

Describe how students were selected to participate in the study:

Participants were recruited from 12 urban, demographically similar elementary schools in a large northwest area district. Six sites were assigned as treatment sites, five sites were assigned to the control/comparison condition, and one site included both control and treatment students. During the first month of first grade 22 teachers referred students they identified to be at risk for reading difficulties for screening. Students had to have parental consent and not be repeating the first grade. (Page 367 of the article).

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Students met the screening criteria of scoring at or below a standard score of 90 (25th percentile) on the Reading subtest of the Wide Range Achievement Test – Revised (WRAT-R). (page 367 of the article).

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

The submitted program is Sound Partners.

Specify which condition is the control condition:

The control condition included the schools’ regular reading instruction.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Upon confirmation of study eligibility, students at treatment sites were assigned to tutors (and thus to treatments) based on classroom schedules and tutoring availability.

What was the unit of assignment?

Teachers

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

Nineteen paraprofessional tutors, hired and paid by their respective schools, were randomly assigned to treatments by research staff prior to training; however, four tutors’ treatment assignments were switched prior to training due to conflicts between the training schedule and the tutors’ schedules. There were nine tutors assigned to the Reading Practice treatment and 10 tutors assigned to the Word Study treatment. Within each treatment, more than half of the tutors (6 each) had at least 1 year of Sound Partners tutoring experience and participated in previous research. Experienced tutors received approximately two hours of initial training in treatment instruction and new tutors received approximately four hours of initial training. After training, all tutors were provided with weekly on-site coaching and modeling throughout the year.

Describe when and how fidelity of treatment information was obtained.

Treatment integrity was assessed through (a) student completion of lessons and (b) tutor fidelity to instructional components. On-site tutor observations were conducted weekly by two researchers using a 41-criterion checklist of critical tutor and student behaviors required for full implementation of the 11 lesson components. Activities were rated on a dichotomous scale and the highest percentage possible was 100%. Tutors were also rated on a 16-item checklist on their use of tutoring time and instructional delivery. For all criteria, researchers scored only items they could observe during the on-site visit.

What were the results on the fidelity-of-treatment implementation measure?

Research staff conducted a total of 437 tutor observations over the course of the year, with a mean of 20 observations per tutor and a mean of 6 observations per student. For each treatment, researchers computed the mean percentage of criteria that the tutors met for lesson components, use of time and instructional delivery, and overall tutoring. The overall mean fidelity percentage for the Reading Practice (RP) treatment group was 95.3% (SD = 5.18%) and for the Word Study (WS) treatment group was 95.1% (SD = 7.23%). In a series of one-way ANOVAs, researchers found no difference between the treatment groups on any criteria (F (1, 19) = 0.006, p = 0.94). Interrater reliability was assessed by examining the results of paired tutor-student observations conducted within a 20-day time frame. The mean overall interrater reliability for 16 sampled tutor-student pairs was 0.98 (SD = 0.06) with an average of 6.0 days (SD = 5.09) between observations.

Was the fidelity measure also used in control classrooms?

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

All analyses used standard scores, unless otherwise noted. All posttest effect sizes (Cohen’s d) were calculated by dividing the difference between the regressed adjusted means (adjusted for pretest covariate) by the square root of the mean square error. Researchers calculated intercorrelations among posttest measures between treatment groups and between treatment and control groups (see page 373-374 of the article)