Tutoring Buddy

Study: DuBois, Volpe, Burns, & Hoffman (Tentatively accepted)

DuBois, M., Volpe, R. J., Burns, M. K., Hoffman, J. A., (Tentatively accepted). Evaluation of a parent-administered computer-aided tutoring program targeting letter-sound knowledge in preschool-aged children. Journal of School Psychology.
Descriptive Information Usage Acquisition and Cost Program Specifications and Requirements Training

This evidence-based intervention aims to improve early literacy skills. 

Tutoring Buddy is intended for use in grades preK-1. The program is intended for use with any student at risk of academic failure. The academic area of focus is early literacy, including print knowledge/awareness, alphabet knowledge, and phonological awareness. 

Where to obtain:
Twin Lights Education, LLC. 

6 Oakland Ave., Rockport, MA 01966
Phone: 617-602-5626

Website: http://www.tutoringbuddyk12.com/  

Cost: Approximately $20 per student for unlimited use for one academic year.

Use of the system including the iOS app, web app, and online training materials and support is $399 per year per grade-level in the school. An entire school can use the program for $499 per year- the number of students and number of sessions is unlimited.

It is recommended Tutoring Buddy is used with individual students.

Tutoring Buddy takes on average 5 minutes per session with a recommended 3-4 sessions per week for results after 6 weeks.

The program includes a highly specified teacher’s manual.

Tutoring Buddy runs native on the iOS (iPad, iPhone, iPod Touch), but also runs as a mobile enabled web-app that can be run on any computer or tablet.

A minimum of 1-4 hours of training is required for instructors. Training can be accomplished via a web-based or in app training materials. It involves reading the procedures, and practicing the pronunciation of each letter sound. Correct pronunciation is modeled by the software.

Tutoring Buddy was designed to be completed by anyone. The program does not assume that the instructor has expertise in any given area.

Updated training manuals and materials are available, that have been used with both graduate research assistants and parents.

Practitioners receive an email address and calls are answered within 24 hours by one of the two developers. 

 

Participants: Unconvincing Evidence

Risk Status: Two of the three students were receiving early intervention due to some kind of disability.

Demographics:

 

Age/ Grade

Gender

Race-ethnicity

Socioeconomic status

Disability Status

ELL status

Other Relevant Descriptive Characteristics

Case 1: Jane

4 years, 2 months

F

Caucasian

Jane’s mother obtained a Master’s of Fine Arts and was a homemaker. Jane’s father obtained a doctoral degree and was employed at a financial firm.

Jane was identified at age 2 as having a communication disability related to expressive language and social pragmatic deficits.

No

No other details provided (DuBois, Volpe, Burns, & Hoffman, Tentatively accepted).

Case 2: Andy

4 years, 9 months

M

Hispanic

Andy’s mother obtained a Bachelor of Arts degree and was a homemaker. Andy’s father obtained a Masters of Business Administration and was employed in information technology.

None

No

Although Andy had not previously been identified as having a disability, his mother was concerned with his literacy development and his academic readiness to begin kindergarten. His mother and father learned English as a second language and spoke both Spanish and English at home.

Case 3: Mary

4 years, 10 months

F

Hispanic

Mary’s mother obtained a Bachelor of Arts degree in social work and was a homemaker. Mary’s father obtained a Bachelors of Arts degree in business and owned a construction company.

Receiving services for a communication disability.

Yes

Mary was identified at age 3 as having a communication disability and received speech and language therapy services to address articulation deficits. These services were discontinued six months prior to treatment. Mary attended a general education preschool for 15 hours a week and had been participating in preschool for 8 months prior to entering the study.

Training of Instructors: Prior to implementing the intervention, caregivers were required to meet with the researcher for a one-hour training session in their home. This session occurred immediately prior to the first intervention session. To begin, the researcher trained caregivers how to pronounce the 24 targeted letter sounds. For each letter sound, the research articulated the sound, provided an example of a word that began with that sound, and asked the caregiver to repeat the sound. Caregivers were then presented with each letter in random order and were asked to orally articulate its sound. Caregivers were required to achieve 100% accuracy on this assessment before administering the intervention. Next, the researcher downloaded the Tutoring Buddy application onto the caregiver’s iPad. Using role-play, the researcher then demonstrated how to implement the intervention, with the caregiver acting as the child.  The caregiver then practiced implementing the intervention with the researcher serving as the child. Role-play continued until the caregiver demonstrated that she was able to implement each component of the intervention accurately (the researcher used the 22-item procedural checklist to measure accuracy).

Design: Convincing Evidence

Does the study include three data points or sufficient number to document a stable performance within that phase? Yes

Is there opportunity for at least three demonstrations of experimental control? Yes

If the study is an alternating treatment design, are there five repetitions of the alternating sequence? NA

If the study is a multiple baseline, is it concurrent? No

Fidelity of Implementation: Partially Convincing Evidence

Description of when and how fidelity of treatment information was obtained: Measures of treatment fidelity were obtained via direct observation and through the tutoring program. Specifically, the researcher observed caregivers engaging in the intervention once or twice a week and completed the 22-item procedural checklist. These data were used to calculate session integrity (or the percentage of steps that were administered correctly during each intervention session). Treatment integrity data were collected for 44% of intervention sessions for Jane and Andy and 53% of intervention sessions for Mary, which exceeds the 20% of intervention sessions criterion recommended for monitoring treatment integrity (Perepletchikova & Kazdin, 2005). In addition to treatment integrity, the Tutoring Buddy program was used to measure weekly dosage (or the number of intervention sessions delivered each week) and total dosage (or the total number of intervention sessions that were delivered during the intervention), as the program recorded the date of each intervention session.

Results on the fidelity of treatment implementation measure: Treatment integrity scores across observed intervention sessions was 88% for Jane (range: 68% - 100%), 97% for Andy (range: 86% - 100%), and 91% for Mary (range: 82% - 100%). One observed intervention session for Jane fell below 80%. In regard to dosage, Jane’s mother and Andy’s mother implemented the intervention three times each week, for a total dosage of 18 intervention sessions. Therefore, they were able to perfectly adhere to intervention schedule outlined prior to treatment initiation. Mary’s mother implemented the intervention 15 times during the 6 weeks. Therefore, she implemented 15 of the desired 18 intervention sessions.

Measures Targeted: Convincing Evidence

Measures Broader: Convincing Evidence

Targeted  Measure Reliability Statistics Relevance to Program Focus Exposure to Related Content Among Control Group

Letter Sound Expression (LSE; Twin Lights Education, 2009)     

Test retest based on ICC over 4 consecutive days (.97). Criterion-related validity coefficients with Letter Sound Fluency and Nonsense Word Fluency were .80 and .74 respectively (DuBois & Volpe, 2016).     

Each Tutoring Buddy session begins with the LSE assessment. Data from the assessment is used to identify the known and unknown facts to be targeted by Tutoring Buddy. It also serves as a progress monitoring measure.

N/A

Letter Sound Fluency (NCS, Pearson, 2012) Test retest = .85 (2 week latency); Alternate form = .82); Predictive validity (3 Year latency) = .61 with CBM-R; between .33 and .52 with comprehensive state tests. Tutoring Buddy uses incremental rehearsal, but presents known and unknown letters in a string. It impacts both acquisition and fluency.  
Broader Measure Reliability Statistics Relevance to Program Focus Exposure to Related Content Among Control Group

Nonsense Word Fluency (Dynamic Measurement Group, 2008)

Alternate form reliability between .71 and .78; Predictive validity (3-year latency) with comprehensive state assessment measures (rs between.42 and .61); and with CBM-R (also 3 Years later, rs between .68 and .72; NCS Pearson, 2012)

NWF is a phonics measure assessing student’s ability to sound out nonsense words (e.g., NAB). It relates to Tutoring Buddy in that Tutoring Buddy targets letter sound correspondence.

N/A

 

Number of Outcome Measures: 3 Prereading

Mean ES - Targeted: N/A

Mean ES - Broader: N/A

Effect Size:

Visual Analysis (Single Subject Design): Convincing Evidence

Description of the method of analyses used to determine whether the intervention condition improved relative to baseline phase (e.g. visual analysis, computation of change score, mean difference): We used visual inspection and calculated PAND and PND.

Results in terms of within and between phase patterns: Baseline data were low and stable for all three students. However, because the IR procedure requires knowledge of at least one known letter sound, Jane was taught one letter sound during the baseline phase. Baseline intercepts for the three children ranged from 1 to 6 for LSK and 0 to 4 for LSF and NWF. The average daily gains for LSK during baseline, as indicated by slope values, were .13, .00, and .00 respectively for Jane, Andy, and Mary. For LSF, baseline slopes were .00 for all children. For NWF, baseline slopes were .00, -.07, and .00 for Jane, Andy, and Mary respectively.

As demonstrated in Figure 1, positive and steady growth in LSK was observed for all three children immediately following introduction of the Tutoring Buddy intervention. Notable changes in level and trend were demonstrated for each child during intervention. The slopes for Jane, Andy, and Mary improved to .39, .41, and .23, respectively. In regard to average weekly growth, Jane, Andy, and Mary learned 2.67 (range: 1 - 5), 3.00 (range: 0 - 5), and 1.83 (range: 1 – 3) letters sound each week, respectively. Importantly, only letter sounds that were targeted during intervention, or were known at baseline, were articulated correctly during the final LSK measurement for all three children. This finding provides support that the observed gains in LSK were attributable to the intervention.

For Andy and Mary, consistent positive growth was observed for LSF and NWF immediately following implementation of the Tutoring Buddy intervention. Notable changes in level and trend were also demonstrated. For LSF, the slopes for Andy and Mary improved to .30 and .27, respectively. For NWF, the slopes for Andy and Mary improved to .33 and .21, respectively. During the NWF task, neither Andy nor Mary engaged in recoding or blending (this was expected given that there was no instruction in blending during the intervention). In regard to average weekly growth, Andy’s LSF and NWF scores increased by 2.67 (range 1 - 4) and 2.00 (range: 0 – 5) units each week, respectively. On average, Mary’s LSF and NWF scores increased by 1.67 (range 0 - 3) and 1.50 (range: 0 – 2) units each week, respectively.

Jane’s trends in LSF and NWF were more variable, however. Jane made no gains in LSF and NWF following the first week of treatment. Although steady gains in these dependent measures were observed in weeks 2 and 3 of treatment, declines in performance were observed during week 4 (i.e., NWF returned to baseline, LSF was 5 correct letter sounds per minute lower in week 4 than in week 3). Following this decline in performance, Jane made positive and steady gains in LSF and NWF during weeks 5 and 6. Despite this variability, Jane’s slopes for LSF and NWF improved during intervention to .14 and .07, respectively. During the NWF task, Jane did not engage in recoding or blending. A summary of means for LSK, LSF, and NWF during the baseline and intervention phases is presented in Table 1.

In addition to visual analyses and slope parameters, effect sizes were derived from Percentage of All Non-Overlapping Data (PAND; Parker, et al., 2007). PAND statistics were then translated into a Pearson’s Phi Coefficient to determine effect sizes. For Andy and Mary, PAND was 100% for LSK, LSF, and NWF. The corresponding effect sizes for Andy and Mary were 1.00 for LSK, LSF, and NWF. For Jane, PAND was 100% for LSK and LSF, and 89% for NWF. The resultant effect sizes were 1.00, 1.00, and 0.83 for LSK, LSF, and NWF, respectively. Across the three children, PAND was 100%, 100%, and 96% for LSK, LSF, and NWF, respectively. The resultant effect sizes were 1.00, 1.00, and 0.91 for LSK, LSF, and NWF, respectively.

Disaggregated Data for Demographic Subgroups: No

Disaggregated Data for <20th Percentile: No

Administration Group Size: Individual

Duration of Intervention: 5 minutes, 3-4 times a week, 6+ weeks

Minimum Interventionist Requirements: No minimum qualification for instructor, 1-4 hours of training required

Reviewed by WWC or E-ESSA: E-ESSA

What Works Clearinghouse Review

This program was not reivewed by What Works Clearinghouse.

 

Evidence for ESSA

No studies considered met Evidence for ESSA's inclusion requirements.

 

Other Research: Potentially Eligible for NCII Review: 0 studies