Daily Report Card

Study: Fabiano, Vujnovic, Pelham, Waschbusch, Massetti, Yu, et. al (2010)

Study Type: Group-Design

Participants: Convincing Evidence

Sample size: 62 students (33 program, 29 control)

Risk Status: Children were diagnosed using evidence-based assessment procedures for ADHD (Pelham, Fabiano, & Massetti, 2005). These included parent and teacher Disruptive Behavior Disorder (DBD), ADHD symptom rating scales (Pelham, Gnagy, Greenslade, & Milich, 1992), and impairment rating scales (Fabiano et al., 2006), plus the semi-structured, DBD diagnostic interview completed with parents (Hartung, McCarthy, Milich, & Martin, 2005; Massetti et al., 2003). Symptoms rated as “pretty much” or “very much” on the DBD rating scale were counted as being present, and the DBD parent interview was used to obtain contextual information on the presence of symptoms, age of onset, and information on pervasiveness and chronicity. Two doctoral level clinicians completed independent file reviews to confirm agreement on diagnoses, and disagreements were resolved by a third doctoral level reviewer.

Demographics:

 

Program

Control

Number

Percentage

Number

Percentage

Grade level

  Kindergarten

       

  Grade 1

       

  Grade 2

 

 

 

 

  Grade 3

 

 

 

 

  Grade 4

 

 

 

 

  Grade 5

 

 

 

 

  Grade 6

 

 

 

 

  Grade 7

 

 

 

 

  Grade 8

 

 

 

 

  Grade 9

 

 

 

 

  Grade 10

 

 

 

 

  Grade 11

 

 

 

 

  Grade 12

 

 

 

 

Mean Age

 

 

 

 

Race-ethnicity

  African-American

 

15%

 

7%

  American Indian

 

0%

 

0%

  Asian/Pacific Islander

 

0%

 

0%

  Hispanic

 

3%

 

3%

  White

 

82%

 

73%

  Other

 

0%

 

3%

Socioeconomic status

  Subsidized lunch

 

 

 

 

  No subsidized lunch

 

 

 

 

Disability status

  Speech-language impairments

 

15%

 

13%

  Learning disabilities

 

18%

 

20%

  Emotional disturbance

 

6%

 

17%

  Intellectual disability

 

0%

 

0%

  Other

 

61%

 

50%

  Not identified with a disability

 

0%

 

0%

ELL status

  English language learner

 

 

 

 

  Not English language learner

 

 

 

 

Gender

  Female

 

91%

 

80%

  Male

 

 

 

 

Training of Instructors: School psychology graduate student consultants were assigned to work with a family and teacher for the entire school year. They were supervised by a Ph.D.-level psychologist.

Design: Partially Convincing Evidence

Did the study use random assignment?: Yes

If not, was it a tenable quasi-experiment?: Not applicable

If the study used random assignment, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures used as covariates or on pretest measures also used as outcomes? No

If not, at pretreatment, were the program and control groups not statistically significantly different and had a mean standardized difference that fell within 0.25 SD on measures central to the study (i.e., pretest measures also used as outcomes), and outcomes were analyzed to adjust for pretreatment differences? Not applicable

Were the program and control groups demographically comparable?: Yes

Was there differential attrition for the program and the control groups?: Yes

Did the unit of analysis match the unit for random assignment (for randomized studies) or the assignment strategy (for quasi-experiments)?: Yes

Implemented with Fidelity: Convincing Evidence

Description of when and how fidelity of treatment information was obtained:

Integrity of implementation of DRC procedures were collected throughout the duration of the program in both groups.

Results on the fidelity of treatment implementation measure:

Analyses to examine the integrity with which the intervention was implemented revealed that, on average, teachers completed 73% of DRCs compared to 77% of the ITBEs, and a chi-square test indicated this difference was not significant (p > .05). Interestingly this rate of adherence was nearly identical to another recent study that utilized DRCs across the school year (Owens et al., 2008). The range of completed DRCs was from 0%-98% with a median of 79%. ITBE completion ranged from 16%-99% with a median of 78%. Furthermore, across the entire school year, parents of participants in the DRC group returned 64% of the DRC’s with a signature; the range of signed DRCs was from 0%-100% with a median of 90%. Parents indicated that a reward was given for 56% of the returned DRC’s that should have earned one. Reward rates ranged from 0%-98% with a median of 68%. To explore the relationship between integrity and outcome, DRC completion rates were not significantly correlated with observations of classroom behavior at endpoint (p > .05), but the percent of DRCs rewarded was significantly correlated (r= -.53, p=.002), suggesting contingency management (i.e., contingent rewards based on school behavior) may be a key aspect of the approach used in this study.

An examination of the integrity with which the behavioral consultation group implemented the DRC intervention revealed that teachers of students in the intervention condition participated in 94% of monthly behavior consultation meetings, completed 93% of meeting checklist items, and met with the behavioral consultant in face-to-face meetings for approximately 20 minutes per month (SD=7.49). These meetings were supplemented by ongoing monitoring of DRC/ITBE data throughout the month and modification of the DRC as needed (e.g., multiple days in a row of missing a target, teacher comments reveal emerging behaviors that warrant targeting). The number of behaviors targeted for children across the study year ranged from 3-9. Children in the DRC group had an average of 5.53 (SD=1.22) behaviors targeted during the school year compared to 4.64 (SD=1.03) behaviors targeted on ITBEs, and this difference was significant, t (58) =3.03, p = .004. Table 4 lists the academic and social behaviors targeted on the DRCs/ITBEs and the percentage of children with each goal (homework was targeted for all children in the study as part of the consultation procedures).

Measures Targeted: Convincing Evidence

Targeted Measure

Reliability statistics

Relevance to program focus

Exposure to related support among control group

Observations of Disruptive Behavior

Inter-rated r = 0.94; Mean difference 0.53

Proximal Outcome

Business as usual

Academic Performance Rating – Academic Productivity

coefficient alpha = 0.94; test-retest reliability over two weeks = 0.93

Proximal Outcome

Business as usual

Academic Performance Rating – Academic Success

coefficient alpha = 0.95; test-retest reliability over two weeks = 0.91

Proximal Outcome

Business as usual

IEP Goal Attainment Global Improvement Rating

The global improvement rating has inter-rater reliability (Tolin et al., 2007; r = 0.81)

Proximal Outcome

Business as usual

 

Broader Measure

Reliability statistics

Relevance to program focus

Exposure to related support among control group

Impairment Rating Scale

Test-retest reliability ranged from 0.60-0.89 over a period of six months, and 0.54 to 0.76 over one year

Distal Outcome

Business as usual

DBD Rating scale ADHD and ODD/CD items

coefficient alpha = 0.91-0.96

Distal Outcome

Business as usual

Student Teacher Relationship Scale

coefficient alpha = 0.89; test-retest reliability over four weeks = 0.89

Distal Outcome

Business as usual

 

Mean ES Targeted Outcomes: Data Unavailable

Mean ES Administrative Outcomes: N/A

Effect Size:

Targeted Measures

Measure

Effect Size

Observations of Disruptive Behavior

Academic Performance Rating--Academic Productivity

Academic Performance Rating--Academic Success

IEP Goal Attainment Global Improvement Rating

 

Broader Measures

Measure

Effect Size

Impairment Rating Scale

DBD Rating Scale ADHD

0.27 U

DBD ODD/CD items

0.49 U

Student Teacher Relationship Scale

 

Key

*        p ≤ .05

**      p ≤ .01

***    p ≤ .001

–      Developer was unable to provide necessary data for NCII to calculate effect sizes

u      Effect size is based on unadjusted means

†      Effect size based on unadjusted means not reported due to lack of pretest group equivalency, and effect size based on adjusted means is not available

 

Visual Analysis (Single-Subject Designs): N/A

Disaggregated Outcome Data Available for Demographic Subgroups: No

Target Behavior(s): Externalizing

Delivery: Individual

Fidelity of Implementation Check List Available: No

Minimum Interventionist Requirements: Less than 1 hour of training required

Intervention Reviewed by What Works Clearinghouse: Yes – Study

What Works Clearinghouse Review

WWC only reviewed the report “Enhancing the effectiveness of special education programming for children with attention deficit hyperactivity disorder using a daily report card.” The findings from this review do not reflect the full body of research evidence on Daily Report Card.

WWC Rating: The research described in this report meets WWC evidence standards without reservations.

Full Report

Other Research: Potentially Eligible for NCII Review: 7 studies

Atkins, M.S., Pelham, W.E., White, .K.J. (1989). Hyperactivity and attention deficit disorders. In M. Hersen (Ed.), Psychological Aspects of Developments and Physical Disabilities: A Casebook. (pp. 137-156). California: Sage Publications.

Kelley, M.L. & McCain, A.P. (1995). Promoting academic performance in inattentive children. Behavior Modification, 19, 357-375.

McCain, A.P. & Kelley, M.L. (1993). Managing the behavior of an ADHD preschooler: The efficacy of a school-home note intervention. Child and Family Behavior Therapy, 15, 33-44.

McCain, A.P. & Kelley, M.L. (1994). Improving the classroom performance in underachieving preadolescents: The additive effects of response cost to a school-home note system. Child and Family Behavior Therapy, 16, 27-41.

Murray, D.W., Raniner, D., Schulte, A., & Newitt, K. (2008). Feasibility and integrity of a parent-teacher consultation intervention for students. Child and Youth Care Forum, 37, 111-126.

O’Leary, K.D., Pelham, W.E., Rosenbaum, A., & Price, G.H. (1976). Behavioral treatment of hyperkinetic children. Clinical Pediatrics, 15, 510-515.

Palcic, J.L., Jurbergs, N., & Kelley, M.L. (2009). Comparison of teacher and parent delivered consequences: Improving behavior in low-income children with ADHD. Child and Family Behavior Therapy, 31, 117-133.