Daily Report Card
Study: Fabiano et al. (2010)

Describe how students were selected to participate in the study:

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Children were diagnosed using evidence-based assessment procedures for ADHD (Pelham, Fabiano, & Massetti, 2005). These included parent and teacher Disruptive Behavior Disorder (DBD) ADHD symptom rating scales (Pelham, Gnagy, Greenslade, & Milich, 1992) and impairment rating scales (Fabiano et al., 2006), plus the semi-structured, DBD diagnostic interview completed with parents (Hartung, McCarthy, Milich, & Martin, 2005; Massetti et al., 2003). Symptoms rated as “pretty much” or “very much” on the DBD rating scale were counted as being present, and the DBD parent interview was used to obtain contextual information on the presence of symptoms, age of onset, and information on pervasiveness and chronicity. Two doctoral level clinicians completed independent file reviews to confirm agreement on diagnoses, and disagreements were resolved by a third doctoral level reviewer.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Specify which condition is the submitted intervention:

Consultants conducted an initial meeting with each teacher of children in the DRC group during October of the school year. During this meeting, consultants and teachers used the IEP and any other related information to construct a DRC. Between the first and second meeting, the teacher was asked to implement the intervention. At the second consultant visit, target behaviors were refined, and using the data collected by the teacher, criteria for each target behavior was modified (e.g., a child who averaged 10 verbally intrusive behaviors per class would have a target behavior changed to “Has eight or fewer verbally intrusive behaviors”). The third consultant visit was conducted to fine-tune and trouble-shoot the DRC and inform the teacher of the home rewards established by the parents. The DRC included a direct accounting for IEP goals as well as other behavior problems common to a child with ADHD, and it is necessarily idiosyncratic – unique to each child. A standard list of common DRC goals has been created and was used to facilitate this target behavior selection (see http://ccf.buffalo.edu/pdf/school_daily_report_card.pdf for a downloadable handout that describes how to create, modify, and reward a DRC). The consultant could also add targets beyond those explicitly listed in the IEP that were appropriate for the current classroom situation, and this was typical, especially for social/behavioral targets (Fabiano, Vujnovic, et al., 2009). The DRC was evaluated and completed by the teacher daily, and feedback was provided to the child throughout the day on progress made toward DRC goals. The teacher was asked to implement the other procedures outlined in the IEP (i.e., academic interventions) as planned. At the end of each day, the teacher sent the DRC home with the child so that the parent received feedback on a daily basis regarding the child’s behavior at school. Parents attended three individual parent training meetings with the consultant conducted in parallel with the teacher meetings (generally held in the school library or cafeteria) to introduce them to the DRC. During these meetings, they established home-based rewards contingent on the child’s DRC performance. For example, a parent might remove computer access for a child where it was previously provided non-contingently, and only provide 10 minutes for each “yes” reported for the day. Parents were encouraged to develop a menu of rewards and to place the rewards in a hierarchy (i.e., the longest duration of computer time was provided for obtaining 90-100% of DRC goals, whereas a shorter duration was available for obtaining 70-80% of DRC goals). In addition to the home-based contingency management based on school feedback, which makes the child accountable at home for school-based behavior, the DRC served as a mechanism of daily communication between the parent and teacher. The consultant and parent also collaboratively constructed a plan for nightly homework and “returning completed homework” was targeted on all DRCs/ITBEs (Power, Karustis, & Habboushe, 2001). After the three initial meetings with the child’s teacher, consultants met monthly with the teacher to provide feedback on the child’s behavior during the month using a graphical representation of DRC performance. This information was used for data-driven decision making, and DRC targets were adjusted throughout the school year.

Specify which condition is the control condition:

Consultants conducted an initial meeting with each teacher of children in the BAU group. During this meeting, consultants and teachers used the IEP and any other related information to construct an Individualized Target Behavior Evaluation (ITBE; Pelham, Fabiano, & Massetti, 2005). Follow-up meetings were conducted in the same manner as described for the DRC group, above. Teachers in the BAU group were instructed to work with the child the same way they would with any other child who had an IEP. Teachers and parents were contacted monthly in the BAU condition and asked general questions about the child’s functioning. The ITBE was completed each day by the teacher, and it was adjusted (i.e., behavioral criteria modified; targets modified) based on parent or teacher report in these phone calls or a review of monthly ITBEs. Teachers were mailed quarterly graphs of ITBE results. Thus, in the BAU condition, the ITBE was constructed in the same manner as the DRC, and it was completed every day. However, it was used as an idiosyncratic rating scale, not an intervention. The BAU condition was used solely to monitor functioning – it did not provide communication between the parent and teacher; it was not used to provide students feedback on behavior; it did not result in any contingency management for the child’s behavior; and it was not formally used to make data-driven decisions related to monitoring/intervening with the child.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

N/A

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

During the spring and summer, children were recruited for participation in the study through mailings, radio advertisements, and school, doctor, and professional referrals. Following an intake to determine diagnosis and eligibility, eligible participants were randomly assigned to a business as usual control condition or an intervention condition.

What was the unit of assignment?

Students

If other, please specify:

Please describe the unit of assignment:

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:

Please describe the unit(s) used for primary data analysis:

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

School psychology graduate student consultants were assigned to work with a family and teacher for the entire school year. They were supervised by a Ph.D.-level psychologist

Describe when and how fidelity of treatment information was obtained.

Integrity of implementation of DRC procedures were collected throughout the duration of the program in both groups.

What were the results on the fidelity-of-treatment implementation measure?

Analyses to examine the integrity with which the intervention was implemented revealed that, on average, teachers completed 73% of DRCs compared to 77% of the ITBEs, and a chi-square test indicated this difference was not significant (p > .05). Interestingly this rate of adherence was nearly identical to another recent study that utilized DRCs across the school year (Owens et al., 2008). The range of completed DRCs was from 0%-98% with a median of 79%. ITBE completion ranged from 16%-99% with a median of 78%. Furthermore, across the entire school year, parents of participants in the DRC group returned 64% of the DRC’s with a signature; the range of signed DRCs was from 0%-100% with a median of 90%. Parents indicated that a reward was given for 56% of the returned DRC’s that should have earned one. Reward rates ranged from 0%-98% with a median of 68%. To explore the relationship between integrity and outcome, DRC completion rates were not significantly correlated with observations of classroom behavior at endpoint (p > .05), but the percent of DRCs rewarded was significantly correlated (r= -.53, p=.002), suggesting contingency management (i.e., contingent rewards based on school behavior) may be a key aspect of the approach used in this study. An examination of the integrity with which the behavioral consultation group implemented the DRC intervention revealed that teachers of students in the intervention condition participated in 94% of monthly behavior consultation meetings, completed 93% of meeting checklist items and met with the behavioral consultant in face-to-face meetings for approximately 20 minutes per month (SD=7.49). These meetings were supplemented by ongoing monitoring of DRC/ITBE data throughout the month and modification of the DRC as needed (e.g., multiple days in a row of missing a target, teacher comments reveal emerging behaviors that warrant targeting). The number of behaviors targeted for children across the study year ranged from 3-9. Children in the DRC group had an average of 5.53 (SD=1.22) behaviors targeted during the school year compared to 4.64 (SD=1.03) behaviors targeted on ITBEs, and this difference was significant, t (58) =3.03, p = .004. Table 4 lists the academic and social behaviors targeted on the DRCs/ITBEs and the percentage of children with each goal (homework was targeted for all children in the study as part of the consultation procedures).

Was the fidelity measure also used in control classrooms?

The integrity/fidelity measure was collected in both groups.

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

We did not include academic achievement as measured by the Woodcock Johnson as this was not a targeted outcome of the intervention (though outcomes on this measure are included in the report).

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

One of the measures of primary outcome collected at baseline and post-treatment was the observations of classroom behavior. Given the repeated measures nested within baseline and post-treatment for this particular measure, the observations were analyzed using a hierarchical linear modeling approach. Academic achievement measures were analyzed using Analysis of Covariance (ANCOVA) procedures, with baseline achievement scores entered as a covariate. For other measures of outcome, collected at pre and post intervention, ANCOVA procedures were also used, where pre-treatment ratings were entered as covariates in the analysis. ANCOVA procedures were used because of the unique situation a school-year study presents - teacher ratings completed at the beginning of the year reflect only a month of observation whereas endpoint ratings included knowledge of the child’s behavior over the entire year. Thus, the primary aim of these analyses was to compare groups at the endpoint rating, and the baseline ratings were entered as covariates to account for the teachers’ perception at the beginning of the school year. Within “families” of statistical tests related to primary outcomes: (1) ADHD symptoms, ODD/CD symptoms, and impairment; (2) academic performance; and (3) academic achievement, corrections to alpha levels using a modified “step down” Bonferroni approach were implemented to account for multiple statistical tests (Holm, 1979; Jaccard and Guilamo-Ramos, 2002). Table 2 contains descriptive information for all dependent measures collected at baseline and end of the intervention. Measures collected at post-treatment only were analyzed using an independent samples t-test (descriptive statistics are listed in Table 3). For all measures, estimates of effect sizes are reported where appropriate. The clinical significance of the findings was investigated by calculating the percent of participants in each group that exhibited behavior within the normative range at post-treatment (Kendall & Grove, 1988).