Social Skills Strategies: The LEARN Strategy, a Cooperative Thinking Strategy
Study: Vernon (2020)

Describe how students were selected to participate in the study:

Participants were regularly enrolled in 25 participating teachers' fourth- and fifth-grade inclusive general education classes.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional or behavioral difficulties (BI):

Their school records were reviewed to verify their status/diagnoses.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

77.0%

Specify which condition is the submitted intervention:

The submitted intervention was used by the experimental teachers in their classes. It comprised the experimental condition.

Specify which condition is the control condition:

The control condition was comprised of the control teachers conducting the same number of cooperative group activities in their classrooms as the experimental teachers.

If you have a third, competing condition, in addition to your control and intervention condition, identify what the competing condition is (data from this competing condition will not be used):

NA

Using the tables that follow, provide data demonstrating comparability of the program group and control group in terms of demographics.

Grade Level

Race–Ethnicity

Socioeconomic Status

Disability Status

ELL Status

Gender

What method was used to determine students' placement in treatment/control groups?

Random

Please describe the assignment method or the process for defining treatment/comparison groups.

Twenty-five teachers participated. Their names were randomly picked out of a hat until 13 had been selected to implement the intervention (teach the LEARN Strategy). their students served as the treatment group. The remaining teachers' students served as the control group.

What was the unit of assignment?

Classes

If other, please specify:

Please describe the unit of assignment:

Twenty-five teachers and their classes were randomly assigned to either the treatment or control group. Thirteen classes served as the treatment group; twelve served as the control group. Thus, the students regularly enrolled in the classes of these teachers served as the treatment and control groups.

What unit(s) were used for primary data analysis?

Schools
Teachers
Students
Classes
Other
If other, please specify:
For some of the measures, individual students' scores were analyzed because the students were either administered tests individually, or their individual social behavior was observed. Cooperative group tests were also administered; thus, some of the data were analyzed for the groups of students.

Please describe the unit(s) used for primary data analysis:

Individual student tests included a test of social knowledge (40 points available) and a test of academic knowledge learned (28 points available). Observers watched the individual students as they worked in their cooperative groups. The measures collected here on individual students were the number of behaviors emitted by a given student during 15 minutes in the categories of prosocial behaviors, teamwork behaviors, and antisocial behaviors. Students were also rated by their classmates on a sociometric measure (5 points). Group measures included a study card measure (98 points available) and a study-group behavior measure (36 points available). Students were assigned to small cooperative groups containing at least one student with LD (or another exceptionality) and two or three average-achieving students. These groups were given an assignment to learn a certain amount of information. Four groups were observed within each classroom for the cooperative group measures (study group behavior and study cards).

How was the program delivered?

Individually
Small Group
Classroom

What were the background, experience, training, and ongoing support of the instructors or interventionists?

They had no experience teaching the program. They were given the instructor's manuals and asked to read the manuals and implement the instruction as specified in the manuals.

Describe when and how fidelity of treatment information was obtained.

Observers watched the experimental teachers as they taught the SCORE Skills and the LEARN Strategy. They recorded the steps that each teacher implemented of the steps listed in the instructors manuals. The percentage of agreement between two independent observers on this measure was 81% on the SCORE Skills (386 agreements out of 474) and 90% (338 agreements out of 376 opportunities to agree) on the LEARN Strategy steps.

What were the results on the fidelity-of-treatment implementation measure?

The experimental teachers implemented a mean of 82% of the instructional steps in the SCORE Skills manual (range = 67% to 100%). They implemented a mean of 84% of the instructional steps in the LEARN Strategy manual (range = 71% to 95%).

Was the fidelity measure also used in control classrooms?

No

For any substantively (e.g., effect size ≥ 0.25 for pretest or demographic differences) or statistically significant (e.g., p < 0.05) pretest differences, please describe the extent to which these differences are related to the impact of the treatment. For example, if analyses were conducted to determine that outcomes from this study are due to the intervention and not pretest characteristics, please describe the results of those analyses here.

Prior to any analyses being performed, the posttest scores were adjusted statistically for any differences between the groups being compared (e.g., for age, gender, pretest scores). The ANCOVA for the study-group behavior measure revealed a significant difference between the adjusted posttest scores of the experimental and control groups, F(1, 22) = 21.23, p < .001,  2 = .49, representing a large effect size. For the experimental groups, results indicated that the students' performance of behaviors associated with the LEARN Strategy significantly increased from the pretest mean to the posttest mean, t(13) = 8.67, p <.001, eta squared = .76, representing a large effect size. No significant difference was found for the control groups. For the study-card measure, the ANCOVA revealed that the adjusted posttest mean for the experimental cooperative groups (M = 40.97) was significantly larger than the adjusted posttest mean for the control groups (M = 17.31), and the difference was significant, F(1, 22) = 8.48, p = .008, eta squared = .28, representing a large effect size. Both experimental and control groups scored significantly higher on the posttest than the pretest. For the Academic Knowledge Test, ANCOVA revealed a significant difference between the adjusted posttest results of the experimental and control groups for all students with exceptionalities , F(1, 22) = 18.59, p < .001, eta squared = .45, and for AA students, F(1, 22) = 6.22, p = .022, eta squared = .22, representing large effect sizes. Both types of students earned significantly higher scores on the posttest than the pretest. For the Social Knowledge Test, ANCOVAs revealed significant differences between the experimental and control students’ adjusted posttest scores for students with exceptionalities, F(1, 22) = 73.10, p < .001, eta squared = .77, and for AA students, F (1, 22) = 400.29, p <.001, eta squared = .95, representing large effect sizes. For social behaviors performed in the cooperative-group activities, ANCOVAs revealed significant differences between the adjusted posttest scores of students with LD in the experimental and control classes with regard to prosocial behaviors, F(1, 32) = 8.44, p = .007, eta squared = .21, and teamwork/study behaviors, F(1, 32) = 6.56, p = .015, eta squared = .17, representing large effect sizes. No differences were found between the experimental and control groups with regard to the sociometric ratings. However, when the data for the least accepted students (rating less than 2.0) were compared across the groups, ratings were significantly higher on the posttest than on the pretest for both the experimental and control groups (for students with LD, t (21) = 3.90, p = .001, eta squared = .28, for all students with exceptionalities, t (12) = 3.91, p = .002, eta squared = .41, but not for AA students, t (12) = 0.82, p = .431, eta squared = .03). (See the article for more details on the results.)

Please explain any missing data or instances of measures with incomplete pre- or post-test data.

NA

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

Experimental teacher and student satisfaction ratings were collected using satisfaction questionnaires. These ratings were not considered to be outcome measures; they are social validity measures. They reflect the teachers' and students' opinions related to the SCORE Skills and LEARN Strategy programs. Thus, they were not reported here.

Describe the analyses used to determine whether the intervention produced changes in student outcomes:

One-way analyses of covariance (ANCOVAs) were conducted to assess differences between the posttest scores of the students in the experimental and control groups on use of the LEARN Strategy, study-card quality, scores on the Social Knowledge Test and the Academic Knowledge Tests, social behaviors performed during the group activities, and student sociometric ratings. In these analyses, the pretest scores served as the covariate. Prior to any analyses being performed, the posttest scores were adjusted statistically for any differences between the groups being compared (e.g., for age, test scores). Next, analyses were performed on data collected for the group scores: study-group behavior and study-card creation. For individual scores, separate analyses were then performed for all students with exceptionalities and for AA students. On the measure of social behavior during group activities, a separate analysis was conducted for the subgroup of students with LD, since these students were observed individually, and research has shown that these students tend to have social deficits (Kavale & Forness, 1996; McDaniel et al., 2019; Authors, 1992; Vaughn et al., 2004). For each of the ANCOVAs, the dependent variable was the posttest score for a given measure, and the covariate was the pretest score for that measure. Individual student scores were used in the analyses for the Social Knowledge Test, the Academic Knowledge Test, the Social Behavior (CASSI) scores, and the peer rating scores, but group scores were used in the analyses of the study-group scores and the study-card scores. Additionally, dependent-samples t-tests were conducted to evaluate differences between pretest scores and posttest scores for participants with and without exceptionalities in the experimental and control groups separately.