Learning Strategies Curriculum: Test-Taking Strategy
Study: Hughes & Schumaker (1991)

Describe how students were selected to participate in the study:

Six students, five eighth graders and one seventh grader (five males and one female), served as subjects. Half the students were white; half were black. Ages ranged between 13.1 years and 17.2 years (M - 15.1 years). Full-scale IQ scores (derived from the Weschler Intelligence Scale for Children—Revised) ranged from 80 to 101 (M = 88), and their reading grade-level scores (derived from the Metropolitan Achievement Test) ranged from 4.0 to 7.1 (M = 5.2). All the students, who were enrolled in a resource class for one or two periods a day, had been formally classified as having learning disabilities according to state of Florida guidelines. To qualify for participation in this study, students had to be (a) classified as having a learning disability; (b) enrolled in a mainstream science or social studies course; (c) capable of reading at or above the fourth-grade level; and (d) earning below-average or failing test scores in at least one science or one social studies course.

Describe how students were identified as being at risk for academic failure (AI) or as having emotional/behavioral difficulties (BI):

Guidelines in the state of Florida at the time the study specified that students were to be achieving two or more standard deviations below the mean in one or more core academic skill areas to be classified as a student with a learning disability. Additionally, both general education and special education teachers nominated the students who participated in the study as meeting the learning disability criteria.

ACADEMIC INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• below the 30th percentile on local or national norm, or
• identified disability related to the focus of the intervention?

100.0%

BEHAVIORAL INTERVENTION: What percentage of participants were at risk, as measured by one or more of the following criteria:

• emotional disability label,
• placed in an alternative school/classroom,
• non-responsive to Tiers 1 and 2, or
• designation of severe problem behaviors on a validated scale or through observation?

%

Please describe the study design:

Multiple Probe Design Across Three Participants Replicated Once. NOTE: The DESIGN was repeated once. Each design contained three participants for a total of six participants.

Clarify and provide a detailed description of the treatment in the submitted program/intervention:

The overriding goal of this intervention was to teach students a strategy for taking tests in the general education classroom (e.g., end of chapter or unit tests) that consist of the most commonly administered types of tests (i.e., those containing multiple-choice, fill-in-the-blank, matching, and short essay questions). A set of steps were designed based on the following factors: (a) the most frequently cited test-taking principles and strategies reported in the literature, (b) an analysis of the nature and extent of cued items in content tests, and (c) a task analysis of test-taking behaviors. The identified behaviors were placed in sequence. To help students remember the behaviors, a first-letter mnemonic device (“PIRATES”) was designed to increase the likelihood that students would remember the strategy steps in a testing situation. In the Prepare to Succeed step, students say a positive statement to themselves to affirm their belief in themselves as test takers, scan the test to determine sections and types of questions, consider difficulty of and rank each section, and begin working on the first-ranked section. In the Inspect the Instructions step, students read the instructions for the first ranked section, underlining key words that suggest how and where to respond. In the Read, Remember, Reduce step, student read the item, remember what they have learned/studied, and eliminate obviously wrong response options. In the Answer or Abandon step, students either answer the item or they abandon it for the moment by marking it with a symbol to remind them to answer it later. Students continue cycling through the third and fourth steps until they reach the end of a section. Then they apply the Inspect the Instructions step to the next set of instructions and repeat other steps until they reach the end of the test. At this point, they complete the Turn back step by going back to the beginning of the test to answer abandoned items. If they don’t know the answer, they apply the Estimate step to help them make an informed guess. They are taught specific guessing techniques based on the empirical literature on test-taking (e.g., avoid absolutes). Finally, they Survey the test by looking over the entire test to ensure that they have not left any question unanswered. To standardize the instructional procedures, an instructional manual was written (Hughes, Schumaker, Deshler, & Mercer, 1988). This manual contains very detailed scripts for teachers to follow to ensure both accuracy and consistency of the instruction. The instructional methodology was comprised of seven stages similar to those described in Ellis, Deshler, Lenz, Schumaker, and Clark (1991) for promoting strategy acquisition and generalization in adolescents with LD. The first three stages were: Describe (introduce the purpose of the Test-taking Strategy and provide rationales for using it and description of each strategy step); Model (the researcher modeled how to complete each strategy step in sequence while thinking aloud so students could witness all the cognitive processes involved in performing the strategy steps); Verbal rehearsal (the researcher led students in a rapid-fire exercise for the purpose of rehearsing the steps of the strategy. This practice occurred initially as a group and then individually to ensure that each student met mastery in naming the steps of the strategy. In the fourth stage, Initial practice, students applied the steps of the test-taking strategy to practice tests constructed at their instructional reading level and received individual elaborated feedback after each practice attempt. Students continued to practice until they met the mastery criterion of 90% of the required responses. Next, they engaged in Advanced practice where the students were administered practice tests constructed at their grade level. After completing each practice test, they were given individual elaborated feedback until they reached a mastery criterion of 90%). In the Generalization stage, students were encouraged to use the Test-taking Strategy each time they took a test in their general education classes. To assist themselves, they made cue cards to carry with them that listed the strategy steps. After they took a test in their general education class, the researcher held an individual meeting with each student to review how they applied the strategy and to provide feedback and suggestions for what they could do differently on their next test. During the Maintenance stage, approximately every two weeks, students completed a practice test to check the maintenance of their use of the strategy steps.

Clarify what procedures occurred during the control/baseline condition (third, competing conditions are not considered; if you have a third, competing condition [e.g., multi-element single subject design with a third comparison condition], in addition to your control condition, identify what the competing condition is [data from this competing condition will not be used]):

During baseline, separate tests were administered to the students on different days. For each administration, the test was distributed, and the students were told that they would have 25 minutes to complete the test. Further, they were instructed to answer each question as well as possible and to do everything they normally would to earn the best grade possible on the test. They were informed that the purpose of the test was to determine how they take tests. They were also told that they might not know the answers to several items on the test, but that they should make their "best guess" in such instances. Finally, they were reassured that the grade they would earn on the test would not be included in their grade for a course. The researcher administered all of the practice tests.

Please describe how replication of treatment effect was demonstrated (e.g., reversal or withdrawal of intervention, across participants, across settings)

A multiple-probe design (Horner & Baer, 1978) was employed. Three students participated in each of two applications of the design. All students received at least two practice tests during baseline. When the baselines of the first two students in each design were stable, they received instruction that was followed by at least two practice tests and then by at least one maintenance-practice test. Once the first student within each multiple-probe design met mastery on two practice tests, the second student in that design was administered at least one more baseline practice test in addition to the two practice tests that had already been administered. When his or her baseline was stable, instruction began. After the second student in a design met mastery on two practice tests, the third student in that design was administered at least one more baseline practice test. Again, the instructional methodology was implemented when the third student’s baseline was stable. All students were administered at least one maintenance practice test. Thus, all students experienced three conditions: baseline, instruction, and maintenance.

Please indicate whether (and how) the design contains at least three demonstrations of experimental control (e.g., ABAB design, multiple baseline across three or more participants).

It was replicated across participants four times – twice in each of the two designs.

If the study is a multiple baseline, is it concurrent or non-concurrent?

Concurrent

How was the program delivered?

Individually
Small Group
Classroom

Study measures are classified as targeted, broader, or administrative data according to the following definitions:

• Targeted measures
  Assess outcomes, such as competencies or skills, that the program was directly targeted to improve.
  • In the academic domain, targeted measures typically are not the very items taught but rather novel items structured similarly to the content addressed in the program. For example, if a program taught word-attack skills, a targeted measure would be decoding of pseudo words. If a program taught comprehension of cause-effect passages, a targeted measure would be answering questions about cause-effect passages structured similarly to those used during intervention, but not including the very passages used for intervention.
  • In the behavioral domain, targeted measures evaluate aspects of external or internal behavior the program was directly targeted to improve and are operationally defined.
• Broader measures
  Assess outcomes that are related to the competencies or skills targeted by the program but not directly taught in the program.
  • In the academic domain, if a program taught word-level reading skill, a broader measure would be answering questions about passages the student reads. If a program taught calculation skill, a broader measure would be solving word problems that require the same kinds of calculation skill taught in the program.
  • In the behavioral domain, if a program taught a specific skill like on-task behavior in one classroom, a broader measure would be on-task behavior in another setting.
• Administrative data measures apply only to behavioral intervention tools and are measures such as office discipline referrals (ODRs) and graduation rates, which do not have psychometric properties as do other, more traditional targeted or broader measures.

If you have excluded a variable or data that are reported in the study being submitted, explain the rationale for exclusion:

No variables have been excluded.