Harnessing what you know

Harnessing what you know: The role of analogy in robust learning

Robert Hausmann and Timothy J. Nokes

Abstract

Knowledge transfer is a core assumption built into the pedagogy of most educational programs from K-12 to college. It is assumed that the material learned in the fourth week of the course is retained and transfers to material taught in the eighth week of the course. This is particularly true for highly structured courses such as physics; however, the empirical literature on learning suggests that far transfer is much more difficult than traditional pedagogy assumes (for reviews, see Bransford, Brown, & Cocking, 2000; Bransford & Schwartz, 1999; Gick & Holyoak, 1983). The goal of the present project is twofold. First, we will use educational data-mining models to identify knowledge components from translational kinematics that fail to transfer to rotational kinematics. Second, we will design an intervention, based upon cognitive principles from self-explanation and analogical comparison, to support knowledge components that fail to transfer.

Background and Significance

Traditional pedagogy assumes knowledge transfers between problems, units, and even courses; however, the learning literature suggests transfer is rarely observed (Detterman, 1993). Is there transfer between units in a complex science course, such as physics? If so, to what extent?

Research Objectives

Phase 1. Revise the initial knowledge-component model from the Andes physics tutor for both the translational and rotational kinematics units.

Phase 2. Develop educational data-mining models to detect the success and failure of the transfer of knowledge components. Student profiles will be defined in an effort to aggregate over individual differences in tutored help-seeking and problem-solving strategies, while still being sensitive to them.

Phase 3. Design an instructional intervention, based on cognitive science principles, to facilitate transfer between units. The format of the intervention will be designed around the literature on analogical comparison and self-explanation. The content of the intervention will be based on the revised knowledge-component model, the identification of failed knowledge-component transfer, and student profiles.

Hypotheses

H1: The learning curves from translational kinematics knowledge components can predict the error rates for rotational kinematics.

H2: Educational interventions that draw upon prior knowledge, such as analogical comparison and self-explanation, can support knowledge components that fail to transfer between translational and rotational kinematics.

Prior Work

Near and Far Transfer

Knowledge transfer is a core assumption built into the pedagogy of most educational programs from K-12 to college. It is assumed that learning is cumulative and that advanced courses will build on the knowledge and skills acquired in the introductory and foundational courses. This is especially true for the STEM disciplines, where students take a highly structured sequence of courses. To illustrate this assumption, consider the case of introductory physics, which is typically split among two semesters. In a traditional curriculum, the first semester covers Mechanics, and the second semester covers Electricity and Magnetism. Instructors of Physics II assume that the material learned in the first semester is retained and can be applied to the problems related to the motion of a charged particle in an electric or magnetic field. Moreover, most pedagogy assumes within-class transfer as well. That is, topics and concepts taught later in a course build upon and extending those taught earlier.

Unfortunately, research on human cognition has shown that knowledge transfer (especially far transfer to novel contexts and applications) is much more rare than traditional pedagogy assumes (for reviews, see Barnett & Ceci, 2002; Bransford, Brown, & Cocking, 2000). For example, in a classic study on transfer, Gick and Holyoak (1980) asked participants to solve a difficult insight problem (i.e., the solution rate was 8%). Before solving this difficult problem, all of the participants read a story that proposed an analogous solution. Half of the participants received a hint that the story will help with the solution, whereas the other half of the students did not receive a hint. The results were clear. The solution rate was much lower (i.e., 20%) for the participants who did not receive any hints, whereas those who received hints demonstrated a dramatic increase in their solution rate (i.e., 92%). These results suggest that spontaneous far transfer is difficult for students to implement.

However, because a fifth of the students were able to spontaneously transfer their knowledge of one domain to another, Gick and Holyoak (1980) demonstrate that spontaneous far transfer is indeed possible. With the appropriate scaffolding in place, it becomes quite likely. This is also true for children learning authentic science material. For instance, Brown and Kane (1988) taught pre-school children animal defense mechanisms such as mimicry. The children’s ability to transfer the concept of mimicry to other animals depended crucially on their depth of understanding. That is, if the child understood mimicry at the level of the causal structure, then they were more likely to demonstrate transfer; whereas if the child was only imitating the behavior of the teacher, then they failed to transfer the concept.

What transfers?

Often, the debate surrounding whether far transfer is tenable must address the issue of the unit of analysis. In other words, what transfers? Several hypotheses have been posited, including the doctrine of formal discipline from antiquity, Thorndike’s theory of identical elements, and Singley and Anderson’s (1989) identical-productions theory of transfer. The formal discipline theory implicated entire domains of knowledge were the units of analysis. For instance, politicians would be well advised to learn mathematics because it will cause them to be quicker thinkers (Lehman, Lempert, & Nisbett, 1988). In other words, the mind is analogous to a muscle that, when exercised properly, will increase in strength. However, early psychologists took issue with the doctrine of formal discipline and challenged it on empirical grounds. Thorndike and Woodword (1901a; 1901b; 1901c) demonstrated, in an impressive series of studies, that transfer could only be expected if the two tasks shared “common elements.” For example, receiving training on estimating the area of a rectangle did not reduce the error rate of estimating the area of a different shape (e.g. triangle).

Similar findings have been demonstrated with abstract reasoning tasks. For instance, Wason (1968) developed a deceptively simple task to assess an individual’s ability to reason about a bi-conditional rule. First-year psychology and statistics students were asked to evaluate the following rule: “If there is a D on one side of any card, then there is a 3 on its other side.” Then they were shown four cards that had a symbol on one side and another symbol in brackets indicating the contents of the back of the card. The cards were: D(3), 3(K), B(5), 7(D). The cards were placed in random order in front of the participant, and the experimenter pointed to each card and asked if that card could be used to determine if the rule was true or false. Collapsing across conditions, only 14.7% of the participants were able to correctly identify the cards that tested the veracity of the rule. In a follow-up study, Cheng, Holyoak, Nisbett, and Oliver (1986) investigated the conditions under which formal training can enhance performance on abstract tasks, such as the Wason 4-card selection task. They found, after an entire semester of instruction on logic, there was no difference in the error rate on the Wason task (Exper. 2; p. 306). Even more to the point, Cheng et al. created their own training materials that were specifically designed to improve logical reasoning. Again, they found that performance on the Wason four-card task was not improved by their customized formal instruction alone (Exper. 1).

From the available evidence, it appears that entire disciplines are not the unit of transfer, nor is the proposal of common elements of transfer specific enough to make predictions about what exactly transfers between two learning situations. A more specific theory of what constitutes an “element” is Singley and Anderson’s (1989) hypothesis that production rules, or skills, are the unit of transfer. In their analysis of learning how to use text editors, they demonstrated that the surface features can vary substantially, yet the production rules that compose the cognitive skill are transferred between editors. In PSLC terminology, production rules are equivalent to knowledge components (“Knowledge component,” 2008).

Support for the knowledge component as the unit of transfer can be found in (Corbett, McLaughlin, & Scarpinatto, 2000). According to the theory of cognitive skill acquisition, the error rate is a function of practice, and it should monotonically decrease with successive opportunities to apply the skill (i.e., the power law of learning). However, when Corbert et al. analyzed the learning curves of 34 students learning how to apply the quadratic formula, they found an inexplicable jump in the error rate at the fourth opportunity to apply the quadratic knowledge component (see Fig. 8 from Corbett et al., 2000). To address this anomaly, they conducted a fine-grained analysis of the problems and discovered that on some of the problems, the constant term, c, was zero, and on other problems, the constant term was a positive integer. They inferred that the knowledge component APPLY-THE-QUADRATIC-FORMULA was an overly general rule and should be decomposed into two smaller skills. After making the decomposition, the error rate aligned with the theoretical prediction (see Fig. 9 from Corbett et al., 2000).

From the available evidence, we believe that the unit of transfer is the knowledge component. However, as Corbett et al. (2000) demonstrates, some knowledge components are overly general and need to be evaluated empirically.

How can far transfer be supported?

Although far transfer is admittedly rare (Detterman, 1993), Gick and Holyoak (1980) and Brown and Kane (1988) demonstrated that it is possible. If it is indeed possible, how can far transfer be supported? One method for supporting far transfer is to look at the cognitive processes and mechanisms that have been identified that support robust learning. Among these are abstract schema induction through analogical comparison (Ross, Holyoak), gap-filling and repair of mental models through the generation of self-explanation inferences (Chi, 2000), meta-cognitive training (Bielaczyc, Pirolli, & Brown, 1995), and self-regulated learning (Pintrich & De Groot, 1990). We chose to focus on analogical comparison because the domain that we have chosen (i.e., translational and rotational kinematics) lends itself to analogical comparison. To illustrate why, consider the equations represented in Table 1.

Table 1. Equation isomorphisms across two units of physics.

Eqn. Translational Rotational Assumption 1

2

3

when ti = 0 4

when si or i = 0 5

Each equation listed in a row is exactly analogous to the equation in its neighboring column. The only difference between the two is that the symbols represent different concepts. For example, in translational kinematics, the vector symbol, , represents the average velocity. Likewise, the vector symbol stands for the average rotational velocity. A similar mapping exists for the other symbols as well: average acceleration ( ) is analogous to average angular acceleration ( ); displacement ( ) is analogous to the angular displacement ( ).

An additional feature that makes these two units attractive to an analogical-comparison approach is that there are additional concepts to learn besides those listed in Table 1. The additional concepts include radial and tangential acceleration, which do not have analogs in translational motion. This presents an opportunity to measure the existence of accelerated future learning.

In addition to the content lending itself to analogical comparison, prior research on analogical comparisons suggests that it is an effective instructional intervention because it draws upon the student’s background knowledge. Prior research has shown that students can be guided to construct abstract schemas from making the explicit mapping between two different domains. Educational applications of analogical comparison is in large part inspired by Gentner’s (1983) structure-mapping framework, which states that analogical reasoning is a process whereby an individual creates a mapping between the target (i.e., the unknown domain) and the base (i.e., the known domain). The literal features of the target and base domains are abstracted away to leave only the second-order relations between the objects. Gentner’s (1983) structure-mapping framework has been used to inform the design of educational interventions. For instance, Ross and Kilbane (1997) attempted to instruct students on solving combination and permutation problems. Specifically, they were interested in measuring the impact of changes made to the variables’ mappings between the study and test problems. For instance, they manipulated whether students solved problems that had identical or dissimilar cover stories. For example, if the study problem was about knights choosing horses for a jousting tournament, a test problem with a similar cover story also used knights and horses. A test problem with a dissimilar cover story, however, used puppies and owners. Both types of test problems, however, reversed the object correspondences such that the horses were now responsible for choosing their riders and puppies choosing their owners. The results from Experiment 2 suggest that the students were able to use the embedded instructional explanations to allow them to see past the superficial features, and make the selection of their variables according to the domain principles.

Although Experiment 2 of Ross and Kilbane (1997) was effective, there are two major constraints placed on the usefulness of analogical comparison as an effective pedagogical intervention. The first constraint is the observation that students tend to rely too heavily on the surface features of the analogy (see Exper. 1 from Ross & Kilbane, 1997). The second constraint is that the base domain needs to be well understood by the learner before the mapping to the target domain can make sense (Gentner, Loewenstein, & Thompson, 2003; Kurtz, Miao, & Gentner, 2001). One proposed solution to this limitation is to bootstrap understanding via analogical encoding, which is the idea that students can use an imperfect understanding of two related base domains to understand their deeper structure and principles. To evaluate the efficacy of analogical encoding, Kurtz, Miao, and Genter (2001) asked students to make an explicit correspondence between two images depicting heat transfer. They demonstrate that students, who were asked to make an explicit list of correspondence between the objects of the two scenarios, rated the two disparate situations as more similar than students who were not asked to make systematic comparisons. Unfortunately, Kurtz et al. (2001) did not administer a pretest to diagnose the participant’s initial understanding of the target domain; therefore, it is difficult to assess if the outcome of the analogical encoding was a robust understanding of heat transfer.