Roll IPL

Invention as preparation for learning

Ido Roll, Vincent Aleven, Bruce M. McLaren, Kenneth Koedinger

Can invention activities prepare students to better learn from subsequent instruction, compared with instruction-and-practice only?

Meta-data

PI's: Ido Roll, Vincent Aleven, Dan Schwartz, Ken Koedinger

Other Contributers: David Klahr

Study #	Start Date	End Date	LearnLab Site	# of Students	Total Participant Hours	DataShop?
1	4/2007	4/2007	North Hills	20	40	No, paper-and-pencil only
2	9/2007	12/2007	Community Day	4	48	No, paper-and-pencil only
3	4/2008	5/2008	Steel Valley	150	900	No, paper-and-pencil only

Abstract

The assistance dilemma asks what form of assistance is most appropriate for different stages of learning. While direct instruction and practice have been shown to be efficient for novices, students often acquire shallow knowledge components and lack robust understanding. Some evidence suggests that invention using contrasting cases, prior to instruction and practice, can accelerate future learning, compared with instruction and practice alone (Schwartz & Martin, 2004). The invention process as described in Schwartz & Martin (2004) includes the following stages: Design (of a mathematical model to solve a class of problems); calculation (of the solution based on the model); evaluation (of its correctness); and debugging (of the faulty model). Notably, most students fail to invent mathematically valid models, so the goal is not for students to discover the correct solution. At the same time, students do make models that capture deep features of the class of problems, which prepares them to learn and understand the significance of expert solutions for handling such situations. Following the invention, students receive instruction on the expert solution (that is, formulas) and practice it. This procedure is based on the hypothesis that students’ own inventions, together with subsequent instruction, are sources for coordinative learning. By attempting to create a model that correctly distinguishes the “contrasting cases” (carefully selected instances within a class of problems) students notice (and to some degree invent) the problem features that an adequate model must take into account, and they attend to them during subsequent instruction. However, alternative explanations for the effectiveness of the IPL process are possible, with different instructional implications. A “debugging hypothesis” suggests that evaluation and debugging of pre-designed models are sufficient to promote future learning by directing students’ attention to the short-comings of the designed models, and thus to the deep features of the domain. Alternatively, an “unfinished goals” hypothesis suggests that the effect is caused by students reaching impasses during invention. According to this hypothesis, calculation-evaluation are sufficient for preparing for future learning. We propose to investigate this in a series of ablation studies with the goal of better defining the invention process and identifying the cognitive processes involved. This includes a combination of in-vivo and lab studies within the Algebra LearnLab, contributing to the Coordinative Learning theoretical framework. Following the ablation studies we plan to implement the procedure in a Cognitive Tutor, which will be evaluated in a lab study. This will allow us to better operationalize the process, do a micro-genetic analysis of it, and identify productive patterns of learning trajectories using log mining.

Background and Significance

One of the main challenges of education is to help students reach meaningful and robust learning. The assistance dilemma raises the question of what form (and ‘amount’) of assistance are most effective with different learners in different stages of the learning process (Koedinger & Aleven, in press). Instruction followed by practice is known to be very efficient for teaching novices (e.g., Koedinger, Anderson, Hadley & Mark, 1997); yet, students often acquire shallow procedural skills, and fail to acquire conceptual understanding (Aleven & Koedinger, 2002). This can be attributed, at least in part, to students using superficial features and not encoding the deep features of the domain (Chi, Feltovich & Glaser, 1981). One approach to getting students to attend and encode the deep features is to add an invention phase prior to instruction. Invention as preparation for leaning (IPL) was shown to help students better cope with novel situations that require learning (Schwartz & Martin, 2004; Sears, 2006). In this process students are presented with a dilemma in the form of contrasting cases, and attempt to invent a mathematical model to resolve this dilemma. For example, Figure 1 shows four possible pitching machines. Students are asked to invent a method that will allow them to pick the most reliable machine. The concept of contrasting cases comes from the perceptual learning literature, since these cases, when appropriately designed, emphasize differences in the deep structure of the examples (Gibson & Gibson, 1955). The invention process includes designing a model, applying it to the given set of contrasting cases, evaluating the result, and debugging the model. This iterative process is very similar to the debugging process as described by Klahr and Carver (1988; Figure 2). Unlike other inquiry-based manipulations (cf. Lehrer et al., 2001; de Jong & van Joolingen, 1998), the goal of the IPL process is not for students to discover the correct model, but to prepare them for subsequent instruction. During the instruction students share their models, critic their peers’ models, and learn the expert solutions (A similar classroom critic process was shown to be effective by White & Frederiksen, 1998). Preparation for learning from the instruction is evaluated using accelerated future learning assessment. The accelerated future learning assessment includes an embedded instruction in the test in the form of solved example. Schwartz (2004) found that only students who invented prior to the test were able to take advantage of that instruction in order to solve novel problem, while students who practiced a given visual method prior to the test did not take advantage of the embedded learning resource and thus could not solve the target problem. This shows that the IPL process has a positive effect on students’ ability to independently learn from the solved examples. In the case of the contrasting cases given in Figure 1, subsequent instruction will introduce the students with the notion (and formulas) of variance. While the invention group was superior to instruction-and-practice group on accelerated future learning measure, there was no direct comparison of normal or transfer measures between invention and instruction-and-practice conditions (though invention students showed pre-to-post gains, and were shown to outperformed college students). Also, it is not yet clear how robust this pedagogy is and what its key features are.

Glossary

See IPL Glossary IPL Glossary

Research questions

1. Do invention activities prepare students to better learn from subsequent instruction?
2. What are several of the cognitive processes that drive that effect?
3. What properties of instruction support these processes?

In addition, the project makes the following contributions:

1. It compares different measures of robust learning, in order to understand what aspect of knowledge can be assessed using what type of measure.

Independent Variables

The existence of structured invention activities, prior to and in addition to conventional instruction and practice.

Dependent variables

• Normal (procedural) measures
• Transfer (conceptual) measures
• Long-term retention measures
• Near (pre-motivated) future learning measures
• Far (un-motivated) future learning measures
• Motivation and affect questionnaire
• Invention skills measurse
• Debugging skills measures

Hypothesis

• Invention activities help students acquire more flexible and adaptive knowledge. The further the task is from the original domain, the more benefit will be shown for the invention activities

(Normal < Transfer < Near future learning < Far future learning).

• Invention students will acquire better debugging skills, but no better invention skills.
• Invention students will be more motivated

Findings

none yet

Connections

• Tim Nokes study