Metacognition and Motivation

The Metacognition and Motivation thrust builds on the Coordinative Learning (CL) cluster, while bringing a significant shift of focus. Work within the CL cluster addressed specific versions of the general cluster question: “When and how does coordinating multiple sources of information or lines of reasoning increase robust learning?”

Examples and Explanations. One of these specific questions is “when and how does the use of worked examples in problem-solving instruction enhance robust learning?” Studies in geometry (c.f., Salden, Aleven, Renkl, & Schwonke, 2008; Schwonke, Wittwer, Aleven, Salden, Krieg, & Renkl, in press) showed that (interactive) worked examples are an effective supplement to tutored problem solving (supported by a Cognitive Tutor), leading to more robust learning. Studies in the Chemistry LearnLab course (c.f., McLaren, Lim, Gagnon, Yaron, & Koedinger, 2006) found that the combination of worked examples and tutored problem solving improved the efficiency of robust learning. Studies in the Physics LearnLab course found that analogical comparisons and self-explanations of worked examples facilitate conceptual learning (Nokes & VanLehn, 2008).

Visualization and multi-modal sources. The second sub-question is, “When does adding visualizations or other multi-modal input enhance robust learning and how do we best support students in coordinating these sources?” A study by Liu, Perfetti, and Mitchell in the Chinese LearnLab took inspiration from the machine learning theory of co-training (Blum & Mitchell, 1998) to explore whether students might benefit from the combination of multiple sources of information. The study confirmed the general prediction (derived from machine learning research) that co-training improves learning. However, it failed to confirm a more specific prediction, that sources need to be non-correlated to enhance learning – it may be that lack of correlation is necessary for machines, but not for humans. Another series of studies in this sub-cluster has explored the potential benefits of a tighter integration of text and diagrams in the Geometry LearnLab. Butcher and Aleven (2007; 2008) found benefits for robust learning of more contiguous text and diagram presentation. Other studies exploring research questions within the Coordinative Learning cluster can be found at http://learnlab.org/research/wiki/index.php/Coordinative_Learning. The M&M thrust will continue some of the work in the Coordinative Learning cluster that focused on the metacognitive aspects of coordinating multiple sources of information, such as studies on analogical comparison and self-explanation of examples by Nokes, and studies on diagrammatic self-explanation by Aleven and Butcher. It will also build on work done in other thrusts, for example the work done by Hausmann and VanLehn on scaffolding self-explanations in peer collaborative settings, the work on help seeking by Roll, Aleven, et al, and the work on studying gaming the system and off-task behavior by Baker et al. In addition, the M&M thrust aims to place greater emphasis on issues of motivation within learning sciences research than has been done so far within the PSLC.

Specifically, the Metacognition and Motivation thrust has two broad goals, 1) to develop a better understanding of how metacognitive processes and motivation interact with learner factors to influence robust student learning outcomes and 2) to test whether and how student learning environments can leverage improved metacognition and motivation to increase the robustness of student learning. Our research will focus on a small number of metacognitive abilities (e.g., help seeking, self-explanation, interpreting peer feedback, and interpreting textual descriptions of domain principles), and a broader range of affective and motivational variables including: challenge perception, boredom, frustration, performance goals, and off-task behavior. During the upcoming year (year 5 of the PSLC), we will focus on planning and preparing for the Metacognition and Motivation thrust’s activities. The thrust will meet regularly, replacing the regular meetings of the Coordinative Learning cluster. We will conduct research that begins to address the main research questions outlined above, through, for example, projects to create tutors with game features as a way of enhancing student motivation. The thrust’s main research activities will occur subsequent to the PSLC renewal, which we hope will start in October 2009. We have recruited three senior and highly distinguished consultants who will help to increase both the quality of the Metacognition and Motivation research and its visibility within broader communities of metacognition and motivation researchers. They are: Dr. Barry J. Zimmerman, a pre-eminent scholar in metacognition and motivation (e.g., Schunk & Zimmerman, 2008), Dr. Josh Aronson, a distinguished expert in stereotypes, self-esteem, motivation, and attitudes, and Dr. Andrew Elliott, a well-known expert in achievement motivation and social motivation. We will pursue the following three broad research directions during the renewal period: Create and validate automated detectors for affect, motivation, and meta-cognition. We will start by enhancing the LearnLab infrastructure with technology for automatically monitoring metacognitive and affective variables, at a much finer grain-size, over longer durations, and for more students, than has been previously possible.

Specifically, we combine observational and questionnaire data with student log data (e.g. response times, patterns of activity), to develop machine-learned models that monitor, in real-time and moment-by-moment, affective, motivational, and metacognitive variables in interactive learning environments. In particular, we will develop detectors of such constructs as gaming the system, off-task behavior, help seeking, boredom, frustration, flow, perception of challenge, self-efficacy, and performance goals. Once created, these detectors will only draw on information that is available to the learning environment in the normal course of its operation (student log data at the keystroke level, timing, and semantic levels), without requiring extra sensors, enabling these detectors to be used in authentic, unmodified learning settings. The combination of machine learning with observational and questionnaire data has already been successful at detecting a limited set of relevant constructs such as perception of challenge (de Vicente & Pain, 2002), performance goals (Baker et al., 2005), and off-task behavior (Baker, 2007).

These detectors will be implemented in learning software used in multiple LearnLabs, and will enable PSLC researchers across thrusts to study motivation and metacognition as mediating variables when evaluating interventions aimed at enhancing the robustness of student learning. Evaluate interventions aimed at supporting different metacognitive abilities. The PSLC’s particular strength in this area is studying metacognition within the context of interactive learning environments. A key question is whether such environments can be as effective in fostering or supporting metacognitive skills as they have been in improving domain-specific learning. A number of recent in vivo experiments have revealed significantly improved domain learning among students given metacognitive tutoring support for self-explanation (Aleven & Koedinger, 2002), error-correction (Mathan & Koedinger, 2005), video-based prompting and peer collaborative scaffolding of self-explanation (Craig et al. 2007, 2008, submitted; Hausmann & Vanlehn, 2007a, 2007b), and remedial instruction for material missed through meta-cognitive errors (Baker et al, 2006). We will build on this earlier work, with interventions that attempt to support four metacognitive abilities: help seeking, self-explanation, interpreting peer feedback, and interpreting textual descriptions of domain principles. An important goal in the Metacognition and Motivation thrust is to develop interactive learning environments that can help students internalize this support, solidifying and generalizing their metacognitive skills so they will no longer need external support in future learning situations, and can approach a new domain with a general set of skills that can facilitate learning.

To evaluate the effectiveness of the interventions aimed at enhancing metacognition, we will look not only at the normal indicators of robust domain-level learning (i.e., transfer and retention), but also (and in particular) at whether future learning is accelerated. As appropriate, the detectors for metacognitive behaviors developed under goal 1 will be used to evaluate whether the targeted metacognitive behavior is enhanced both while the intervention is in place (as a manipulation check) as well as in future learning situations (to evaluate its role as a potential cause of accelerated future learning).

Evaluate interventions aimed at inducing positive affect and motivation to persist. As a complement to investigating how cognitive learning principles can improve and support metacognitive ability, we will also study the effect of interventions aimed at enhancing motivation, as a way of uncovering relationships between motivation, affect, and metacognition in interactive learning environments. We will focus on two types of interventions. First, inspired by the motivational impact of computer games, we will (a) identify features of games that could be adopted for use in interactive learning environments, and (b) evaluate the effect of adding these features. Our initial investigations will focus on trivial choice, “boss problems,” (challenge problems – e.g. Siegler & Jenkins, 1981, designed to look like end-of-level bosses in video games) student control over challenge level, and rewards. In addition, we will evaluate the effect of putting the student in a care-taking role where they need to tutor a synthetic student. (The synthetic student will be driven by the PSLC’s SimStudent learning agent technology.) Second, we will evaluate social interventions aimed at enhancing motivation: peer pressure and comparison with peers, including competition with simulated students.