Major Accomplishments by Center Strategy & Research Thrust
LearnLab had three major strategies: research integration, research facilitation, and education and diversity. Accomplishments associated with these strategies are summarized below.
LearnLab Research Integration
In 2012, we published a center-level integrated theoretical framework in Cognitive Science (Koedinger, Perfetti, & Corbett, 2012 cited by 147). This publication introduced the Knowledge-Learning-Instruction (KLI) Framework that identifies and interconnects three taxonomies: kinds of knowledge, kinds of learning processes, and kinds of instructional methods. Many subsequent projects and publications have pursued the KLI Framework. One of those publications appeared in Science in 2013 (Koedinger, Booth, & Klahr, 2013). It presented the challenge of “instructional complexity” and recommendations for addressing it employing the KLI Framework and the kind of socio-technical cyberinfrastructure that LearnLab has developed (see Figure 3).
Figure 3. Widely-used educational technologies provide the basis for running hundreds of in vivo studies as part of real courses and, at the same time, collecting long-term (over months) and detailed (every 10 seconds) learning process data that informs new learning theory, such as the KLI framework.
A transformative scientific discovery of LearnLab is the “KLI dependency”, namely, that optimal learning depends on the nature of the knowledge to be acquired (Figure 4). A one-size-fits-all approach to unlocking the mysteries and potential of human learning has been pursued for too long and does not work (Koedinger, Booth, Klahr, 2013). The KLI dependency is substantiated through 100s of scientifically rigorous and practically relevant in vivo experiments run through educational technology use in courses in math, science, and language learning. It is explained in terms of elements of detailed taxonomies of knowledge types (e.g., facts, skills, and principles), of learning processes (e.g. memory, induction, and sense making), and of instructional principles. Specific theoretical alignments of these elements — facts are best learned by memory processes, skills by induction processes, and principles by sense making processes — are supported by vast amounts of empirical evidence — as elaborated in the research thrust sections below.
Figure 4. Designing for learning depends on understanding the nature of the to-be-acquired knowledge (facts, skills, principles). Based on 100s of studies, KLI explains why some instructional principles are effective in some cases (+), ineffective in others (0), and sometimes counter-productive (-).
The KLI framework is transformative because it provides a theoretical and methodological alternative to the failed idea that learning science can be directly applied to improve educational outcomes. KLI has profound practical implications in guiding the design of powerful personalized learning systems that have been demonstrated in large-scale field trials to double school algebra achievement (Pane, et al., 2014; Ritter, Anderson, Koedinger & Corbett, 2007, cited by 246; Koedinger & Corbett, 2006, cited by 359) and cut in half that time it takes to learn college statistics (Lovett, Meyer, & Thille, 2008). KLI shows how to go beyond simple application of instructional recommendations (cf., Pashler et al., 2007, cited by 278) to consider how to manage an “assistance dilemma” created by competing benefits of challenging students to enhance long-term learning outcomes versus assisting them toward immediate success (Koedinger & Aleven, 2007; cited by 335).
KLI also has profound scientific implications. One is that computational models of brain function and machine learning cannot be monolithic, but must include multiple learning mechanisms that address different demands. LearnLab researchers developed computational models of representation learning through exposure (Li et al., 2015), skill learning through practice with feedback (Matsuda, Cohen & Koedinger, 2015), and language-based learning to facilitate sense-making (Rose et al., 2008, cited by 242). They have demonstrated across multiple domains (e.g., math, science, and language) how in-depth considerations of learning theory can yield new insights and better student learning. For example, Nokes-Malach, & Mestre, (2013) provide an interesting and important elaboration of the connection between the KLI constructs of sense-making and robust learning, as measured by assessments of transfer. They detail and illustrate how knowledge transfer consists of constructing a representation of context, generating a solution, and evaluating whether the solution makes sense given that context. Contributing evidence to the KLI dependency, Nokes, Hausmann, VanLehn, & Gershman (2011) demonstrated how the efficacy of instruction depends on the match between the cognitive processing that the instruction elicits, how those processes modify the underlying knowledge representations for the task, and the utility of those representations for the task or problem. They found that self-explanation prompts designed to focus the learner on gap-filling led to greater learning and less tutoring assistance than prompts designed for mental-model revision.
A second scientific implication of KLI is that that laboratory studies of learning are insufficient to guide improvements in educational practice. Instead schools, colleges, and learning content providers need to adopt an iterative data-driven engineering process (cf., Koedinger, Booth, & Klahr, 2013). In this process the choice and application of learning principles must be guided cognitive task analysis of the specific nature of the target content knowledge. It must be supported through educational technologies and a social cyberinfrastructure for collecting, analyzing, and sharing learning data and associated analytics (Koedinger, Booth, & Klahr, 2013; see also the Global Learning Council recommendations). In other words, all educational institutions should operate like LearnLabs (see Figure 5)!
Figure 5. With new opportunities for technology-enhanced learning and the challenge of effective learning design, educational institutions can and should be engaging in their own efforts to do learning science. Changing culture to facilitate continuous improvement and global collaboration are critical.
The KLI framework grow out of research integration efforts across and within four LearnLab research thrusts: Social Communicative, Cognitive Factors, Cognitive Modeling and Data Mining, and Motivation and Metacognition. We next summarize key scientific discoveries from these four thrusts.