Main Page

Robust Learning Theoretical Framework

A key goal of LearnLab is to support learning scientists in providing explanations of results using, as much as possible, the same core terminology and addressing an accumulating body of precise theoretical principles of instruction. While a single theory of learning may emerge in the long term, the immediate goal is to encourage researchers to maximize the overlap between each others' theories. We want to help the field get beyond the "Toothbrush Problem" in theorizing.

In 2012, we published a theoretical framework, called the Knowledge-Learning-Instruction (KLI) framework. This framework builds on our 2006 theoretical framework document and, importantly, on the contributions to this wiki. In 2013, we published a paper in Science reviewing instructional principles, the complexity of their combinations, and recommendations for LearnLab style research to address this complexity.

The contents of this wiki can be accessed through multiple pathways including 1) an outline of Instructional Principles, 2) a set of early LearnLab research cluster page: Coordinative Learning, Interactive Communication, and Refinement and Fluency, and 3) a later set of LearnLab research thrust pages: Cognitive Factors Metacognition and Motivation, Social Communication, and Computational Modeling and Data Mining. See also these summaries: Cognitive Factors, Metacognition and Motivation, Social Communication, and Computational Modeling and Data Mining.

Many other on-line lists of instructional and learning principles exist [please add more!]:

• The principles below are summarized in this Instructional Complexity paper by LearnLab Director Ken Koedinger and colleagues. Thirty instructional principles are merged from lists below as shown here: File:Instructional Principles Table.xlsx. An interactive website provides a quick overview of these 30 instructional principles.
• Organizing Instruction and Study to Improve Student Learning, one of the Practice Guides of the Department of Education, Institute for Education Sciences. See also:
  • A web-based update of this guide
• Principles of Learning from Lifelong Learning at Work and at Home, a subgroup of the Association for Psychological Science.
• Design Principles Database maintained by the NSF-funded TELS (Technology Enhanced Learning in Science) project
• Principles of Teaching and Learning from CMU's Eberly Center for Teaching and Learning
• Wikipedia entry for principles of learning
• Universal Design for Learning guidelines
• See a review of principles and the role of technology in refining them in this talk by Art Graesser.

Ken Koedinger HCII Seminar Talk: Why Designing Effective Learning Interactions Is Not Easy and How We Can Do Better: Part I

Toward a Hierarchical Structure of Robust Learning Hypotheses and Findings

The structure of the research cluster and study pages is as follows (a template can be found at Project Page Template and Creation Instructions). When a set of explanations share many terms and hypotheses, we make a node for each explanation, make a node for their common features, and link the nodes so that the common-feature node is the parent of each explanation node. In most cases a "node" is a single wiki page, but sometimes a node involves several wiki pages.

In order to more clearly display the integration, each node contains:

1. An abstract that briefly describes the research encompassed by the node;
2. A glossary that defines terms used elsewhere in this node but not defined in the nodes that are parents, grandparents, etc. of this node;
3. The research question stated as concisely as possible, usually in a single sentence;
4. A background and significance section that briefly summarizes prior work on the research question and why it is important to answer it;
5. The dependent variables, which are observable and typically measure competence, motivation, interaction, meta-learning, or some other pedagogically desirable outcome;
6. The independent variables, which typically include instructional environment, activity or method (the instructional "treatment" vs. "control"), and perhaps some student individual difference variables, such as gender or first language;
7. The hypothesis, which is a concise statement of the relationship among the variables that answers the research question;
8. The findings, which are the results of the study if it has been performed or the expected findings from the study if it has not -- explicitly indicate if the findings are preliminary;
9. An explanation that describes the theoretical rationale for the hypothesis using the PSLC theoretical framework. It should be a paragraph or two and provide a causal chain that mentions mediating variables -- unobservable, hypothetical attributes of the students (e.g., knowledge components or path choices), how the treatment affects these, and how they, in turn, affect the dependent variables;
10. The descendants, which lists links to descendant nodes of this one, if there are any;
11. A further information section that points to documents using hyper links and/or references in APA format. Each indicates briefly the document's relationship to the node (e.g., whether the document is a paper reporting the node in full detail, a proposal describing the motivation and design of the study in more detail, the node for a similar PSLC research study, etc.).

Experience suggests that the glossaries carry much of the load in explaining the research, and that carefully defining and exemplifying terms often pays off later in reducing confusion and facilitating collaboration. Consequently, the glossaries are sometimes so long that they are spit off as separate wiki pages.

The root node of the hierarchy represents the overarching research question of how to achieve robust learning particularly in academic settings, like K12 schools and colleges. The question is necessarily abstract and is not the sort of question that can actually be tested by a single decisive experiment.

The immediate descendants of the root node are three nodes representing somewhat more specific research questions. There are nodes for each of Coordinative Learning, Interactive Communication and Refinement and Fluency. These present somewhat more concrete research questions. They are specializations to the overarching questions, and form a bridge to testable hypotheses posed by individual research projects.

The leaves of the hierarchy (i.e., nodes with no descendants) represent individual research studies. A leaf node can also represent a group of studies or a whole project if the activities are sufficiently similar that it makes sense to summarize them with a single node. Each leaf node is maintained by its project’s leader and may or may not be publicly accessible depending on the state of the research.

Between the cluster nodes and the leaves, there may be some intervening nodes. For instance, if a group of Coordinative Learning studies all address a similar research question (e.g., how to use verbal and visual instruction together effectively), then a node may be created to summarize their shared aspects. Its parent is the Coordinative Learning cluster node, and its descendants are the relevant project nodes.

For a draft of an alternative framework based on independent dimensions of instruction click here.