Difference between revisions of "Optimizing the practice schedule"

From LearnLab
Jump to: navigation, search
(Independent variables)
 
(72 intermediate revisions by 4 users not shown)
Line 1: Line 1:
=== Optimizing the practice schedule project === 
 
 
[[Category:Project]]
 
 
 
=== Abstract ===
 
=== Abstract ===
This project plan extends dissertation work of Pavlik. In this initial work, a model-based algorithm was described to maximize the rate of learning for simple facts using flashcard like practice by determining the best schedule of presentation for a set of facts. The goal of this project plan is to develop this initial work to allow this tutor with optimized scheduling to handle more complex information and different types of learning in more natural settings (like LearnLabs). Specifically, this project plan describes extensions to the theory in two main areas.  
+
This project plan extends dissertation work of Pavlik. In this initial work, a model-based algorithm was described to maximize the rate of learning for simple facts using flashcard like practice by determining the best [[instructional schedule]] for a set of facts. The goal of this project plan is to develop this initial work to allow this tutor with [[optimized scheduling]] to handle more complex information and different types of learning in more natural settings (like LearnLabs). Specifically, this project plan describes extensions to the theory in two main areas.  
  
:1.  Specification of a theory of refined encoding
+
:1.  Specification of a theory of [[refinement]]
 
::a.  Generalization practice (multimodal and bidirectional training)
 
::a.  Generalization practice (multimodal and bidirectional training)
 
::b.  Discrimination practice (detailed error remediation)
 
::b.  Discrimination practice (detailed error remediation)
:2.  Specification of a theory of co-training
+
:2.  Specification of a theory of [[co-training]]
::a.  Effect of declarative memory chunk sequence during learning
+
::a.  Effect of [[declarative]] memory chunk [[schedule of presentation]]  during learning
::b.  Effect of declarative memory chunks on production rule learning
+
::b.  Effect of [[declarative]] memory chunks on [[procedural]] learning
  
These theoretical directions are intended to enhance the optimization tutor by greatly extending its capabilities.  
+
These theoretical directions are intended to enhance the [[FaCT System]] tutor by greatly extending its capabilities.  
  
A secondary goal of the project is to link the optimization algorithm used in this project with the larger CTAT project. In this linkage the optimization algorithm would be integrated onto the current CTAT system as a curriculum management system that could select or generate problems according to the algorithm, but using CTAT interfaces. This integration will make it easier for people to use the optimal learning system and therefore increase its impact and usefulness.
+
A secondary goal of the project is to link the optimization algorithm used in this project with the larger [[CTAT]] project. In this linkage the optimization algorithm would be integrated onto the current [[CTAT]] system as a curriculum management system that could select or generate problems according to the algorithm, but using [[CTAT]] interfaces. This integration will make it easier for people to use the [[optimized scheduling]] system and therefore increase its impact and usefulness.
  
 
=== Glossary ===
 
=== Glossary ===
* [[Optimal Spacing Interval]]
+
* [[Optimal spacing interval]]
* [[Expanding Spacing Interval]]
+
* [[Expanding spacing interval]]
* [[Wide Spacing Interval]]
+
* [[Optimized scheduling]]
* [[Narrow Spacing Interval]]
 
  
 
=== Research question ===
 
=== Research question ===
How can the optimal sequence of learning be computed?
+
How can the optimal sequence of [[learning event]]s be computed? The descendants section below links to LearnLab and laboratory research tracks that have employed and invetigated these methods of optimal sequencing.
  
 
=== Background and significance ===
 
=== Background and significance ===
  
Since the early 60's researchers in learning theory have been describing models of practice which attempt to capture the effect of practice on performance at a later time. These models are applicable to describing many types of learning situations, but are easier to apply where information to be learned can be broken up into small chunks that can be learned independently. For instance, Atkinson (1972) applied a Markov model of learning to schedule drill of German vocabulary.
+
Since the early 60's researchers in learning theory have been describing models of practice which attempt to capture the effect of [[practice]] on performance at a later time. These models are applicable to describing many types of learning situations, but are easier to apply where information to be learned can be broken up into small chunks that can be learned independently. For instance, Atkinson (1972) applied a Markov model of learning to schedule [[drill]] of German vocabulary.
  
 
More recently there has been a renewed emphasis on repeated practice. For instance, the National Council of Teachers of Mathematics new report [http://online.wsj.com/article_email/SB115802278519360136-lMyQjAxMDE2NTE4MjAxMjIyWj.html WSJ article] emphasizes the importance of this type of learning for simple math skills.
 
More recently there has been a renewed emphasis on repeated practice. For instance, the National Council of Teachers of Mathematics new report [http://online.wsj.com/article_email/SB115802278519360136-lMyQjAxMDE2NTE4MjAxMjIyWj.html WSJ article] emphasizes the importance of this type of learning for simple math skills.
 +
 +
More information and demonstrations of tutors in this project can be found at [http://optimallearning.org Lab Website]
  
 
=== Dependent variables ===
 
=== Dependent variables ===
  
Current performance -- These measures are usually taken as the student drills in the tutor.
+
[[Long-term retention]] -- These measures are usually taken in the tutor after at least one day of retention (much longer intervals occur in some of the most recent studies).
  
Long-term performance -- These measures are usually taken in the tutor after at least one day of retention (longer intervals occur in the most recent studies).
+
[[Transfer]] -- Many of the studies in this project will look at how learning in the tutor transfers to situations where that knowledge can be applied in a different configuration.
  
Transfer -- Many of the studies in this project will look at how learning in the tutor transfers to situations where that knowledge can be applied in a different configuration.
+
[[Accelerated future learning]] -- Some studies in this project will investigate the effect of tutor practice on the learning of items that depend upon the tutor practice.
  
Accelerated future learning -- Some studies in this project will investigate the effect of tutor practice on the learning of items that depend upon the tutor practice.
+
=== Independent variables ===
 +
Alternative structures of [[instructional schedule]] for [[practice]] based on the predictions of an ACT-R based cognitive model. Further independent variables include how the material is presented for [[learning events]] and the assumptions of the model used to compute the [[instructional schedule]]. The assumptions of the model include alternative analyses of [[task demands]], the structure of relevant [[knowledge components]], and learner [[individual differences]].
  
=== Independent variables ===
+
Example screen shot of instructional event presentation (Demo versions at [http://optimallearning.org/demos/ Demo Page]):
Alternative structures of [[instructional schedule]] for [[knowledge component training]] based on the predictions of an ACT-R based cognitive model. Further independent variables include how the material is presented for each learning event and the assumptions of the model used to presents schedule the learning. The assumptions of the model include alternative analyses of task demands, the structure of relevant knowledge components, and learner background.
+
[[Image:Examplescreen1FaCT.JPG]]
  
 
=== Hypothesis ===
 
=== Hypothesis ===
Robust learning occurs more quickly when practice is scheudled efficiently. In this case efficiently means according to a complex model of the gain and cost of possible scheudling decisions.
+
[[Robust learning]] occurs more quickly when [[practice]] is scheduled efficiently. In this case efficiently means according to a complex model of the [[robust learning]] gain and time cost of possible scheduling decisions. Given a single type of learning event, such schedules tend to have an [[expanding spacing interval]], since as [[practice]] accumulates knowledge components gain [[stability]]. See [[optimal scheduling]] for a discussion of learning principles and other examples.
 +
 
 +
=== Findings ===
 +
 
 +
This is a summary of the main findings for the various lines of research associated with this project. The following work has utilized the Java based [[FaCT System]] for trial based learning to deliver experiments. This system is described here: [http://optimallearning.org/ website].
 +
 
 +
* [[Applying optimal scheduling of practice in the Chinese Learnlab]] (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
 +
**This section discusses our efforts (a series of classroom studies) to show that the [[optimized scheduling]] provided by the [[FaCT System]] is better at producing robust learning than various [[Ecological control group|Ecological Control Group]]s. Initial results indicate that the system improves student performance for vocabulary quizzes, results in more practice by students and has better participation than control practice conditions.
 +
 
 +
* [[Understanding paired associate transfer effects based on shared stimulus components]] (Pavlik, MacWhinney, Bolster, Koedinger)
 +
**This study shows how a [[knowledge component]] analysis leads to predictions about [[transfer]] that are supported experimentally. After making a model of these effects, the results of this study will be applied in the classroom to improving the [[optimized scheduling]] algorithm. Three effects were found: Unit knowledge component learning - This hypothesis proposes that the stimulus items (sound file, Hanzi character, pinyin, or English) are learned as individual components somewhat independent of the pairings they occur in. Supports the notion of knowledge decomposition. Resonant learning - This hypothesis proposes that people spontaneously recall related knowledge components (spreading activation) when prompted to recall a specific pair. Further, this covert practice results in measurable learning. Stimulus mapping - This is the straightforward notion that learning of the connection between an orthography and a sound is advantaged because there are mapping rules (knowledge components) that allow this conversion.
 +
 
 +
* [[Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice]] (Pavlik)
 +
**This study used a complex design to see the effects of errors on learning. If errors should have an effect on learning it will require revisions of the model (i.e. if an error on practice at time t has an effect on practice at time t+1, then the model's accuracy will be increased if this is accounted for.)
 +
 
 +
* [[French gender cues]] (Presson-MacWhinney)
 +
**This project is part of Nora Presson's dissertation research and explores how to optimize practice for a skill that generalizes to multiple exemplars using the FaCT system.
 +
 
 +
* [[Providing optimal support for robust learning of syntactic constructions in ESL]] (Levin, Frishkoff, De Jong, Pavlik)
 +
**This project will use the FaCT system to explore a learning paradigm where multiple general factors compete to determine the response (whether to produce the NP PP or NP NP construction).
  
 
=== Explanation ===
 
=== Explanation ===
Line 52: Line 69:
 
   
 
   
 
When prior practice occurred
 
When prior practice occurred
*How many prior practices occurred
+
*How many prior [[learning events]] occurred
*Spacing between prior practice was
+
*[[Temporal spacing]] between prior [[learning events]] was
*Whether prior practice occurred as testing or passive study
+
*Whether prior [[learning events]] occurred as testing or passive study
*Duration of prior practices
+
*Duration of prior [[learning events]]
 
*An individuals history of success or failure with tests
 
*An individuals history of success or failure with tests
*What type of practice occurs (phonological, orthographic, English to Foreign or Foreign to English)  
+
*What type of practice occurs (phonological, orthographic, English to Foreign or Foreign to English, [[implicit instruction]], [[explicit instruction]]).
 
   
 
   
Optimal scheudules are mainly controlled by the benefit of wide spacing, which results in better long-term learning and the benefit of short spacing, which results faster learning.
+
Optimized scheduling is mainly controlled by the benefit of wide [[temporal spacing]], which results in better [[long-term retention]] and the benefit of short [[temporal spacing]], which reduce time cost.
  
=== Descendents ===
+
=== Descendants ===
 
 
* [[Understanding paired asociate transfer effects based on shared stimulus components]] (Pavlik, MacWhinney, Bolster, Koedinger)
 
  
 
* [[Applying optimal scheduling of practice in the Chinese Learnlab]] (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
 
* [[Applying optimal scheduling of practice in the Chinese Learnlab]] (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
 +
* [[Understanding paired associate transfer effects based on shared stimulus components]] (Pavlik, MacWhinney, Bolster, Koedinger)
 +
* [[Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice]] (Pavlik)
 +
* [[French gender cues]] (Presson-MacWhinney)
 +
* [[Providing optimal support for robust learning of syntactic constructions in ESL]] (Levin, Frishkoff, De Jong, Pavlik)
  
 
=== Annotated bibliography ===
 
=== Annotated bibliography ===
  
Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586 [http://optimallearning.org/people/Articles/2005 Pavlik Anderson.pdf (Article)].
+
*Atkinson, R. (1972) Optimizing the learning of a second language vocabulary. Journal of Experimental Psychology, 96, 124- 129.
 
+
*Pavlik Jr., P. I., Presson, N., Dozzi, G., Wu, S., MacWhinney, B. & Koedinger, K. (2007, accepted). The FaCT (Fact and Concept Training) System: A new tool linking cognitive science with educators. In D. McNamara & G. Trafton (Eds.), Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum. [http://www.learnlab.org/uploads/mypslc/publications/pavlik_1_31.pdf (Article)]
Pavlik Jr., P. I. (in press). Timing is an order: Modeling order effects in the learning of information. In F. E. Ritter, J. Nerb, T. O'Shea & E. Lehtinen (Eds.), In order to learn: How the sequences of topics affect learning. New York, NY: Oxford University Press.
+
*Pavlik Jr., P. I. (2006). Transfer effects in Chinese vocabulary learning. In R. Sun (Ed.), Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society (pp. 2579). Mahwah, NJ: Lawrence Erlbaum. [http://www.learnlab.org/uploads/mypslc/publications/pavlik-transfereffects.pdf (Article)]
 +
*Pavlik Jr., P. I. (in press-a). Timing is an order: Modeling order effects in the learning of information. In F. E., Ritter, J. Nerb, E. Lehtinen & T. O'Shea (Eds.), In order to learn: How order effects in machine learning illuminate human learning. New York: Oxford University Press.
 +
*Pavlik Jr., P. I. (in press-b). Understanding and applying the dynamics of test practice and study practice. Instructional Science.
 +
*Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586 [http://optimallearning.org/people/Articles/2005%20Pavlik%20Anderson.pdf (Article)]
 +
*Pavlik Jr., P. I., & Anderson, J. R. (2004,November). Optimizing Paired-Associate Learning. Poster presented at the 45th Annual Meeting of the Psychonomic Society, Minneapolis, MN.

Latest revision as of 22:30, 8 December 2008

Abstract

This project plan extends dissertation work of Pavlik. In this initial work, a model-based algorithm was described to maximize the rate of learning for simple facts using flashcard like practice by determining the best instructional schedule for a set of facts. The goal of this project plan is to develop this initial work to allow this tutor with optimized scheduling to handle more complex information and different types of learning in more natural settings (like LearnLabs). Specifically, this project plan describes extensions to the theory in two main areas.

1. Specification of a theory of refinement
a. Generalization practice (multimodal and bidirectional training)
b. Discrimination practice (detailed error remediation)
2. Specification of a theory of co-training
a. Effect of declarative memory chunk schedule of presentation during learning
b. Effect of declarative memory chunks on procedural learning

These theoretical directions are intended to enhance the FaCT System tutor by greatly extending its capabilities.

A secondary goal of the project is to link the optimization algorithm used in this project with the larger CTAT project. In this linkage the optimization algorithm would be integrated onto the current CTAT system as a curriculum management system that could select or generate problems according to the algorithm, but using CTAT interfaces. This integration will make it easier for people to use the optimized scheduling system and therefore increase its impact and usefulness.

Glossary

Research question

How can the optimal sequence of learning events be computed? The descendants section below links to LearnLab and laboratory research tracks that have employed and invetigated these methods of optimal sequencing.

Background and significance

Since the early 60's researchers in learning theory have been describing models of practice which attempt to capture the effect of practice on performance at a later time. These models are applicable to describing many types of learning situations, but are easier to apply where information to be learned can be broken up into small chunks that can be learned independently. For instance, Atkinson (1972) applied a Markov model of learning to schedule drill of German vocabulary.

More recently there has been a renewed emphasis on repeated practice. For instance, the National Council of Teachers of Mathematics new report WSJ article emphasizes the importance of this type of learning for simple math skills.

More information and demonstrations of tutors in this project can be found at Lab Website

Dependent variables

Long-term retention -- These measures are usually taken in the tutor after at least one day of retention (much longer intervals occur in some of the most recent studies).

Transfer -- Many of the studies in this project will look at how learning in the tutor transfers to situations where that knowledge can be applied in a different configuration.

Accelerated future learning -- Some studies in this project will investigate the effect of tutor practice on the learning of items that depend upon the tutor practice.

Independent variables

Alternative structures of instructional schedule for practice based on the predictions of an ACT-R based cognitive model. Further independent variables include how the material is presented for learning events and the assumptions of the model used to compute the instructional schedule. The assumptions of the model include alternative analyses of task demands, the structure of relevant knowledge components, and learner individual differences.

Example screen shot of instructional event presentation (Demo versions at Demo Page): Examplescreen1FaCT.JPG

Hypothesis

Robust learning occurs more quickly when practice is scheduled efficiently. In this case efficiently means according to a complex model of the robust learning gain and time cost of possible scheduling decisions. Given a single type of learning event, such schedules tend to have an expanding spacing interval, since as practice accumulates knowledge components gain stability. See optimal scheduling for a discussion of learning principles and other examples.

Findings

This is a summary of the main findings for the various lines of research associated with this project. The following work has utilized the Java based FaCT System for trial based learning to deliver experiments. This system is described here: website.

  • Understanding paired associate transfer effects based on shared stimulus components (Pavlik, MacWhinney, Bolster, Koedinger)
    • This study shows how a knowledge component analysis leads to predictions about transfer that are supported experimentally. After making a model of these effects, the results of this study will be applied in the classroom to improving the optimized scheduling algorithm. Three effects were found: Unit knowledge component learning - This hypothesis proposes that the stimulus items (sound file, Hanzi character, pinyin, or English) are learned as individual components somewhat independent of the pairings they occur in. Supports the notion of knowledge decomposition. Resonant learning - This hypothesis proposes that people spontaneously recall related knowledge components (spreading activation) when prompted to recall a specific pair. Further, this covert practice results in measurable learning. Stimulus mapping - This is the straightforward notion that learning of the connection between an orthography and a sound is advantaged because there are mapping rules (knowledge components) that allow this conversion.
  • French gender cues (Presson-MacWhinney)
    • This project is part of Nora Presson's dissertation research and explores how to optimize practice for a skill that generalizes to multiple exemplars using the FaCT system.

Explanation

The algorithm for scheduling practice uses a mathematical model of learning to predict when new practice should occur for recall to be optimal later. This model accounts for:

When prior practice occurred

Optimized scheduling is mainly controlled by the benefit of wide temporal spacing, which results in better long-term retention and the benefit of short temporal spacing, which reduce time cost.

Descendants

Annotated bibliography

  • Atkinson, R. (1972) Optimizing the learning of a second language vocabulary. Journal of Experimental Psychology, 96, 124- 129.
  • Pavlik Jr., P. I., Presson, N., Dozzi, G., Wu, S., MacWhinney, B. & Koedinger, K. (2007, accepted). The FaCT (Fact and Concept Training) System: A new tool linking cognitive science with educators. In D. McNamara & G. Trafton (Eds.), Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum. (Article)
  • Pavlik Jr., P. I. (2006). Transfer effects in Chinese vocabulary learning. In R. Sun (Ed.), Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society (pp. 2579). Mahwah, NJ: Lawrence Erlbaum. (Article)
  • Pavlik Jr., P. I. (in press-a). Timing is an order: Modeling order effects in the learning of information. In F. E., Ritter, J. Nerb, E. Lehtinen & T. O'Shea (Eds.), In order to learn: How order effects in machine learning illuminate human learning. New York: Oxford University Press.
  • Pavlik Jr., P. I. (in press-b). Understanding and applying the dynamics of test practice and study practice. Instructional Science.
  • Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586 (Article)
  • Pavlik Jr., P. I., & Anderson, J. R. (2004,November). Optimizing Paired-Associate Learning. Poster presented at the 45th Annual Meeting of the Psychonomic Society, Minneapolis, MN.