Optimizing the practice schedule

Abstract

This project plan extends dissertation work of Pavlik. In this initial work, a model-based algorithm was described to maximize the rate of learning for simple facts using flashcard like practice by determining the best instructional schedule for a set of facts. The goal of this project plan is to develop this initial work to allow this tutor with optimized scheduling to handle more complex information and different types of learning in more natural settings (like LearnLabs). Specifically, this project plan describes extensions to the theory in two main areas.

1. Specification of a theory of refinement

a. Generalization practice (multimodal and bidirectional training)

b. Discrimination practice (detailed error remediation)

2. Specification of a theory of co-training

a. Effect of declarative memory chunk schedule of presentation during learning

b. Effect of declarative memory chunks on procedural learning

These theoretical directions are intended to enhance the FaCT System tutor by greatly extending its capabilities.

A secondary goal of the project is to link the optimization algorithm used in this project with the larger CTAT project. In this linkage the optimization algorithm would be integrated onto the current CTAT system as a curriculum management system that could select or generate problems according to the algorithm, but using CTAT interfaces. This integration will make it easier for people to use the optimized scheduling system and therefore increase its impact and usefulness.

Glossary

• Optimal spacing interval
• Expanding spacing interval
• Optimized scheduling

Research question

How can the optimal sequence of learning events be computed? The descendants section below links to LearnLab and laboratory research tracks that have employed and invetigated these methods of optimal sequencing.

Background and significance

Since the early 60's researchers in learning theory have been describing models of practice which attempt to capture the effect of practice on performance at a later time. These models are applicable to describing many types of learning situations, but are easier to apply where information to be learned can be broken up into small chunks that can be learned independently. For instance, Atkinson (1972) applied a Markov model of learning to schedule drill of German vocabulary.

More recently there has been a renewed emphasis on repeated practice. For instance, the National Council of Teachers of Mathematics new report WSJ article emphasizes the importance of this type of learning for simple math skills.

More information and demonstrations of tutors in this project can be found at Lab Website

Dependent variables

Long-term retention -- These measures are usually taken in the tutor after at least one day of retention (much longer intervals occur in some of the most recent studies).

Transfer -- Many of the studies in this project will look at how learning in the tutor transfers to situations where that knowledge can be applied in a different configuration.

Accelerated future learning -- Some studies in this project will investigate the effect of tutor practice on the learning of items that depend upon the tutor practice.

Independent variables

Alternative structures of instructional schedule for practice based on the predictions of an ACT-R based cognitive model. Further independent variables include how the material is presented for learning events and the assumptions of the model used to compute the instructional schedule. The assumptions of the model include alternative analyses of task demands, the structure of relevant knowledge components, and learner individual differences.

Hypothesis

Robust learning occurs more quickly when practice is scheduled efficiently. In this case efficiently means according to a complex model of the robust learning gain and time cost of possible scheduling decisions. Given a single type of learning event, such schedules tend to have an expanding spacing interval, since as practice accumulates knowledge components gain stability.

Findings

This is a summary of the main findings for the various lines of research associated with this project. The following work has utilized the Java based FaCT System for trial based learning to deliver experiments. This system is described here: website.

• Applying optimal scheduling of practice in the Chinese Learnlab (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
  • This section discusses our efforts (a series of classroom studies) to show that the optimized scheduling provided by the FaCT system is better at producing robust learning than various Ecological Control Groups. Initial results indicate that the system improves student performance for vocabulary quizzes, results in more practice by students and has better participation than control practice conditions.

• Understanding paired associate transfer effects based on shared stimulus components (Pavlik, MacWhinney, Bolster, Koedinger)
  • This study shows how a knowledge component analysis leads to predictions about transfer that are supported experimentally. After making a model of these effects, the results of this study will be applied in the classroom to improving the optimized scheduling algorithm. Three effects were found: Unit knowledge component learning - This hypothesis proposes that the stimulus items (sound file, Hanzi character, pinyin, or English) are learned as individual components somewhat independent of the pairings they occur in. Supports the notion of knowledge decomposition. Resonant learning - This hypothesis proposes that people spontaneously recall related knowledge components (spreading activation) when prompted to recall a specific pair. Further, this covert practice results in measurable learning. Stimulus mapping - This is the straightforward notion that learning of the connection between an orthography and a sound is advantaged because there are mapping rules (knowledge components) that allow this conversion.

• Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice (Pavlik)
  • This study just began collecting data for a complex design to see the effects of errors on learning. If errors should have an effect on learning it will require revisions of the model (i.e. if an error on practice at time t has an effect on practice at time t+1, then the model's accuracy will be increased if this is accounted for.)

• French gender cues (Presson-MacWhinney)
  • This project is part of Nora Presson's dissertation research and explores how to optimize practice for a skill that generalizes to multiple exemplars using the FaCT system.

• Providing optimal support for robust learning of syntactic constructions in ESL (Levin, Frishkoff, De Jong, Pavlik)
  • This project will use the FaCT system to explore a learning paradigm where multiple general factors compete to determine the response (whether to produce the NP PP or NP NP construction).

Explanation

The algorithm for scheduling practice uses a mathematical model of learning to predict when new practice should occur for recall to be optimal later. This model accounts for:

When prior practice occurred

• How many prior learning events occurred
• Temporal spacing between prior learning events was
• Whether prior learning events occurred as testing or passive study
• Duration of prior learning events
• An individuals history of success or failure with tests
• What type of practice occurs (phonological, orthographic, English to Foreign or Foreign to English, implicit instruction, explicit instruction).

Optimized scheduling is mainly controlled by the benefit of wide temporal spacing, which results in better long-term retention and the benefit of short temporal spacing, which reduce time cost.

Descendants

• Applying optimal scheduling of practice in the Chinese Learnlab (Pavlik, MacWhinney, Sue-mei Wu, Koedinger)
• Understanding paired associate transfer effects based on shared stimulus components (Pavlik, MacWhinney, Bolster, Koedinger)
• Understanding encoding inhibition, retrieval inhibition and destructive interference effects of errors during practice (Pavlik)
• French gender cues (Presson-MacWhinney)
• Providing optimal support for robust learning of syntactic constructions in ESL (Levin, Frishkoff, De Jong, Pavlik)

Annotated bibliography

• Atkinson, R. (1972) Optimizing the learning of a second language vocabulary. Journal of Experimental Psychology, 96, 124- 129.

Pavlik Jr., P. I., Presson, N., Dozzi, G., Wu, S., MacWhinney, B. & Koedinger, K. (2007, accepted). The FaCT (Fact and Concept Training) System: A new tool linking cognitive science with educators. In D. McNamara & G. Trafton (Eds.), Proceedings of the Twenty-Ninth Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum. (Article)

Pavlik Jr., P. I. (2006). Transfer effects in Chinese vocabulary learning. In R. Sun (Ed.), Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society (pp. 2579). Mahwah, NJ: Lawrence Erlbaum.

Pavlik Jr., P. I. (in press-a). Timing is an order: Modeling order effects in the learning of information. In F. E., Ritter, J. Nerb, E. Lehtinen & T. O'Shea (Eds.), In order to learn: How order effects in machine learning illuminate human learning. New York: Oxford University Press.

Pavlik Jr., P. I. (in press-b). Understanding and applying the dynamics of test practice and study practice. Instructional Science.

Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and Forgetting Effects on Vocabulary Memory: An Activation-Based Model of the Spacing Effect. Cognitive Science, 29, 559-586 (Article)

Pavlik Jr., P. I., & Anderson, J. R. (2004,November). Optimizing Paired-Associate Learning. Poster presented at the 45th Annual Meeting of the Psychonomic Society, Minneapolis, MN.