Long-term retention
Long-term retention involves the recall and use of knowledge after a relatively long period of time has passed since instruction on that knowledge. Long-term retention is one of the three measures of robust learning.
In PSLC studies, a post-test given immediately after the instruction is not a long-term retention test. As a weak heuristic, a long-term retention interval (the time between the end of instruction and the test) should be as least as long as the time of the instruction (the time between the beginning and end of the instructional period of the study). Difference between control and treatment instruction tend to be harder to detect at longer retention intervals but the longer the interval at which a difference is detected the greater the evidence of the treatment leading to long-term retention.
The justification for using retention tests is both empirical and theoretical, but comes down to both data and theory supporting the claim that test results can sometimes be different at retention than at immediate "normal post-test". Empirically, one example within PSLC is Aaron Bauer's first study of on-line note-taking: He found that when taking on-line notes, those students given cut-and-paste did just as well on a normal, immediate post-test as those with just a type-in editor, but did worse at a retention test.
Other empirical justification includes work by Schmidt & Bjork on better retention but sometimes worse normal post-test performance from delayed vs. immediate feedback in motor learning tasks. Their theoretical explanation involves the "guidance hypothesis", that suggests that if one learns both "evaluative" as well as "generative" knowledge, one is better prepared for long-term retention. See:
Schmidt, R.A., Bjork, R.A. (1992). New conceptualizations of practice: common principles in three paradigms suggest new concepts for training. Psychological Science, 3 (4), 207-217.
Perhaps, the clearest theoretical (and empirical) justification for long-term retention tests is the "spacing effect" and associated theory (e.g., like that in ACT-R). ACT-R theoretically predicts (with formulas) circumstances under which spaced practice yields better long-term retention but worse immediate (normal test) performance. This is basically the strengthening argument pathway in the PSLC framework (see talk slides).
Another theoretical line of argumentation in the PSLC framework is that some instruction, perhaps self-explanation (or 'deep' note taking, not simple copying), may put students in a better position to re-derive knowledge at a retention test. My work is needed here both in expanding empirical results, but especially in providing a predictive theory (e.g., with formulas) and that can be applied a priori to aid instructional design prior to experimentation.
The guidance hypothesis may be one productive route to pursue in this regard. PSLC member Julie Booth is pursuing a related idea that having students self-explain given incorrect solutions (essentially they are engaging in Schmidt & Bjork's evaluative process) may improve robust learning. Another related study found better robust learning (long-term retention and transfer) as well as better normal post-test performance from tutoring toward an "intelligent novice" model of desired performance rather than an experiment model:
Mathan, S. A. & Koedinger, K. R. (2005) Fostering the Intelligent Novice: Learning from errors with metacognitive tutoring. Educational Psychologist. 40(4), 257-265.
Another line of work relevant to long-term retention is the following:
Fong, G. T., & Nisbett, R. E. (1991). Immediate and delayed transfer of training effects in statistical reasoning. Journal of Experimental Psychology: General, 120, 34-45.
PSLC member Tim Nokes is further pursing this line of research and particularly how instruction that upports integrated or coordinated processing of examples and rules. See also Coordinative Learning.
