Pavlik and Koedinger - Generalizing the Assistance Formula

Generalizing the Assistance Formula across multiple dimensions of instructional assistance

Abstract

To foster more robust student learning, when should instruction provide information and assistance to students and when should it request students to generate information, ideas, and solutions? In different forms, this dilemma for instructors has been a part of debates on education since Plato. However, it is fair to say that we remain far from a precise and sound scientific response. We believe this “Assistance Dilemma” is one of the fundamental unsolved problems in the cognitive and learning sciences. To address this dilemma, we suggest a four step strategy for more clearly articulating the problem and tackling it with computational models that can be used to make precise, replicable, and testable predictions about when instructional assistance should be given vs. withheld. We illustrate these steps on two different dimensions of instructional assistance. On the “problem spacing” dimension, we present a computational model that generates precise predictions of the kind we call for. On the more complex “example-problem” dimension, we illustrate how the field is at a point where such a precise computational model may be possible.

Project Description

We will use DataShop log data to make progress on the Assistance Dilemma by targeting dimensions of assistance one at a time and creating parameterized mathematical models that predict the optimal level of assistance to enhance robust learning (cf., Koedinger et al., 2008). Such a mathematical model has been achieved for the practice-interval dimension (changing the amount of time between practice trials), and progress is being made on study-test dimension (changing the ratio of study trials to test trials) and the example-problem dimension (changing the ratio of examples to problems). These models generate the inverted-U shaped curve characteristic of the Assistance Dilemma as a function of particular parameter values that describe the instructional context. This function has a general form (L = [P*Sb+(1-P)Fb]/[P*Sc+(1-P)Fc]), which we call the “Assistance Formula”. We hypothesize that the Assistance Formula can be effectively instantiated for many other dimensions of assistance. These models address limitations of current instructional theory (e.g., Cognitive Load Theory) by generating a priori predictions of what forms of assistance or difficulty will enhance learning. Further, these models will provide the basis for on-line algorithms that adapt to individual student differences and changes over time, optimizing the assistance provided to each student for each knowledge component at each time in their learning trajectory. The benefits of such student-adapted optimization have already been demonstrated in PSLC projects on optimized practice scheduling (Pavlik & Anderson, 2008) and adaptive fading of worked examples (Salden, Renkl, Aleven et al. 2008). Similar efforts are needed for the many other dimensions of instructional assistance (e.g., study time, study-test, concrete-abstract, feedback timing, etc.).

Further Information

Annotated Bibliography

• Koedinger, K. R., Pavlik, P., McLaren, B. M., & Aleven, V. (2008). Is it Better to Give than to Receive? The assistance dilemma as a fundamental unsolved problem in the cognitive science of learning and instruction. In V. Sloutsky, B. Love & K. McRae (Eds.), Proceedings of the 30th Conference of the Cognitive Science Society. Washington, D.C.

References

• Anderson, J. R., Fincham, J. M., & Douglass, S. (1997). The role of examples and rules in the acquisition of a cognitive skill. Journal of Experimental Psychology: Learning, Memory, & Cognition, 23, 932-945.
• Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers.
• Atkinson, R. C., & Paulson, J. A. (1972). An approach to the psychology of instruction. Psychological Bulletin, 78, 49-61.
• Clark, R. C., & Mayer, R. E. (2003). Does Practice Make Perfect? In In e-Learning and the Science of Instruction (pp. 149-171). San Francisco: Pfeiffer.
• Collins, A., Brown, J. S., & Newman, S. E. (1990). Cognitive apprenticeship. In L. B. Resnick (Ed.), Knowing, learning and instruction. Hillsdale, NJ: Erlbaum.
• Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York: Irvington Publishers.
• Gilmore, D. J. (1996). The Relevance of HCI Guidelines for Educational Interfaces. Machine-Mediated Learning, 5, 119-133.
• Hausman, R., & VanLehn, K. (2007). Explaining Self-Explaining: A Contrast between Content and Generation. In Proceedings of the 13th international conference on artificial intelligence in education. (pp. 417-424).
• Jonassen, D. H. (1991). Objectivism versus Constructivism: Do We Need a New Philosophical Paradigm? Educational Technology, Research and Development, 39, 5-14.
• Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93, 579-588.
• Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41, 75-86.
• Koedinger, K. R., & Aleven, V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19, 239-264.
• Matsuda, N., Cohen, W., Sewall, J., Lacerda, G., & Koedinger, K. (2008). Why Tutored Problem Solving May be Better Than Example Study: Theoretical Implications from a Simulated-Student Study. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems (pp. 111-121).
• Mayer, R. E. (2004). Should There Be a Three-Strikes Rule Against Pure Discovery Learning? American Psychologist, 59, 14-19.
• McLaren, B., Lim, S.-J., & Koedinger, K. (2008). When Is Assistance Helpful to Learning? Results in Combining Worked Examples and Intelligent Tutoring. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems (pp. 677-680). Montreal.
• Newell, A. (1973). You can't play 20 questions with nature and win: Projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual Information Processing. New York: Academic Press.
• Paas, F. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84, 429-434.
• Paas, F. G., & Van Merrièenboer, J. J. G. (1994). Variability of worked examples and transfer of geometrical problem-solving skills: A cognitive-load approach. Journal of Educational Psychology, 86, 122-133.
• Pashler, H., Bain, P. M., Bottge, B. A., Graesser, A., Koedinger, K. R., McDaniel, M., et al. (2007). Organizing Instruction and Study to Improve Student Learning. (NCER 2007-2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.
• Pashler, H., Cepeda, N. J., Wixted, J. T., & Rohrer, D. (2005). When does feedback facilitate learning of words? Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 3-8.
• Pashler, H., Zarow, G., & Triplett, B. (2003). Is temporal spacing of tests helpful even when it inflates error rates? Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 1051-1057.
• Pavlik Jr., P. I. (2007). Understanding and applying the dynamics of test practice and study practice. Instructional Science, 35, 407-441.
• Pavlik Jr., P. I., & Anderson, J. R. (2005). Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cognitive Science, 29, 559-586.
• Pavlik Jr., P. I., & Anderson, J. R. (2008). Using a model to compute the optimal schedule of practice. Journal of Experimental Psychology: Applied, 14, 101-117.
• Pavlik Jr., P. I., Bolster, T., Wu, S., Koedinger, K. R., & MacWhinney, B. (2008). Using Optimally Selected Drill Practice to Train Basic Facts. In B. Woolf, E. Aimer & R. Nkambou (Eds.), Proceedings of the 9th International Conference on Intelligent Tutoring Systems. Montreal, Canada.
• Ringenberg, M., & VanLehn, K. (2006). Scaffolding Problem Solving with Annotated, Worked-Out Examples to Promote Deep Learning. In Proceedings of the 8th International Conference on Intelligent Tutoring Systems (pp. 625-634).
• Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1, 181-210.
• Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.
• Schwonke, R., Wittwer, J., Aleven, V., Salden, R. J. C. M., Krieg, C., & Renkl, A. (2007). Can tutored problem solving benefit from faded worked-out examples? In Proceedings of the 2nd European Cognitive Science Conference (pp. 59-64).
• Stark, R., Gruber, H., Renkl, A., & Mandl, H. (2000). Instruktionale Effekte einer kombinierten Lernmethode: Zahlt sich die Kombination von Lösungsbeispielen und Problemlöseaufgaben aus? (Does the combination of worked-out examples and problem-solving tasks pay off?). Zeitschrift für Pädagogische Psychologie, 14, 206-218.
• Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59-89.
• Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251-296.
• Trafton, J. G., & Reiser, B. J. (1993). The contributions of studying examples and solving problems to skill acquisition. Paper presented at the Fifteenth Annual Conference of the Cognitive Science Society.
• Vygotsky, L. S. (1978). Mind in society. Cambridge: Harvard University Press.

Future Plans

Year 6 Project Deliverable: Journal publication on the Assistance Dilemma. 6th Month Milestone: Submission of journal article on the Assistance Dilemma.