McLaren - The Assistance Dilemma And Discovery Learning

The Assistance Dilemma and Discovery Learning

Bruce M. McLaren

Overview

PI: Bruce M. McLaren, Carnegie Mellon University, Pittsburgh

Others who have contributed 160 hours or more:

• Alex Borek, University of Karlsruhe, Germany, research, programming, statistical analysis
• Dave Yaron, Carnegie Mellon University, Chemistry domain expertise, Support of classroom study
• Mike Karabinos, Carnegie Mellon University, Chemistry domain expertise, Support of classroom study

Abstract

How much help helps in discovery learning? This question is one instance of the assistance dilemma, an important issue in the learning sciences and educational technology research. To explore this question, we conducted a study involving 87 college students solving problems in a virtual chemistry laboratory (VLab), testing three points along an assistance continuum: (1) a minimal assistance, inquiry-learning approach, in which students used the VLab with no hints and minimal feedback; (2) a mid-level assistance, tutored approach, in which students received intelligent tutoring hints and feedback while using the VLab (i.e., help given on request and feedback on incorrect steps); and (3) a high assistance, direct-instruction approach, in which students were coaxed to follow a specific set of steps in the VLab. Although there was no difference in learning results between conditions on near transfer posttest questions, students in the tutored condition did significantly better on conceptual posttest questions than students in the other two conditions. Furthermore, the more advanced students in the tutored condition, those who performed better on a pretest, did significantly better on the conceptual posttest than their counterparts in the other two conditions. Thus, it appears that students in the tutored condition had just the right amount of assistance, and that the better students in that condition used their superior metacognitive skills and/or motivation to decide when to use the available assistance to their best advantage.

Glossary

• Assistance dilemma

Research Questions

How much help helps in discovery learning?

Background and Significance

A key goal of educational technology research is to find the right level of support to imbue in computer-based educational systems. The so-called assistance dilemma is central to this goal: “How should learning environments balance assistance giving and withholding to achieve optimal student learning?” (Koedinger & Aleven, 2007). Assistance giving allows students to move forward when they are struggling and truly need help, yet can rob them of the motivation to learn on their own. On the other hand, assistance withholding encourages students to think and learn for themselves, yet can cause frustration when they are unsure of what to do next.

Although the “assistance dilemma” is a relatively new term, it describes a central issue in the learning sciences that has been debated for some time. The extreme position of assistance giving is usually called direct-instruction or guided learning. Supporters of this position (e.g. Kirschner, Sweller, & Clark, 2006, Klahr & Nigam, 2004, Mayer, 2004) argue that higher assistance (direct instruction and/or tutoring of basic skills) leads to better learning results because it provides information that students cannot create on their own. Supporters of the opposing position (e.g. Bruner, 1961, Steffe & Gale, 1995) advocate a much lower assistance approach (i.e.,assistance withholding), often called discovery or inquiry learning.

Independent Variables

The study compared three conditions in which students used different versions of the VLab to solve problems in thermo chemistry:

• (Condition 1) The Inquiry-learning Condition, in which students worked with a version of VLab with no hints and minimal feedback,
• (Condition 2) The Tutored Condition, in which students could request hints and received feedback only when they were severely off track, and
• (Condition 3) The Direct-instruction Condition, in which students were directed to follow a prescribed problem-solving path.

Dependent Variables

Our plan is to include the following robust learning dependent variables in our studies.

• Normal post-test: Students will take an immediate post-test, right after completing work with the stoichiometry tutor
• Transfer: Conceptual, transfer questions will be included in the post-tests
• Long-term retention: Students will take a second post-test, including conceptual, transfer questions, 7 days after the initial post-test

Findings

We first scored and ran an ANOVA on students’ pretests, to assure equality between conditions, with conditions as a between-subjects factor. Tasks had only one acceptable solution and were graded by a program. As there was no significant difference in the pretest between the three conditions, F(2,77)=0.292, p=.748, we assume that students in the three conditions started with a similar level of knowledge.

Next, we evaluated the posttest scores. Tasks in the near-transfer part of the posttest also had only one acceptable solution and were scored by a program. Three reviewers graded the conceptual-understanding tasks of the posttest, answered in free-form text, using the same rubric to ensure objectivity. In approximately 90% of cases there was agreement by at least two graders, in the other 10% the average of all three grades was taken. We removed seven outliers from the population – students who scored less than a quarter of the maximal reachable points in the posttest. The means of the overall posttest scores, as well as the means of the individual components of the posttest (i.e., the near-transfer scores and conceptual-understanding scores), are shown below.

We then ran ANCOVAs on the posttest scores, using the pretest scores as the covariate, to evaluate differences in the posttest scores between the conditions. Although the mean scores were higher in the Tutored Condition for both the overall score and the near-transfer score, the differences were not significant, F(2,77)=2.035, p=.138; F(2,77)=0.057, p=.944. However, we did find a significant result on the conceptual-understanding part of the posttest: Students in the Tutored Condition did better on conceptual-understanding tasks than students in the other two conditions, F(2,77)=3.783, p=.007. These results support our hypothesis: Students in the Tutored Condition – the mid-level assistance approach – showed better learning results than students in the other two conditions.


Finally, we segmented students into strong (best 50%) and weak (worst 50%) ups based on their pretest scores. In another ANCOVA, again using pretest scores as the covariate, students in the Tutored Condition who did better on the pretest benefitted more regarding conceptual understanding than students in the other conditions, F(2,37)=4.699, p=.015. Weaker students in the Tutored Condition also did better on the conceptual-understanding part than weaker students in the other conditions, but not significantly, F(2,37)=1.193, p=.315.

Explanation

This study is part of the Cognitive Factors thrust.

Annotated Bibliography

• Borek, A., McLaren, B.M., Karabinos, M., & Yaron, D. (2009). How Much Assistance is Helpful to Students in Discovery Learning? In U. Cress, V. Dimitrova, & M. Specht (Eds.), Proceedings of the Fourth European Conference on Technology Enhanced Learning, Learning in the Synergy of Multiple Disciplines (EC-TEL 2009), LNCS 5794, September/October 2009, Nice, France. (pp. 391-404). Springer-Verlag Berlin Heidelberg.

References

• Kirschner, P.A., Sweller, J., & Clark, R.E. (2006). Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psychologist, 75—86.
• Klahr, D. & Nigam, M. (2004). The Equivalence of Learning Paths in Early Science Instruction - Effects of Direct Instruction and Discovery Learning. Psychological Science, 661—667.
• Koedinger, K.R. & Aleven, V. (2007). Exploring the Assistance Dilemma in Experiments with Cognitive Tutors. Educational Psychology Review 19, 239—264.
• Mayer, R.E. (2004). Should There Be a Three-Strikes Rule Against Pure Discovery Learning? - The Case for Guided Methods of Instruction. American Psychologist, 14—19.
• Bruner, J.S. (1961). The Art of Discovery. Harvard Educational Review (31), 21—32.
• Steffe, L. & Gale, J. (1995). Constructivism in Education. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.