Craig questions

Investigating the robustness of vicarious learning: Sense Making with Deep-level reasoning questions

Scotty Craig, Kurt VanLehn, and Micki Chi

Abstract

Earlier work (Craig, at al. 2006; Gholson & Craig, in press) found that inserting relevant deep-level questions into observed material both increased deep level question asking and improved learning. These lab studies had student learn topics in computer literacy by viewing videos of various kinds; some material included deep questions, some included shallow questions and some included no questions. The conditions that include deep questions learned more than the others. However, it is not known how this method works compared to other methods for enhancing learning from observed materials (e.g. prompting for self-explanation). It is also not known if this effect can be useful for learning outside the lab setting.

Our in vivo experiment presented identical core content on magnetism using examples problems from the Andes tutoring system in three different ways. The material was presented as a worked example. The content was divided into knowledge components. The knowledge components were preceded by a deep-level question (e.g. What are the implications of having the magnetic field close to an electrified wire?), a prompt for learners to reflection on the material (i.e. a pause in the video) or a self-explanation prompt (e.g. Please begin your self explanation). Measures of Andes transfer, and long term robust learning were be measured. The learners’ interaction with Andes will be observed for differences on completion time, within task behavior, and the completion rates of the Andes homework.

Glossary

See Craig deep-level questions Glossary

• Vicarious learning - a form of learning that occurs when learners see and/or hear content from a learning situation for which they are not the addressees and have no way of physically interacting with the source of the content they are attempting to master
• Deep-level reasoning question( deep-level question - A question that draws links between mechanisms, components or processes.

Research question

Is robust learning better achieved by observing multimedia displays intigrated with deep-level reasoning questions, prompts for reflection, or self-explanation?

Independent variables

The current study varied the level of guidance provided. The level of guidance was varied by presenting students with a deep-level reasoning condition, a self-explanation condition and a reflection condition. The deep-level reasoning questions provided a step-by-step guide that scaffolded the learner during the learning process. The self-explanation condition asked that students build the links of these scaffolds by self-explaining the steps. The reflection condition presented the participants with the steps and asked them to reflect on the material as it was presented.

Hypothesis

A guided learning hypothesis would predict that since the deep-level questions provided a constant cognitive guide the deep-level question condition would improve learning over the reflection condition and possibly the self-explanation condition if the students could not produce the guidance while producing the self-explanations. Alternatively, a content equivalency hypothesis would be that since all three conditions provide the same content they should all produce learning of the material (Klahr & Nigam, 2004).

Dependent variables

• Homework: After training, students did their regular homework problems using Andes. Students can do them whenever they want, but most normally complete them just before the exam. Homework for similar problems (near transfer) was analyzed.

Results

Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a marginally significant trend found on the completion time data in favor of participants in the deep-level question condition over those in the reflection condition (t (9) = 2.14, p = .07). This difference for completion time became significant when participants in the two unguided conditions were collapsed and compared against participants in the guided condition (t (15) = 2.41, p < .05). This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition.

Explanation

This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met:

• The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the deep-level question condition and the self-explanation condition could comprise the “learning-by-doing paths” in that learners are guided to produce clearer mental models of the material. Alternatively the participants in the reflection condition only received pauses during the presentation, thus these participants were not guided to produce better mental models. These participants relied more on the video to provide relevant links for them instead of actively constructing these links.
• The student should take the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with deep-level questions that guide them in building better mental models. However, the self-explanation and reflection conditions require the students to produce the learning by doing path. In these conditions, if the production becomes too difficult for the students then they will not learn. This study is testing whether students will learn more by being encouraged to take a "learning by doing" path, via deep-level questions, than an alternative path. Since none of the students attempted more than a few self-explanations, it appears that the students in the self-explanation conditions took this path.

Annotated bibliography

• Presented at LRDC Supergroup meeting July, 2006
• Presented at PSLC Roadshow - Memphis November, 2006

References

• Chi, M. T. H., Roy, M., & Hausmann, R. G. M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. Cognitive Science.
• Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
• Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477.
• Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. Journal of Educational Multimedia and Hypermedia, 13, 163-183. [1]
• Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. Cognition and Instruction, 24(4), 565-591.
• Gholson, B. & Craig, S. D. (in press/2006). Promoting constructive activities that support vicarious learning during computer-based instruction. Educational Psychology Review, 18, 1XX-1XX. [2]
• Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. Psychological Science, 15, 661-667.

--Scotty 12:12, 19 September 2006 (EDT)