Ringenberg Examples-as-Help

Scaffolding Problem Solving with Embedded Examples to Promote Deep Learning

Michael Ringenberg and Kurt VanLehn

Abstract

This in vivo experiment which occurred in the Physics LearnLab compared the relative utility of an intelligent tutoring system that used procedure-based hints to a version that used worked-out examples for learning college level physics. In order to test which strategy produced better gains in competence, two version of Andes were used: one offered participants graded hints and the other offered annotated, worked-out examples in response to their help requests. We found that providing examples was at least as effective as the hint sequences and was more efficient in therms of the number of problems it took to obtain the same level of mastery.

Background and Significance

In this study, we were investigating different types of content an intelligent tutoring system agent could give a student. The system used for this investigation was the highly successful Andes system that uses targeted hint sequences when students ask it for help. If students find these hints confusing because they lack the proper knowledge components, then they have been observed to engage in help abuse to construct a worked-out example. Noting that examples can be effective aids in instructional material, we wished to see if they could be effectively used during problem solving in order to foster the learning of knowledge components. We suspect that examples would be effective because they are content rich and can be used effectively by varying competence levels and therefor more likely to hit the students Zone of Proximal Development than targeted hints.

Glossary

Annotated, Worked-out Example

An problem statement or description and the steps necessary to solve it as demonstrated by an expert where each step is annotated with the name of the principle used to generate it.

Problem matching task

Participants are presented with a model problem statement and then asked which of two alternate problem statements would be solved most similarly to the model problem. Only one of the alternate problems will use the same knowledge components and be considered the match. Each of the alternate problems could match some of the surface features of the model problem.

Research question

Will robust learning be fostered if students are presented with relevant, annotated, worked-out examples instead of a targeted hint message when a learning event is encountered?

Independent variables

The manipulation in this study was based on the feedback the Andes system provided while participants worked on assigned homework problems covering Inductors:

Examples Condition

Participants were presented a relevant, annotated, worked-out example when help was requested.

Hints Condition

Participants were presented a targeted hint when help was requested.

Hypothesis

Providing annotated, worked-out examples instead of hints during problem solving will promote the learning of knowledge components and help appropriately generalize the knowledge components.

Dependent variables & Results

Near Transfer, retention

Performance on problems involving inductors on the normal mid-term exam that were similar to the training problems. There was not significant difference in performance between the two conditions. Both conditions did better than a baseline of participants who solved no homework problems.

Transfer task, deep structure assessment

Problem matching task: No significant difference in performance between the two conditions; however, participants in the examples condition solved fewer training problems. Both conditions did better than a baseline of participants who solved no homework problems.

Homework

Number of problems completed: Participants in the examples condition solved significantly fewer problems than participants in the hints condition.

Time on task: Participants in the examples condition spent less time solving problems than those in the hints condition. Participants in both conditions spent about the same amount of time per problem.

Annotated bibliography

• Ringenberg, Michael A. & VanLehn, Kurt (2006). Scaffolding Problem Solving with Annotated, Worked-Out Examples to Promote Deep Learning. Paper presented at the ITS 2006, Taiwan. Winner of Best Paper First Authored by a Student Award.