Worked example principle

Brief statement of principle

In contrast to the traditional approach of giving a list homework (or seatwork) problems for students to solve, students learn more efficiently and more robustly when more frequent study of worked examples is interleaved with problem solving practice.

Description of principle

"In courses that are teaching new tasks, learning time can be saved by replacing some practice problems with worked examples" (Clark & Mayer, 2004, p. 177). In addition, most studies comparing interleaved worked examples and problems with all problems have also shown improved learning outcomes, including robust learning outcomes.

"It would be an unusual (not to mention incompetent) teacher who did not use worked examples.  Similarly, textbooks universally use worked examples to illustrate new concepts.  The suggestion being made here goes beyond this limited use of worked examples.  Rather than using them merely to demonstrate how to use a mathematical or scientific rule, the proposal is that they should be used in large numbers as a form of practice.  In other words, instead of practicing by solving many problems (an activity engaged in by most conscientious students), it is proposed that many of these problems could profitably be replaced by worked examples." Sweller, J. (1999) p73

Examples

Imagine instead of giving students a typical homework or seatwork assignment involving 8 problems, you give them an assignment where every other problem comes with a complete worked out solution. The even numbered items would be usual problems, like the following algebra problem:

Solve 12 + 2x = 15 for x

The odd numbered problems, come with solutions, like this:

Solve 12 + 2x = 15 for x
Study each step in this solution, so that you can better solve the next problem on your own:
12+2x = 15
   2x = 15-12
   2x = 3
    x = 3/2
    x = 1.5

Which approach, asking for solutions to all 8 problems or interleaving 4 examples with 4 problems, will lead to better student learning? You might think that the 8 problems require more work or that students might ignore the examples and thus, the 8 problems would lead to more learning. But, much research has shown that students typically learn more deeply and more easily from the second approach, when examples are interleaved between problems.

Teachers often think so many examples “give it away” or that students will not pay attention to the example. But, by having problems in between students are motivated to pay more attention to the example so as to prepare for the next problem or to resolve a question from the past problem. The problems break a students’ “illusion of knowing” that might otherwise lead them to skim the example and believe it is obvious.

It is important that students spend time actively engaged in learning and in genuine problem solving and reasoning. However, an emphasis on “learn by doing” is sometimes taken too far and students end up with homework problems or projects that are beyond their means. In such cases, they may spend much unproductive study time struggling without success. This time is often not only wasted but may increase a students’ frustration with the subject-matter and lead to unjustified feelings of not being good at math or science particularly. In contrast, during example study, students can focus their attention on understanding the principles underlying the examples instead of simply on finishing the problem. In early learning, the thought that goes simply into trying to solve the problem seems to distract students from trying to understand the principles underlying the solution.

Notice that in the example above, explanations for each step are not provided. It is best when students provide these explanations themselves (see the prompted self-explanation hypothesis) and, while more research is needed, providing explanations can sometimes distract students from doing so themselves and in other cases seems to provide no additional enhancement in student learning.

In whole classroom situation a teacher might implement this principle by going back and forth between a classroom or small group discussion around an example solution followed by small groups or individuals solving a problem (just one!) on their own. Then back to example study, for instance, by having students present their solutions and having others attempt to explain the steps (see the prompted self-explanation hypothesis). Now back to a second problem.

By giving the students frequent opportunities to study examples in between problem solving, students can more easily and more deeply acquire the big ideas, key concepts, or key principles that we want them to learn. With greater understanding, students will do better on harder problems in the future that require them to transfer these key concepts beyond the problems just like those they have seen before.

Experimental support

As summarized in Clark & Mayer, 2003 (pp 179): "There is a lot of evidence for the effectiveness of learning from worked examples.  As an example, in one study twelve [statistics] problems were used.  In the conventional group the learners solved all twelve problems as practice.  In the worked examples group, the learners received eight problems already worked out to study and then four problems to solve as practice.  Students in the worked examples group spent significantly less time studying and scored higher on a test than did those in the conventional group.  Furthermore, the worked examples group scored higher not only on test problems similar to those used during practice but also on different types of problems requiring application of the principles taught (Paas, 1992).  The investigators conclude that 'training with partly or completely worked-out problems leads to less effort-demanding and better transfer performance and is more time efficient' (p. 433).  In fact, in one study, the use of worked examples allowed learners to complete a three-year mathematics course in two years (Zhu and Simon, 1987).  Positive effects of worked examples have been reported in a variety of courses teaching well-defined problems, including algebra, geometry, statistics, and programming". 

Laboratory experiment support

See papers Cooper & Sweller and many others, such as Atkinson, Ayres, Catrambone, Paas, Renkl, van Gog, van Merrienboer ... See the recommendation "Repeatedly alternating problems with their solutions provided and problems that students must solve" on this web page which is a web-based update of this IES practice guide: Organizing Instruction and Study to Improve Student Learning. Some relevant references can be found within this page.

In vivo experiment support

McLaren's three stoichiometry studies provide mixed support of the worked example principle. Although students did not learn more through the study of worked examples followed by problem solving, as in (Paas, 1992; Zhu and Simon, 1987; Trafton & Reiser, 1993), they did learn more efficiently as in the earlier studies. On the other hand, only normal pre-post gains were evaluated in the stoichiometry studies; robust learning was not measured. In addition, the control condition of these three studies was different -- and potentially much more rigorous -- than the earlier studies: students solved problems with the support of an intelligent tutor. This may explain why students did not learn more: perhaps the additional support of the tutor -- in which students theoretically could create their own "worked examples" by clicking through to bottom out hints -- equalizes the advantage of learning from the examples.

Salden, Renkl, Schwonke and Aleven (2008) studied the use of 'faded' examples as an adjunct to tutored problem solving with the Geometry Cognitive Tutor. Following Renkl and Atkinson's (2003) example fading methods, the example-enhanced version of the tutor, after first presenting fully-worked-out examples, gradually reduced the number of solution steps given, thus increasing the number of open steps that students had to solve. Salden et al. found that the faded examples help students learn more efficiently and effectively (e.g., Schwonke, Renkl, Krieg, Wittwer, Aleven, & Salden, in press) especially when the examples are faded in an adaptive manner, responsive to the students' explanations of worked-out steps.

Paas (1992): Across 12 tasks, interleaving 2 worked examples and 1 problem to solve leads to:

• shorter learning time-on-task than solving all active problem solving (computing mean, median, mode)
• similar time on task and accuracy for solving the 4 active problem solving items
• better near and far transfer for worked examples vs. active problem solving
• greater perceived mental effort for active problem solving than worked examples

Theoretical rationale

The original rational for the worked example effect came from Sweller's Cognitive Load Theory (Sweller, 1988; Sweller & Cooper, 1985):

"Working memory has a limited capacity that becomes inefficient when having to retain even a few items. If the only way to build job-relevant skills is to perform many practice exercises, working memory can become overloaded by the mental work required to complete these exercises. However, if limited working memory resources could be used to study worked examples and build new knowledge from them, some of this labor-intensive effort could be bypassed. Worked examples are more efficient for learning new tasks because they reduce the load in working memory, thereby allowing the learner to learn the steps in problem solving. Sweller and his colleagues distinguished between the intrinsic load of instructional materials that result from the inherent complexity of the content itself and the extraneous load imposed by the instructional design (Sweller, 1999; Sweller, Van Merrienboer and Paas, 1998). Learners who are studying complex topics will have to deal with high intrinsic mental load, especially if it's new information. However, good e-learning can help learners manage that lead by using effective instructional methods. Replacing some assigned problems with worked examples reduces the extraneous load, freeing working memory to allocate resources to the learning process. This recommendation applies primarily to courses for novice learners who are most susceptible to cognitive overload". (Clark & Mayer, 2003, pp. 178-179)

Another line of rationale suggests that worked examples make students engage in more self-explanation than they do during problem solving.

One (of perhaps many) open questions is what motivates students to process examples more deeply, that is, to engage in "generative processing" (Mayer) or "germane load" (Van Merrienboer and Paas?). The importance of interleaving examples and problems may be primarily about motivating students to deeply process the examples. Such an explanation is different from the "knowledge compilation" explanation for interleaving articulated by Trafton & Reiser (1993).

A related line of reasoning suggests that example study better engages explicit learning (based on verbal rules or principles communicated in instruction) than does problem solving practice. By engaging in explicit reasoning about the domain rules or principles, students are more likely to discriminate relevant from irrelevant features of those rules, that is, more likely to engage in explicit refinement. In contrast, problem solving drives attentive example study. It breaks students "illusion of knowing" and motivates more careful example study (as mentioned above). Problem solving appears important for turning slow explicit processing into fast habit-like processing. This role of problem solving is what Trafton & Reiser called "knowledge compilation", which is further elaborated in Anderson's ACT-R theory (Anderson, Fincham, & Douglass, 1997). Combining example study and problem solving thus draws on their complementary benefits. By interleaving the two, example study remains more focused and problem solving is more likely to "stamp in" accurate knowledge components that employ the relevant retrieval features and avoid irrelevant ones (i.e., have high feature validity).

Conditions of application

1. Interleave examples and problems. Trafton & Reiser (1993) showed that examples and problems should be given in an alternating or interleaved order (Example, Problem, Example, Problem, ...) and not blocked (Example, Example, ..., Problem, Problem, ...). This was the approach taken in McLaren's PSLC studies.

2. Switch to problems later in learning. The "expertise-reversal effect" suggests that it is earlier in skill development when the Worked Example Principle will be applicable, whereas later in development have students just solve problems without interleaved examples may be better (Kalyuga, Chandler, Tuovinen, & Sweller, 2001).

3. Including explanations in examples helps when there are no self-explanation prompts, but hurts when there are self-explanation prompts.  See the discussion of not providing explanations in the example above in the Examples section.  Schworm and Renkl have explored this issue contrasting whether instructional explanations (given on demand) are present or not, and (in a second study) whether self-explanation prompts are present or not (ADD REFS to Renkl).  

4. Indicate subgoals in the example.  In constrast to null or negative effects of adding explanations to examples (i.e., statements that justify a step), indicating how the steps fit into a hierarchy of goals and subgoals (e.g., by labeling some steps as key subgoals) does appear to aid learning.  (ADD REFS to Catrambone).

5. Separate example study from problem solving.  Having the example present during problem solving may encourage shallow processing (i.e., copying and small edits without understanding) of the example and may not yield benefit.   While there is clear theoretical support for this condition of application, there does not seem to be more solid experimental evidence for it.  Preliminary results from Anthony's PSLC study are consistent with the idea that the worked example effect is not found when examples are provided to students while they are asked to solve an analogous problem.

6. Tell students to study the example to prepare for upcoming problem solving.  According to John Sweller (personal communication with Ken Koedinger), in his experiments, students were instructed at the beginning to study each example in preparation for upcoming problem solving.  The prompting is recommended as critical to give students motivation to attend to and study the example.  It is not clear whether there is any experimental support for this condition of application (i.e., comparing learning with this instruction vs. without). [Need to add references, this may be described in Sweller's book, Sweller, 1999] Note that, unlike the Sweller studies, McLaren's PSLC studies did not instruct students to study worked examples in preparation for problem solving. Rather, students were simply presented worked examples, with no preparation. These studies resulted in a worked example benefit, with respect to efficiency but not with respect to learning (at least standard pre-post learning).

Generalizations (ascendants)

Example-rule coordination principle

References

• Anderson, J. R., Fincham, J. M., & Douglass, S. (1997). The role of examples and rules in the acquisition of cognitive skill. Journal of Experimental Psychology: Learning Memory, and Cognition, 23(4), 932–945.

• Atkinson, R., Derry, S.J., Renkl, A., & Wortham, D. (2000). Learning from examples: Instructional principles from the worked examples research. Review of Educational Research, 70(2), 181-214.

• Clark, R. C., & Mayer, R. E. (2003). e-Learning and the Science of Instruction : Proven Guidelines for Consumers and Designers of Multimedia Learning. San Francisco: Jossey-Bass.

• Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is superior to studying worked examples. Journal of Educational Psychology, 93(3), 579–588.

• Lovett, M.C. (1992). Learning by problem solving versus by examples: The benefits of generating and receiving information. In: 14th Annual Conference of the Cognitive Science Society, pp. 956-961. Hillsdale, NJ: Erlbaum.

• Paas, F. (1992). Training strategies for attaining transfer of problem solving skill in statistics: A cognitive load approach. Journal of Educational Psychology, 84, 429–434.

• Renkl, A. & Atkinson, R. K. (2003). Structuring the transition from example study to problem solving in cognitive skills acquisition: A cognitive load perspective. Educational Psychologist, 38, 15-22.

• Salden, R., Aleven, V., Renkl, A., & Schwonke, R. (2008). Worked examples and tutored problem solving: redundant or synergistic forms of support? In C. Schunn (Ed.) Proceedings of the Annual Meeting of the Cognitive Science Society, CogSci 2008. New York, NY: Lawrence Erlbaum. Cognition and Student Learning Prize.

• Schwonke, R., Renkl, A., Krieg, C., Wittwer, J., Aleven, V., & Salden, R. (in press). The worked-example effect: is it just an artefact of lousy control conditions? Computers in Human Behavior.

• Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and Instruction, 2, 59–89.

• Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257-285.

• Sweller, J. (1999). Instructional design in technical areas.  Camberwell, Australia: ACER Press

• Sweller, J., van Merrienboer, J.J.G., & Paas, F. (1998).  Cognitive architecture and instructional design.  Educational Psychology Review, 10, 251-296

• Trafton, J. G., & Reiser, B. J. (1993). The contribution of studying examples and solving problems to skill acquisition. Proceedings of the 15th Annual Conference of the Cognitive Science Society (pp. 1017–1022). Hillsdale: Lawrence Erlbaum Associates, Inc.

• Zhu, X., & Simon, H. A. (1987). Learning mathematics from examples and by doing. Cognition and Instruction, 4(3), 137-166.