Ringenberg Ill-Defined Physics

Does Solving Ill-Defined Physics Problems Elicit More Learning than Conventional Problem Solving?

Michael Ringenberg and Kurt VanLehn

Summary Table

PIs	Michael Ringenberg (Pitt) & Kurt VanLehn (Pitt)
LearnLab Course	Physics
Number of Students	N = 40

Abstract

Students who complete an introductory physics course often do not have a good conceptual understanding of the principles taught. There have been various attempts at increasing conceptual learning, often with only modest improvements. One promising avenue is the use of ill-defined problems. However, it can be very difficult for students to solve these problems without proper support. If ill-defined problem solving can be supported using intelligent tutoring systems, then it will be possible to investigate the potential of ill-defined problems and their influence on conceptual learning.

Background and Significance

One of the great challenges in physics education is that traditional physics teaching methods lead to shallow learning. Most physics students, regardless of their grades in class, have a poor understanding of the concepts being taught (Halloun & Hestenes, 1985). One possible source of this discrepancy between conceptual understanding and performance is that traditional teaching methods rely heavily on the use of well-defined physics problems as both the primary practice and primary assessment activity. While it is important for students of physics to be able to solve these well-defined problems, it is obviously not enough.

The homework and exam problems typically presented in a physics class are so constrained that students do not have to do any conceptual analysis of the problem in order to solve them. They tend to look at the quantities supplied in a problem description, match them with known equations, and simply use algebra to find the value of the variable requested in the problem (Chi, Feltovich, & Glaser, 1981). Additionally, successful novices will match surface features of the problem (particular keywords, phrases, or quantities) to previously solved problems or worked examples to decide which equation to use (Vanlehn & Jones, 1993). These algebraic methods may be reliable methods of solving straight-forward physics problems, but do not require any conceptual knowledge of physics.

In contrast, experts in physics solve problems by conceptualizing the problem first, forming a qualitative solution, and then finally using the relevant given quantities to arrive at the numeric solution (Larkin & Reif, 1979). Performing more expert-like problem solving strategies can be important for fostering this conceptual knowledge. When novices are given the specific task of specifying which physics principles are needed to solve a problem as part of homework and exams, which requires them to perform an intermediate step in the expert problem-solving strategy, their understanding of these concepts improves more than just solving problems (Leonard, Dufresne, & Mestre, 1996).

One problem with having students specify the principles used to solve a given homework problem is that students can still use surface features of the problem to deduce the principles and without immediate corrective feedback they could use naive problem solving strategies and then simply report what principles they used by examining the equations used to solve the problem.

One way to reduce the reliance on surface features is to remove them. For example in the physics domain, key terms such as force, momentum, and energy can be avoided and no given quantities could be specified in the problem statement. As a consequence of this, they become ill-defined problems in that key information is missing. Students typically have a difficult time solving ill-defined problems, but are able to if they have the support of well constructed pier groups (Heller & Hollabaugh, 1992) or additional support (Ge & Land, 2003).

This study aims a providing support for solving ill-defined problems in the domain of physics in order to investigate their effects on conceptual understanding as compared to solving well-defined problems.

Research question

Does Solving Ill-Defined Physics Problems Elicit More Learning than Conventional Problem Solving?

Independent variables

• Type of problems solved: ill-defined problems vs. well-defined problems

For this study, the ill-defined problems used lacked key information needed to solve the problem. The problem statements did not include the quantities needed to derive a numeric solution to the problem. Part of the task of solving these problems was to have the participants request the necessary quantities from the system. The system will provide hints and corrective feedback for this task. Once all of the necessary values are elicited by the participant, then the problem becomes a well-defined problem. The well-defined problems used were identical to the ill-defined problems except that all of the necessary information was given as part of the problem statement.

Hypothesis

If students are required to figure out what information is needed to solve ill-defined physics problems before solving them, then they will develop better conceptual understanding than if they had been presented with the same problems with all the necessary information provided.

Dependent variables

• Normal post-test
  • Multiple-choice conceptual questions
• Transfer
  • Judgement task
    • Problem matching task: participants are given a target problem statement and are asked which of two additional problem statements are solved most similarly to the target problem without solving any of the problems (Dufresne, Gerace, Hardiamnn, & Mestre, 1992).
• Performance on Andes problems
  • Solution times
  • Error rates
  • Help requests

Expected Findings

Participants who solve the ill-defined versions of the problems will:

• Perform better on conceptual questions.
• Perform better on the problem matching task.
• Have faster solution times.
• Have lower error rates.
• Have fewer help requests.

Explanation

Because the experimental participants will be required to engage in more conceptual analysis of the problems, they will more deeply analyze and encode the knowledge components used in the problems. This will lead to better performance on tasks that use this better encoding. It will also have effects on problem solving because with the conceptual analysis done before problem solving, there will be less floundering and help abuse during problem solving.

Further Information

References

• Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
• Dufresne, R. J., Gerace, W. J., Hardiman, P. T., & Mestre, J. P. (1992). Constraining Novices to Perform Expertlike Problem Analyses: Effects on Schema Acquisition. Journal of the Learning Sciences, 2(3), 307-331.
• Ge, X., & Land, S. M. (2003). Scaffolding students' problem-solving processes in an ill-structured task using question prompts and peer interactions. [Article]. Etr\&D-Educational Technology Research and Development, 51(1), 21-38.
• Halloun, I. A., & Hestenes, D. (1985). The initial knowledge state of college physics students. American Journal of Physics, 53(11), 1043-1055.
• Heller, P., & Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. Part 2: Designing problems and structuring groups. American Journal of Physics, 60(7), 637-644.
• Larkin, J. H., & Reif, F. (1979). Understanding and Teaching Problem-Solving in Physics. International Journal of Science Education, 1(2), 191-203.
• Leonard, W. J., Dufresne, R. J., & Mestre, J. P. (1996). Using qualitative problem-solving strategies to highlight the role of conceptual knowledge in solving problems. [Article]. American Journal of Physics, 64(12), 1495-1503.
• VanLehn, K., & Jones, R. M. (1993). Better learners use analogical problem solving sparingly. Paper presented at the Proceedings of the Tenth International Conference on Machine Learning, San Mateo, CA.