Difference between revisions of "Ringenberg Ill-Defined Physics"

From LearnLab
Jump to: navigation, search
(Annotated bibliography)
(References)
 
(One intermediate revision by the same user not shown)
Line 25: Line 25:
  
 
=== Background and Significance ===
 
=== Background and Significance ===
One of the great challenges in physics education is that traditional
+
The goal of a physics course is not to teach how to solve highly constrained \physics problems" but to increase a student's understanding of the world around them and how it can be modeled using equations and formulas. Although students are often good at solving the highly constrained physics problems that are part of homework and exams, they often lack an understanding of the concepts and principles exemplified by those problems [Halloun and Hestenes, 1985].  
physics teaching methods lead to shallow learning. Most physics
 
students, regardless of their grades in class, have a poor
 
understanding of the concepts being taught (Halloun & Hestenes, 1985).
 
One possible source of this discrepancy between conceptual
 
understanding and performance is that traditional teaching methods
 
rely heavily on the use of well-defined physics problems as both the
 
primary practice and primary assessment activity.  While it is
 
important for students of physics to be able to solve these
 
well-defined problems, it is obviously not enough.
 
  
The homework and exam problems typically presented in a physics class
+
One possible reason for this is that students develop adequate strategies for solving physics problems, but these strategies are highly procedural and do not rely on a conceptual understanding of the materialThere is evidence that students employ shallow problem solving methods. For example, when asked to sort problems, novices will rely on surface features of the problem like what objects (inclines, boxes, pucks, ice, etc.) are used and what quantities are specified (mass, velocity, coefficient of friction, etc.), whereas experts will match problems based on what principles are used to solve them [Chi et al., 1981].
are so constrained that students do not have to do any conceptual
 
analysis of the problem in order to solve themThey tend to look at
 
the quantities supplied in a problem description, match them with
 
known equations, and simply use algebra to find the value of the
 
variable requested in the problem (Chi, Feltovich, & Glaser, 1981).  Additionally,
 
successful novices will match surface features of the problem
 
(particular keywords, phrases, or quantities) to previously solved
 
problems or worked examples to decide which equation to use
 
(Vanlehn & Jones, 1993). These algebraic methods may be reliable methods
 
of solving straight-forward physics problems, but do not require any
 
conceptual knowledge of physics.
 
  
In contrast, experts in physics
+
More evidence for shallow problem solving methods includes observed use of examples. When referring back to examples or previously solved problems, novices will focus on matching surface features and try to mimic all the steps presented in the example [VanLehn and Jones, 1993]. However, when supplied with a relevant example that matches the current problem based on the principles used and may or may not match surface features, students tend to rapidly develop better principle-based problem analysis strategies [Ringenberg and VanLehn, 2006].  
solve problems by conceptualizing the problem first, forming a
 
qualitative solution, and then finally using the relevant given
 
quantities to arrive at the numeric solution (Larkin & Reif, 1979).
 
Performing more expert-like problem solving strategies can be
 
important for fostering this conceptual knowledge.  When novices are
 
given the specific task of specifying which physics principles are
 
needed to solve a problem as part of homework and exams, which
 
requires them to perform an intermediate step in the expert
 
problem-solving strategy, their understanding of these concepts
 
improves more than just solving problems (Leonard, Dufresne, & Mestre, 1996).
 
  
One problem with having students specify the principles used to solve a
+
On the other hand, physics experts tend to solve these (and real-world) problems by first conceptualizing the problem, then forming a qualitative solution, and finally using relevant given quantities to derive a numeric solution [Larkin and Reif, 1979]. When students are required to perform more expert-like problem solving strategies, they can develop better conceptual knowledge. For example, when students are given the task of specifying which physics principles are needed to solve well-defined physics problems as part of their homework and exams (i.e. conceptualizing the problem), their understanding of these concepts improves more than just solving the problems [Leonard et al., 1996].  
given homework problem is that students can still use surface features
 
of the problem to deduce the principles and without immediate corrective
 
feedback they could use naive problem solving strategies and then simply
 
report what principles they used by examining the equations used to solve
 
the problem.
 
  
One way to reduce the reliance on surface features is to remove them.  For
+
Unfortunately, simply requiring students to perform more expert-like problem solving strategies may notbe enough. The Andes tutoring system requires students to draw diagrams and specify necessary variables which are steps that experts perform, and students do better on these skills but do not perform any better on "conceptual" skills like principle identification [VanLehn et al., 2005]. A shortcoming of just prescribing and enforcing a problem solving strategy is that students may not see the value of using it and just adapt their shallow methods to include these extra steps.  
example in the physics domain, key terms such as force, momentum, and energy
 
can be avoided and no given quantities could be specified in the problem statement.
 
As a consequence of this, they become [[ill-defined problem]]s in that key information is missing. Students typically have a difficult time solving [[ill-defined problem]]s, but
 
are able to if they have the support of well constructed pier groups (Heller & Hollabaugh, 1992)
 
or additional support (Ge & Land, 2003).
 
  
This study aims a providing support for solving [[ill-defined problem]]s in the domain of physics in order to investigate their effects on conceptual understanding as compared to solving well-defined problems.
+
Having physics students solve more ill-defined problems that are difficult to solve with shallow strategies seems like a viable method for encouraging more conceptual methods. According to [Simon, 1973], "ill-defined" problems covers a broad category of problem types, including problems that are missing necessary information, have multiple equally valid though not equivalent solutions, or have no definitive solution that experts can agree upon. Whereas experts often deal with these sorts of problems, students often have great difficulty solving these problems.
 +
 
 +
Programs like Activity-Based Physics and Project SCALE-UP break students up into peer groups to solve ill-defined problems. It is hard, however, to determine the effectiveness of the task in such situations as factors such as group size, variations in member's abilities, member's personalities, and gender composition all influence performance measures [Heller and Hollabaugh, 1992]. Maintaining an effective and productive group is very difficult, particularly when a single "bad apple" can negatively affect the entire group's performance [Felps et al., 2006].
 +
 
 +
Upon analysis of groups that successfully solve ill-defined problems, it was found that these groups tended to be more consistent at following a prescribed problem-solving strategy and are more likely to state the physics concepts and principles being used [Heller et al., 1992]. It should be possible to develop a system that can support solving ill-defined problems by enforcing the use of some better problem-solving strategies.
 +
 
 +
In the proposed study, participants will be presented with problems that lack some of the contextual clues of standard physics textbook problems. They will not reference idealized objects like "''box A'' slides down an ''incline''." Problems will cover concepts from Newton's Laws, Momentum, or Energy, which should prevent simple guessing on the part of the participants based on what they have studied or used most recently. In addition, participants in the "ill-defined"(experimental) condition will initially have all of the given quantities (mass, velocities, etc.) removed. This causes the problems to become ill-defined in that information necessary for producing a solution is missing. Participants in this condition will be asked to specify what information they would need to solve the problem. After they have done this, they will be provided with the necessary quantities and asked to solve the problem normally. This task is designed to get them to think about what principles they would used to solve the problem and how they are going to apply them. The hints and feedback they receive will facilitate this by asking what principles they plan to use and what would be necessary to use a given principle.
 +
 
 +
It is expected that participants in the ill-defined condition will develop more expert-like conceptual analysis as their shallow methods are not applicable for this task. This more expert-like conceptual analysis should be easily discernible using a [Dufresne et al., 1992] like problem matching task. In this task, participants are presented with a model problem statement and then asked which of two target problem statements would be solved most similarly to the model. Only one of the targets will use the same principles to solve it, but each of them may share surface features with the model. If participants are doing a "deeper" analysis, then they will select the targets that match based on the principles used to solve it, regardless of the amount of surface similarity. It is expected that engaging the principles in a more conceptual way will lead to greater conceptual understanding, as opposed to just thinking of them as labels for equations. To see if this task encourages a greater conceptual understanding, a multiple-choice conceptual inventory [Singh and Rosengrant, 2003] will be used to assess any gains made over the intervention.
  
 
=== Glossary ===
 
=== Glossary ===
Line 134: Line 102:
  
 
==== References ====
 
==== References ====
*Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
+
*[Chi et al., 1981] Chi, M. T. H., Feltovich, P., and Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5:121-152.
*Dufresne, R. J., Gerace, W. J., Hardiman, P. T., & Mestre, J. P. (1992). Constraining Novices to Perform Expertlike Problem Analyses: Effects on Schema Acquisition. Journal of the Learning Sciences, 2(3), 307-331.
+
 
*Ge, X., & Land, S. M. (2003). Scaffolding students' problem-solving processes in an ill-structured task using question prompts and peer interactions. [Article]. Etr\&D-Educational Technology Research and Development, 51(1), 21-38.
+
*[Dufresne et al., 1992] Dufresne, R. J., Gerace, W. J., Hardiman, P. T., and Mestre, J. P. (1992). Con- straining novices to perform expertlike problem analyses: Effects on schema acquisition. Journal of the Learning Sciences, 2(3):307-331.
*Halloun, I. A., & Hestenes, D. (1985). The initial knowledge state of college physics students. American Journal of Physics, 53(11), 1043-1055.
+
 
*Heller, P., & Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. Part 2: Designing problems and structuring groups. American Journal of Physics, 60(7), 637-644.
+
*[Felps et al., 2006] Felps, W., Mitchell, T. R., and Byington, E. (2006). How, when, and why bad apples spoil the barrel: Negative group members and dysfunctional groups. Research in Organizational Behavior, 27:175-222.
*Larkin, J. H., & Reif, F. (1979). Understanding and Teaching Problem-Solving in Physics. International Journal of Science Education, 1(2), 191-203.
+
 
*Leonard, W. J., Dufresne, R. J., & Mestre, J. P. (1996). Using qualitative problem-solving strategies to highlight the role of conceptual knowledge in solving problems. [Article]. American Journal of Physics, 64(12), 1495-1503.
+
*[Halloun and Hestenes, 1985] Halloun, I. A. and Hestenes, D. (1985). The initial knowledge state of college physics students. American Journal of Physics, 53(11):1043-1055.
*VanLehn, K., & Jones, R. M. (1993). Better learners use analogical problem solving sparingly. Paper presented at the Proceedings of the Tenth International Conference on Machine Learning, San Mateo, CA.
+
 
 +
*[Heller and Hollabaugh, 1992] Heller, P. and Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. part 2: Designing problems and structuring groups. American Journal of Physics, 60(7):637-644.
 +
 
 +
*[Heller et al., 1992] Heller, P., Keith, R., and Anderson, S. (1992). Teaching problem solving through cooperative grouping. part 1: Group versus individual problem solving. American Journal of Physics, 60(7):627-636.
 +
 
 +
*[Larkin and Reif, 1979] Larkin, J. H. and Reif, F. (1979). Understanding and teaching problem-solving in physics. International Journal of Science Education, 1(2):191-203.
 +
 
 +
*[Leonard et al., 1996] Leonard, W. J., Dufresne, R. J., and Mestre, J. P. (1996). Using qualitative problem- solving strategies to highlight the role of conceptual knowledge in solving problems. American Journal of Physics, 64(12):1495-1503.
 +
 
 +
*[Ringenberg and VanLehn, 2006] Ringenberg, M. A. and VanLehn, K. (2006). Scaffolding problem solving with annotated, worked-out examples to promote deep learning. In Ikeda, M., Ashley, K., and Chan, T.- W., editors, ITS 2006, LNCS 4053, pages 625-634, Taiwan. Springer-Verlag Berlin Heidelberg. Winner of Best Paper First Authored by a Student Award.
 +
 
 +
*[Simon, 1973] Simon, H. A. (1973). The structure of ill-structured problems. Artificial Intelligence, 4(3):181- 201.
 +
 
 +
*[Singh and Rosengrant, 2003] Singh, C. and Rosengrant, D. (2003). Multiple-choice test of energy and momentum concepts. American Journal of Physics, 71(6):607-617.
 +
 
 +
*[VanLehn and Jones, 1993] VanLehn, K. and Jones, R. M. (1993). Better learners use analogical problem solving sparingly. In Utgoff, P. E., editor, Proceedings of the Tenth International Conference on Machine Learning, pages 338-345, San Mateo, CA. Morgan Kaufmann.
 +
 
 +
*[VanLehn et al., 2005] VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., and Wintersgill, M. (2005). Andes physics tutoring system: Five years of evaluations. In McCalla, G., Looi, C. K., Bredeweg, B., and Breuker, J., editors, Artificial Intelligence in Education, pages 678-685, Amsterdam, Netherlands. IOS Pres.
  
 
==== Connections ====
 
==== Connections ====

Latest revision as of 19:16, 15 May 2009

Does Solving Ill-Defined Physics Problems Elicit More Learning than Conventional Problem Solving?

Michael Ringenberg and Kurt VanLehn

Summary Table

PIs Michael Ringenberg (Pitt) & Kurt VanLehn (Pitt)
LearnLab Course Physics
Number of Students N = 40


Abstract

Students who complete an introductory physics course often do not have a good conceptual understanding of the principles taught. There have been various attempts at increasing conceptual learning, often with only modest improvements. One promising avenue is the use of ill-defined problems. However, it can be very difficult for students to solve these problems without proper support. If ill-defined problem solving can be supported using intelligent tutoring systems, then it will be possible to investigate the potential of ill-defined problems and their influence on conceptual learning.

Background and Significance

The goal of a physics course is not to teach how to solve highly constrained \physics problems" but to increase a student's understanding of the world around them and how it can be modeled using equations and formulas. Although students are often good at solving the highly constrained physics problems that are part of homework and exams, they often lack an understanding of the concepts and principles exemplified by those problems [Halloun and Hestenes, 1985].

One possible reason for this is that students develop adequate strategies for solving physics problems, but these strategies are highly procedural and do not rely on a conceptual understanding of the material. There is evidence that students employ shallow problem solving methods. For example, when asked to sort problems, novices will rely on surface features of the problem like what objects (inclines, boxes, pucks, ice, etc.) are used and what quantities are specified (mass, velocity, coefficient of friction, etc.), whereas experts will match problems based on what principles are used to solve them [Chi et al., 1981].

More evidence for shallow problem solving methods includes observed use of examples. When referring back to examples or previously solved problems, novices will focus on matching surface features and try to mimic all the steps presented in the example [VanLehn and Jones, 1993]. However, when supplied with a relevant example that matches the current problem based on the principles used and may or may not match surface features, students tend to rapidly develop better principle-based problem analysis strategies [Ringenberg and VanLehn, 2006].

On the other hand, physics experts tend to solve these (and real-world) problems by first conceptualizing the problem, then forming a qualitative solution, and finally using relevant given quantities to derive a numeric solution [Larkin and Reif, 1979]. When students are required to perform more expert-like problem solving strategies, they can develop better conceptual knowledge. For example, when students are given the task of specifying which physics principles are needed to solve well-defined physics problems as part of their homework and exams (i.e. conceptualizing the problem), their understanding of these concepts improves more than just solving the problems [Leonard et al., 1996].

Unfortunately, simply requiring students to perform more expert-like problem solving strategies may notbe enough. The Andes tutoring system requires students to draw diagrams and specify necessary variables which are steps that experts perform, and students do better on these skills but do not perform any better on "conceptual" skills like principle identification [VanLehn et al., 2005]. A shortcoming of just prescribing and enforcing a problem solving strategy is that students may not see the value of using it and just adapt their shallow methods to include these extra steps.

Having physics students solve more ill-defined problems that are difficult to solve with shallow strategies seems like a viable method for encouraging more conceptual methods. According to [Simon, 1973], "ill-defined" problems covers a broad category of problem types, including problems that are missing necessary information, have multiple equally valid though not equivalent solutions, or have no definitive solution that experts can agree upon. Whereas experts often deal with these sorts of problems, students often have great difficulty solving these problems.

Programs like Activity-Based Physics and Project SCALE-UP break students up into peer groups to solve ill-defined problems. It is hard, however, to determine the effectiveness of the task in such situations as factors such as group size, variations in member's abilities, member's personalities, and gender composition all influence performance measures [Heller and Hollabaugh, 1992]. Maintaining an effective and productive group is very difficult, particularly when a single "bad apple" can negatively affect the entire group's performance [Felps et al., 2006].

Upon analysis of groups that successfully solve ill-defined problems, it was found that these groups tended to be more consistent at following a prescribed problem-solving strategy and are more likely to state the physics concepts and principles being used [Heller et al., 1992]. It should be possible to develop a system that can support solving ill-defined problems by enforcing the use of some better problem-solving strategies.

In the proposed study, participants will be presented with problems that lack some of the contextual clues of standard physics textbook problems. They will not reference idealized objects like "box A slides down an incline." Problems will cover concepts from Newton's Laws, Momentum, or Energy, which should prevent simple guessing on the part of the participants based on what they have studied or used most recently. In addition, participants in the "ill-defined"(experimental) condition will initially have all of the given quantities (mass, velocities, etc.) removed. This causes the problems to become ill-defined in that information necessary for producing a solution is missing. Participants in this condition will be asked to specify what information they would need to solve the problem. After they have done this, they will be provided with the necessary quantities and asked to solve the problem normally. This task is designed to get them to think about what principles they would used to solve the problem and how they are going to apply them. The hints and feedback they receive will facilitate this by asking what principles they plan to use and what would be necessary to use a given principle.

It is expected that participants in the ill-defined condition will develop more expert-like conceptual analysis as their shallow methods are not applicable for this task. This more expert-like conceptual analysis should be easily discernible using a [Dufresne et al., 1992] like problem matching task. In this task, participants are presented with a model problem statement and then asked which of two target problem statements would be solved most similarly to the model. Only one of the targets will use the same principles to solve it, but each of them may share surface features with the model. If participants are doing a "deeper" analysis, then they will select the targets that match based on the principles used to solve it, regardless of the amount of surface similarity. It is expected that engaging the principles in a more conceptual way will lead to greater conceptual understanding, as opposed to just thinking of them as labels for equations. To see if this task encourages a greater conceptual understanding, a multiple-choice conceptual inventory [Singh and Rosengrant, 2003] will be used to assess any gains made over the intervention.

Glossary

Research question

Does Solving Ill-Defined Physics Problems Elicit More Learning than Conventional Problem Solving?

Independent variables

For this study, the ill-defined problems used lacked key information needed to solve the problem. The problem statements did not include the quantities needed to derive a numeric solution to the problem. Part of the task of solving these problems was to have the participants request the necessary quantities from the system. The system will provide hints and corrective feedback for this task. Once all of the necessary values are elicited by the participant, then the problem becomes a well-defined problem. The well-defined problems used were identical to the ill-defined problems except that all of the necessary information was given as part of the problem statement.

Figure 1. Example of one of the ill-defined problems. It is missing key information which would be required to produce a quantitative solution.

Regina is practising skateboard tricks. She grinds her board along a horizontal rail and falls of the end onto a mattress she placed there. How fast is she travelling just before hitting the mattress?

Figure 2. Example of the corresponding well-defined version of the problem in Figure 1. It includes all of the necessary and sufficient information needed to produce a quantitative solution to the problem.

Regina is practising skateboard tricks. She grinds her board along a horizontal rail and falls of the end onto a mattress she placed there. How fast is she travelling just before hitting the mattress?

Height of rail: 0.5 m
Regina's velocity when she leaves the rail: 0.45 m/s

Hypothesis

If students are required to figure out what information is needed to solve ill-defined physics problems before solving them, then they will develop better conceptual understanding than if they had been presented with the same problems with all the necessary information provided.

Dependent variables

  • Normal post-test
    • Multiple-choice conceptual questions
  • Transfer
    • Judgement task
      • Problem matching task: participants are given a target problem statement and are asked which of two additional problem statements are solved most similarly to the target problem without solving any of the problems (Dufresne, Gerace, Hardiamnn, & Mestre, 1992).
  • Performance on Andes problems
    • Solution times
    • Error rates
    • Help requests

Expected Findings

Participants who solve the ill-defined versions of the problems will:

  • Perform better on conceptual questions.
  • Perform better on the problem matching task.
  • Have faster solution times.
  • Have lower error rates.
  • Have fewer help requests.

Explanation

Because the experimental participants will be required to engage in more conceptual analysis of the problems, they will more deeply analyze and encode the knowledge components used in the problems. This will lead to better performance on tasks that use this better encoding. It will also have effects on problem solving because with the conceptual analysis done before problem solving, there will be less floundering and help abuse during problem solving.

Further Information

Annotated bibliography

  • Paper and poster presented at ITS 2008 Conference (young researcher's track).

References

  • [Chi et al., 1981] Chi, M. T. H., Feltovich, P., and Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5:121-152.
  • [Dufresne et al., 1992] Dufresne, R. J., Gerace, W. J., Hardiman, P. T., and Mestre, J. P. (1992). Con- straining novices to perform expertlike problem analyses: Effects on schema acquisition. Journal of the Learning Sciences, 2(3):307-331.
  • [Felps et al., 2006] Felps, W., Mitchell, T. R., and Byington, E. (2006). How, when, and why bad apples spoil the barrel: Negative group members and dysfunctional groups. Research in Organizational Behavior, 27:175-222.
  • [Halloun and Hestenes, 1985] Halloun, I. A. and Hestenes, D. (1985). The initial knowledge state of college physics students. American Journal of Physics, 53(11):1043-1055.
  • [Heller and Hollabaugh, 1992] Heller, P. and Hollabaugh, M. (1992). Teaching problem solving through cooperative grouping. part 2: Designing problems and structuring groups. American Journal of Physics, 60(7):637-644.
  • [Heller et al., 1992] Heller, P., Keith, R., and Anderson, S. (1992). Teaching problem solving through cooperative grouping. part 1: Group versus individual problem solving. American Journal of Physics, 60(7):627-636.
  • [Larkin and Reif, 1979] Larkin, J. H. and Reif, F. (1979). Understanding and teaching problem-solving in physics. International Journal of Science Education, 1(2):191-203.
  • [Leonard et al., 1996] Leonard, W. J., Dufresne, R. J., and Mestre, J. P. (1996). Using qualitative problem- solving strategies to highlight the role of conceptual knowledge in solving problems. American Journal of Physics, 64(12):1495-1503.
  • [Ringenberg and VanLehn, 2006] Ringenberg, M. A. and VanLehn, K. (2006). Scaffolding problem solving with annotated, worked-out examples to promote deep learning. In Ikeda, M., Ashley, K., and Chan, T.- W., editors, ITS 2006, LNCS 4053, pages 625-634, Taiwan. Springer-Verlag Berlin Heidelberg. Winner of Best Paper First Authored by a Student Award.
  • [Simon, 1973] Simon, H. A. (1973). The structure of ill-structured problems. Artificial Intelligence, 4(3):181- 201.
  • [Singh and Rosengrant, 2003] Singh, C. and Rosengrant, D. (2003). Multiple-choice test of energy and momentum concepts. American Journal of Physics, 71(6):607-617.
  • [VanLehn and Jones, 1993] VanLehn, K. and Jones, R. M. (1993). Better learners use analogical problem solving sparingly. In Utgoff, P. E., editor, Proceedings of the Tenth International Conference on Machine Learning, pages 338-345, San Mateo, CA. Morgan Kaufmann.
  • [VanLehn et al., 2005] VanLehn, K., Lynch, C., Schulze, K., Shapiro, J. A., Shelby, R., Taylor, L., Treacy, D., Weinstein, A., and Wintersgill, M. (2005). Andes physics tutoring system: Five years of evaluations. In McCalla, G., Looi, C. K., Bredeweg, B., and Breuker, J., editors, Artificial Intelligence in Education, pages 678-685, Amsterdam, Netherlands. IOS Pres.

Connections

Future plans