Reflective Dialogues (Katz)

Do Reflective Dialogues that Explicitly Target the “What? How? and Why (not)?” Knowledge of Physics Problem Solving Promote Expert-like Planning Ability?

Sandra Katz

Summary Table

PIs: Sandra Katz, John Connelly, Donald Treacy Others who have contributed 160 hours or more: Faculty: -- Post-Docs: -- Graduate Students: -- Start Date: 03-01-06 End Date: 06-30-07 LearnLab Site and Course: USNA, General Physics I Number of Students: 67 Participant Hours: ??? Data in DataShop? No; Andes data still not compatible with Datashop.

Abstract

One of the main differences between experts and novices in physics is that experts are more adept at identifying relevant principles and generating solution plans before starting to solve a problem (Dufresne, Gerace, Hardiman, & Mestre, 1992; Chi, Glaser & Rees, 1982; Priest & Lindsay, 1992). We propose that this difference may be due to two reasons: (1) in traditional physics courses, students are not explicitly asked to plan or are they given scaffolding to support planning during problem solving, and (2) many students lack the basic knowledge of physics concepts, principles, and procedures that is prerequisite for effective planning.

In this project, we are testing the effectiveness of engaging students in reflective dialogues after they solve problems in the Andes that explicitly target the three main types of knowledge that experts employ during planning: knowledge about what principle(s) to apply to a given problem, how to apply these principles (e.g., what equations to use), and why (or why not) to apply them—that is, what the applicability conditions are (Leonard, Dufresne, & Mestre, 1996). The main hypothesis to be tested is that explicit training on the “what? how? and why?” components of planning, via strategically staged reflective dialogues, will be more effective and efficient than the traditional approach of letting students acquire these components implicitly, through lots of practice with solving problems.

To test this hypothesis, we will conduct a Physics LearnLab study in which we compare the effectiveness of an experimental version of Andes that engages students in reflective “what? how? and why?” dialogues after physics problem solving (explicit instruction condition) with a control version that gives students additional problem-solving practice after they solve an Andes problem—e.g., by having them identify bugs in a sample student solution to a problem (implicit instruction condition). We predict that students in the explicit condition will outperform students in the implicit condition with respect to gain score from pre-test to post-test and scores on course exams that target far transfer. The main contribution of this study will be to determine if explicit instruction in planning and in the component knowledge and skills needed for planning via reflective dialogues can promote students’ acquisition of solution schemata.

Glossary

• Post-practice reflection
• Reflection questions
• Knowledge Construction Dialogues (KCDs)

Research question

Does explicit training in the three main components of problem-solving knowledge—i.e., knowledge about what principles apply to a problem, how to apply these principles, and why to apply them—enhance students’ problem-solving ability?

Background and Significance

Because experts develop schemata largely through a good deal of practice with solving various types of problems, traditional introductory college physics courses take a similar approach. They are structured so that students are introduced to concepts, principles, and procedures in their text and through lectures, which they are then asked to apply to many problem-solving exercises. The hope is that, with extensive practice, students will integrate conceptual and procedural knowledge and develop expert-like schemata and planning skills. Unfortunately, many students don’t. In addition to exiting these courses with lingering naïve misconceptions (e.g., Halloun & Hestenes, 1985; McDermott, 1984), they continue to solve problems largely by manipulating equations until the desired quantity is isolated—that is, by means-end analysis—instead of by identifying relevant principles and generating solution plans before starting to solve a problem, as experts do (Dufrene et al., 1992; Larkin, McDermott, Simon, & Simon, 1980; Priest & Lindsay, 1992).

We refer to the traditional approach to physics instruction described in the preceding paragraph as implicit instruction, because it encourages the inductive development of abstractions (concepts, principles, and schemata) through repeated exposure to instances, instead of by explicitly reifying these abstractions (O'Malley & Chamot, 1994). In response to the limitations of implicit approaches to physics instruction that neither ask students to plan nor scaffold them in doing it, several instructional scientists have proposed methods that explicitly engage students in planning exercises (Dufresne et al., 1992; Leonard, Dufresne, & Mestre, 1996; Mestre, Dufresne, Gerace, & Hardiman, 1993). These methods have met with modest success when tested mainly using high-achieving students (B or above in an introductory college physics course). We propose that the main reason that many students, even high achievers, are unable to use explicit planning methods effectively is that they lack the basic concepts, principles, and procedures that are prerequisites for effective planning.

This project tests the effectiveness of engaging students in reflective dialogues after they solve problems in Andes (e.g., Gertner & VanLehn, 2000) which explicitly target the three main types of knowledge that experts employ during planning: knowledge about what principle(s) to apply to a given problem, how to apply these principles (e.g., what equations to use), and why to apply them—that is, what the applicability conditions are (Leonard, Dufresne, & Mestre, 1996). If explicit instruction on these knowledge components proves to be effective, we will have identified an instructional strategy that facilitates the acquisition of problem-solving schema—one that is more efficient than repeated practice in solving many problems.

Dependent Variables

• Gains in qualitative and quantitative knowledge. Post-test score,and pre-test to post-test gain scores, on near and far transfer items.
• Short-term retention. Performance on course exams that cover target topics (e.g., work and energy, translational dynamics, rotation, momentum)
• Longer-term retention. Performance on final exam, taken several weeks after the intervention.

Independent Variables

• Number of problems completed before the post-test is administered.
• Number of reflective dialogues that the student completed.
• CQPR—grade point average
• College major
• Pre-test score

Hypothesis

The main hypothesis to be tested is that explicit, part-task training on the “what? how? and why?” components of planning, via strategically staged reflective dialogues, will be more effective and efficient than the traditional approach of letting students acquire these components implicitly, through lots of practice with solving problems.

Findings

Data collection is in progress.

Annotated bibliography

SANDY, REPLACE WITH YOUR INFO:

• Presentation to the PSLC Advisory Board, December, 2006 [1]
• Presentation to the NSF Follow-up Site Visitors, September, 2006
• Preliminary results were presented to the Intelligent Tutoring in Serious Games workshop, Aug. 2006 [2]
• Presentation to the NSF Site Visitors, June, 2006
• Poster presented at the annual meeting of the Science of Learning Centers, Oct. 2006.
• Symposium accepted to EARLI 2007
• Symposium accepted at AERA 2007
• Full-paper accepted at AIED 2007
• Full-paper submitted to CogSci 2007

References

Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in the Psychology of Human Intelligence, Vol. 1 (pp. 7-75). Hillsdale, NJ: Lawrence Erlbaum Associates.