Extending Reflective Dialogue Support (Katz & Connelly)
Contents
Extending Automated Dialogue Support for Robust Learning of Physics
Sandra Katz
Summary Table
| PIs | Sandra Katz & John Connelly |
| Study Start Date | 10/1/07 |
| Study End Date | 9/30/08 |
| LearnLab Site | USNA |
| LearnLab Course | General Physics I |
| Number of Students | N = 75 |
| Total Participant Hours | approx. 125 hrs |
| DataShop | Yes |
Abstract
Research on student understanding and problem-solving ability in first-year college physics courses shows that instructors deal with a double-edged sword. Some students become adept at solving quantitative problems but do poorly on tests of conceptual knowledge and qualitative problem-solving ability. Other students display the reverse problem: they show at least a glimmer of understanding of basic physics concepts and principles, but are unable to use this knowledge to solve quantitative problems. Still other students master neither qualitative nor quantitative understanding of physics; very few master both. Thus, the instructional challenge motivating this project is to find effective pedagogical strategies to integrate quantitative and qualitative knowledge. Our scientific goal is to determine whether explicit and implicit learning can be effectively combined via post-practice dialogues that guide students in reflecting on the concepts and principles associated with a just-solved physics problem. The main hypothesis tested is that, in the context of tutored problem solving, integrative reflective dialogues that explicitly tie qualitative knowledge to quantitative knowledge can improve quantitative problem-solving ability and retention of qualitative knowledge better than problem-solving practice (implicit learning) alone.
To test this hypothesis, we conducted an experiment in the PSLC Physics LearnLab at the US Naval Academy in sections that use the Andes physics tutoring system (VanLehn et al., 2005a, 2005b). We compared students who were randomly assigned to one of three conditions on measures of qualitative and quantitative problem-solving performance. The two treatment conditions engaged in automated reflective dialogues after solving quantitative physics problems, while the control condition solved the same set of problems (plus a few additional problems to balance time on task) without any reflective dialogues, using the standard version of Andes. In one treatment condition, the reflective dialogues individually targeted the three main types of knowledge that experts employ during problem solving, according to Leonard, Dufesne, & Mestre (1996): knowledge about what principle(s) to apply to a given problem, how to apply these principles (e.g., what equations to use), and why to apply them—that is, what the applicability conditions are (Leonard, Dufresne, & Mestre, 1996). In a prior LearnLab study (Katz, Connelly & Treacy), this intervention significantly improved students’ qualitative understanding of basic mechanics, as measured by pre-test to post-test gain scores. However, students did not outperform standard Andes users on more robust learning measures of transfer (e.g., performance on quantitative course exams) and on a measure of retention of qualitative problem-solving ability (Katz, Connelly, & Wilson, 2007). The alternative dialogue condition we evaluated differs from the other dialogue condition in three main ways: (1) reflective dialogues contained more problem variations (what if scenarios), designed to support both qualitative and quantitative knowledge (most of our previous what if scenarios were qualitative only); (2) these “what if” scenarios were tied to the corresponding Andes problem-solving context and to new contexts, to help support near and far transfer; and (3) students were prompted to state the rules (knowledge components) applied to solving the problem variations, in order to promote a principle-based approach to learning, and they were given feedback that makes these rules explicit. Our goal is to determine whether reflective dialogues that make the links between qualitative and quantitative physics knowledge explicit are more effective than both our previous dialogues and an implicit learning condition that is based on problem-solving practice alone.
