Difference between revisions of "Using Elaborated Explanations to Support Geometry Learning (Aleven & Butcher)"

From LearnLab
Jump to: navigation, search
 
(→‎Annotated Bibliography)
 
(112 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
== Using Elaborated Explanations to Support Geometry Learning  ==
 
== Using Elaborated Explanations to Support Geometry Learning  ==
−
Vincent Aleven & Kirsten Butcher
+
''Vincent Aleven and Kirsten Butcher''
  
 +
=== Summary Tables ===
 +
====Pilot Study====
 +
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
 +
| '''PIs''' || Vincent Aleven & Kirsten R. Butcher
 +
|-
 +
| '''Other Contributers''' || <b>Graduate Students:</b> Andy Tzou (CMU HCII), Carl Angioli (CMU HCII), Michael Nugent (Pitt, Computer Science)<br>
 +
<b>Research Programmers/Associates:</b> Octav Popescu (Research Programmer, CMU HCII), Grace Lee Leonard (Research Associate, CMU HCII), Thomas Bolster (Research Associate, CMU HCII)
 +
 +
|-
 +
| '''Study Start Date''' || January 24, 2006
 +
|-
 +
| '''Study End Date''' || January 31, 2006
 +
|-
 +
| '''LearnLab Site''' || Central Westmoreland Career & Technology Center (CWCTC)
 +
|-
 +
| '''LearnLab Course''' || Geometry
 +
|-
 +
| '''Number of Students''' || 90
 +
|-
 +
| '''Total Participant Hours''' || 90
 +
|-
 +
| '''DataShop''' || n/a (Pencil & Paper Test)
 +
|}
 +
 +
<br>
 +
====Classroom Study====
 +
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"
 +
| '''PIs''' || Vincent Aleven & Kirsten R. Butcher
 +
|-
 +
| '''Other Contributers''' || <b>Graduate Students:</b> Carl Angioli (CMU HCII), Michael Nugent (Pitt, Computer Science)<br>
 +
<b>Research Programmers/Associates:</b> Octav Popescu (Research Programmer, CMU HCII), Grace Lee Leonard (Research Associate, CMU HCII), Thomas Bolster (Research Associate, CMU HCII)
 +
 +
|-
 +
| '''Study Start Date''' || November 28, 2006
 +
|-
 +
| '''Study End Date''' || January 15, 2007
 +
|-
 +
| '''LearnLab Site''' || Central Westmoreland Career & Technology Center (CWCTC)
 +
|-
 +
| '''LearnLab Course''' || Geometry
 +
|-
 +
| '''Number of Students''' || 90
 +
|-
 +
| '''Total Participant Hours''' || 540
 +
|-
 +
| '''DataShop''' || Status: In Progress. Log files and CTAT online test files have been anonymized, passed to DataShop, and are awaiting conversion and input.
 +
|}
 +
 +
<BR>
 
=== Abstract ===
 
=== Abstract ===
−
Does integration of visual and verbal knowledge during learning support deep understanding? Can visually-related explanations during student learning promote [[robust learning]] in geometry? In our research, we are exploring how student explanations in an intelligent tutoring environment can support learning. We are investigating the use of [[elaborated explanations]]. Elaborated explanations are a form of interactive communication in which students justify problem solving steps by giving an explanation that connects to both visual and verbal knowledge components. In our studies, students give a verbal explanation by stating a geometry rule as well as a visual explanation by stating the diagram features that are being used by the stated rule. We hypothesize that explanations that reference visual and verbal knowledge components will support the integration of visual and verbal knowledge during learning, leading to better understanding of geometry principles.
+
Does integration of visual and verbal knowledge during learning support deep understanding? Can interactive communication using visual and verbal [[knowledge component]]s support the development of integrated representations? In our research, we are exploring how student explanations in an intelligent tutoring environment that includes visual (pictorial) and verbal (textual) information can support [[robust learning]]. In domains like Geometry, where visual (diagram) and verbal (textual and conceptual) information is critical to deep understanding and successful problem solving, focusing student communication acts on conceptual relationships between critical [[knowledge component]]s should support deep understanding. Thus, our research is investigating explanations that link the diagram features (visual knowledge components) and geometry rules (verbal knowledge components) related to successful problem-solving steps. We call these visual-verbal explanations "[[Elaborated Explanations]]." In our studies, students give a verbally-focused explanation (by stating a relevant geometry rule) as well as a visually-focused explanation (by stating the diagram features that used to apply the stated rule). The intelligent tutor provides feedback and hints on these elaborated, verbal-visual explanations. We hypothesize that this type of interactive communication will support [[visual-verbal integration|integration of visual and verbal knowledge]] during learning, leading to better [[transfer]] and [[long-term retention]].
  
−
This study is part of a larger project targeted toward developing a better understanding of 1) visual and verbal [[knowledge components]] in a problem-solving environment and, 2) how interacting with visual information can support the development of deep understanding. Ultimately, we are interested in [[coordination]] and [[integration]] processes in learning with visual and verbal [[knowledge components]], and how these processes may support [[robust learning]].  
+
=== Background & Significance ===
 +
 
 +
A rich body of prior research has demonstrated that students develop deeper understanding of instructional materials when they [[self-explanation | self-explain]] to themselves during learning (e.g., Bielaczyc, Pirolli, & Brown, 1995; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, de Leeuw, Chiu, & LaVancher, 1994). Although much of this research has focused on students self-generated and natural language explanations, research has also shown benefits of simple, menu-based explanations in supporting learning. An existing version of the Geometry Cognitive Tutor implements student self-explanations in this simple manner. After correctly answering a geometry problem step, students must select the geometry rule or theorem that justifies their answer from a glossary menu of terms.
 +
 
 +
Despite the limitations of simple, menu-based explanations, they have been shown to promote student learning in the Geometry Cognitive Tutor (Aleven & Koedinger, 2002). However, these menu-based explanations may not be supporting the development of deep, expert-like connections between conceptual geometry principles and visual features of geometry problem diagrams. Students complete these explanations after they have successfully solved a numerical problem-solving step. There there is no requirement that students make explicit connections between the rule being applied and the visual features that are required for application. Because prior work has shown that experts' geometry problem solving is driven by recognition of key diagram features that cue relevant conceptual knowledge (Koedinger & Anderson, 1990), supporting deep student understanding in geometry may require that students explicitly connect relevant conceptual knowledge to visual diagram features.
 +
 
 +
Our approach is to support students' connections between and integration of visual and verbal [[knowledge component]]s through interactive communication, where students' explanations of their problem solving steps reference both verbal, conceptual knowledge (in the form of a relevant geometry rule) and its relationship to key diagram features. Thus, our question is whether explicit forms of communication that link verbal and visual knowledge components can be more successful at promoting [[robust learning]] than communication that requires expression of verbal, [[declarative]] knowledge only.
  
 
=== Glossary ===
 
=== Glossary ===
 
See [[:Category:Visual-Verbal Learning (Aleven & Butcher Project)|Visual-Verbal Learning Project Glossary]]
 
See [[:Category:Visual-Verbal Learning (Aleven & Butcher Project)|Visual-Verbal Learning Project Glossary]]
  
−
=== Research question ===
+
=== Research questions ===
−
# Can explanations that connect visual and verbal knowledge components support learning better than explanations that reference verbal information only?
+
# Can [[Elaborated Explanations|elaborated explanations]] that connect visual (pictorial) and verbal (textual/conceptual) [[knowledge components]] support [[robust learning]] better than explanations that reference verbal information only?
−
# What [[robust learning]] processes are affected by elaborated explanations during problem-solving?
+
# Are the effects of [[Elaborated Explanations|elaborated explanations]] strongest when considering [[long-term retention]] and [[transfer]] as opposed to results on a [[normal post-test]]?
 +
 
 +
=== Independent Variables ===
 +
*Type of Explanation
 +
**Simple, Textual Explanations (students state geometry principles only)
 +
 
 +
Figure 1. Screen shot of Geometry Cognitive Tutor interface with simple, verbal-only explanations.<br>
 +
[[Image:Diagram_SimpleExpl.jpg]]
 +
<br>vs.<br>
  
−
=== Background & Significance ===
+
*[[Elaborated Explanations]] (students state geometry principles and the diagram features which allow the principle to be applied)
  
−
A rich body of prior research has demonstrated that students develop deeper understanding of instructional materials when they self-explain to themselves during learning (e.g., Bielaczyc, Pirolli, & Brown, 1995; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, de Leeuw, Chiu, & LaVancher, 1994). An existing version of the Geometry Cognitive Tutor implements student self-explanations of their problem solving using a very simple process: after correctly answering a geometry problem step, students must select the geometry rule or theorem that justifies their answer from a glossary menu of terms. Despite the limitations of the menu-based explanations, they have been shown to promote student learning in the Geometry Cognitive Tutor (Aleven & Koedinger, 2002). Thus, our question is whether more explicit forms of communication that link verbal and visual knowledge components can be more successful at promoting [[robust learning]]. We argue that scaffolds that promote [[sense making]] using both visual and verbal knowledge may be of greater benefit to students than scaffolds that require student explanations using only verbal [[declarative]] knowledge.
+
Figure 2. Screen shot of Geometry Cognitive Tutor interface with elaborated explanations<br>
 +
[[Image:Diagram_ElabExpl.jpg]]
  
 
=== Dependent variables ===
 
=== Dependent variables ===
−
*Pretest, [[normal post-test]], and immediate [[transfer]] test measuring student performance on:
+
*[[Normal post-test]], and immediate [[transfer]] test measuring student performance on:
−
**Problem-solving items isomorphic to the practiced problems (near-term knowledge [[retention]])
+
**Problem-solving items isomorphic to the practiced problems (immediate [[retention]])
−
**Problem-solving items unlike those seen during problem practice (near-term knowledge [[transfer]])
+
**Problem-solving items unlike those seen during problem practice (immediate [[transfer]])
−
*Delayed posttest, measuring student performance on:
+
*Delayed [[long-term retention]] posttest, measuring student performance on:
−
**Problem-solving items isomorphic to the practiced problems (far-term knowledge [[retention]])
+
**Problem-solving items isomorphic to the practiced problems ([[long-term retention]])
−
**Problem-solving items unlike those seen during problem practice (far-term knowledge [[transfer]])
+
**Problem-solving items unlike those seen during problem practice (long-term [[transfer]])
−
*Log data collected during tutor use, used to assess:
+
*Log data collected during study-related tutor use, used to assess:
−
**Learning curves
+
**[[Learning curve]]s
 
**Time on task
 
**Time on task
 
**Error rates
 
**Error rates
 
**Latency of responses
 
**Latency of responses
  
−
=== Independent Variables ===
+
*(Planned): Log data collected during Cognitive Tutor completion, after study. Will be used to assess:
−
;2. Type of Explanation
+
**[[Accelerated future learning]]
−
:''Verbal explanations (students state geometry principles only) vs. Elaborated Explanations (students state geometry principles and their application to the diagram)''
 
−
 
 
−
<b>ADD SCREEN SHOTS</b>
 
  
−
=== Hypotheses ===
+
=== Hypothesis ===
  
−
*Contiguous representations in geometry decrease cognitive load (by supporting [[mapping]]) and increase strategic inferences during problem-solving, thus supporting immediate and long-term knowledge retention as well as transfer.
+
*[[Elaborated Explanations|Elaborated explanations]] promote integration of visual and verbal [[knowledge component]]s during problem-solving. Thus, students who engage in this form of interactive communication (explaining verbal and visual information, and receiving feedback on all explanations) will demonstrate deeper understanding as evidenced by improved [[transfer]] and [[long-term retention]].
−
*Elaborated explanations promote integration of visual and verbal knowledge components during problem-solving, thus supporting knowledge transfer.
 
−
*Integrated hints promote [[sense making]] by promoting coordination through self-explanation and contiguity, thus supporting knowledge transfer.
 
  
 
=== Findings ===
 
=== Findings ===
  
−
Current findings suggest that interaction with visual representations during problem-solving supports deep [[transfer]] during learning. However, problem-solving performance is also supported by integrated [[self-explanation]]s where students justify their problem solving steps with both verbal reasons (geometry rules) and visual information (diagram features relevant to the named geometry rule). Thus, integration may be supported by materials or explanations that support processing of visual and verbal information during learning.
+
====Pilot Study: Paper-based Difficulty Factors Analysis ====
−
 
+
*Study Summary
−
====Study 1 (In Vivo, Geometry Cognitive Tutor) ====
+
**<b>Participants: </b> Three 10th grade geometry classes in a rural Pennsylvannia school
−
*Summary
+
**<b>Design:</b> [[In vivo experiment]]: Paper-based Difficulty Factors Analysis (DFA) covering Angles content from the Geometry Cognitive Tutor
−
**In Vivo Study: 10th grade geometry classes in rural Pennsylvannia school
+
**Problems varied along two dimensions:
−
**Domain: Angles curriculum in the Geometry Cognitive Tutor
+
***Problem Format: Diagram vs. Table. In Diagram-format problems, students entered answers in the appropriate location within the geometry diagram. In the Table format, students entered answers in a table separate from the geometry diagram.
−
**Grade-matched pairs of students were randomly assigned to one of two conditions:
+
***Explanation Type: Simple vs. [[Elaborated Explanations]]. Simple explanations required students to name only the geometry rule that justified their problem-solving steps. Elaborated explanations required students to name the geometry rule and the known diagram features that allowed them to use the stated geometry rule.
−
***Diagram (Contiguous) Condition: Students interacted directly with geometry diagrams and accepted answers are displayed directly in the diagram
+
**<b>Implementation:</b> Students using the standard version of the Geometry Cognitive Tutor took the paper-based DFA test midway through their completion of the Angles unit on the tutor.
−
***Table (Noncontiguous) Condition: Students work separate from the diagrams, in a distally located table
 
  
 
*Findings
 
*Findings
−
**No overall effect of experimental condition on students' performance on geometry answers or reasons at posttest
+
**Although there is an overall trend that, with no practice, students perform best in the familiar Table format that they were using during their Cognitive Tutor practice (F (1, 88) = 3.47, p = .07), the type of explanations that students provided significantly influenced performance (F (1, 88), = 6.75, p = .01). Students who gave elaborated explanations (the geometry rule <i>and</i> relevant diagram features) performed better on problem-solving (M = .42, SE = .04) than students who gave simple explanations (the geometry rule only, M = .34, SE = .04).
−
**Although working in the Diagram condition improved lower-knowledge students' explanations at posttest, higher-knowledge students performed best when working in the Table condition. The result was evidenced by a significant 3-way interaction of Test Time (Pre- vs. Posttest) X Condition (Table vs. Diagram) X Prior Knowledge (Higher vs. Lower) for students' performance on geometry rules at posttest (F(1,39) = 6.2, p < .02).
 
  
−
====Study 2 (In Vivo, Geometry Cognitive Tutor) ====
+
====Classroom: Elaborated Explanations with Tutored Practice ====
−
*Summary
+
*Study Summary
−
**In Vivo Study: 10th grade geometry classes in rural Pennsylvannia school
+
**<b>Participants: </b>Six 10th grade geometry classes in a rural Pennsylvannia school
−
**Domain: Circles curriculum in the Geometry Cognitive Tutor
+
**<b>Design:</b> [[In vivo experiment]]: 2 (Simple vs. Elaborated Explanations) X 2 ([[Contiguous Representations for Robust Learning (Aleven & Butcher)|Contiguous vs. Noncontiguous tutor interface]]) 
−
**Assessment was expanded to include not only answers and explanations for problem-solving items (as in Study 1), but also explanations on deep transfer items (explanations of unsolvable problems) and non-numerical reasoning items (true/false items that require students to judge whether a geometry rule is appropriate to relate named diagram elements).
+
**<b>Implementation:</b> Students were randomly assigned to one of the four experimental conditions. Time in tutor was held constant: students completed three classroom sessions with their assigned version of the tutor over a three week period. Immediate posttesting was conducted one week after the final tutor session. Delayed posttesting was completed after a four-week delay.
−
**Grade-matched pairs of students were randomly assigned to one of two conditions:
 
−
***Diagram (Contiguous) Condition: Students interacted directly with geometry diagrams and accepted answers are displayed directly in the diagram
 
−
***Table (Noncontiguous) Condition: Students work separate from the diagrams, in a distally located table
 
  
 
*Findings
 
*Findings
−
**Problem-solving: No condition differences for numerical answers (F(1, 89) = 1.03, p > .3) or explanations for solvable problems (F(1, 89) <1).
+
**<b>Practiced Skills</b> At posttest, success with problem-solving skills practiced in the tutor (finding numerical answers and stating geometry rules) were not different for any experimental condition. These practiced results are consistent with what should be expected given mastery practice in a successful intelligent tutor.
−
**Deep Transfer Explanations: There was a significant effect of condition on students' explanations of unsolvable problems (F(1, 89) = 4.1, p = .046). Students in the Diagram (Contiguous) condition explained unsolvable problems better (M = .13, SE = .03) than students in the Table (Noncontiguous) condition (M = .06, SE = .02).
+
**<b>Transfer Tasks</b>
 +
***<i>Solvability Judgments. </i> At posttest students who had interacted with diagrams tended to outperform students who had worked with tables in detecting solvable and unsolvable problems (F (1, 76) = 2.70, p = .10). The type of explanation completed did not affect solvability decisions (F < 1).
 +
***<i>Diagram-Rule Analysis. </i> These items require students to indicate the diagram features (typically one or two angles) that are used in the application of the geometry rule selected in their answer. Experimental conditions differed in whether or not they had practiced this skill in the tutor. Students in the geometry rule and diagram application explanation condition did practice this skill during tutoring; they selected a geometry rule and the relevant diagram features at every problem-solving step during practice and the tutor provided feedback for these explanations. However, students in the geometry rule only explanation condition received no practice or feedback in selecting diagram elements relevant to geometry rules.<p>As seen in Figure 3 below, students in the diagram interaction conditions performed better on diagram application items than students in the table interaction condition (F (1, 76) = 6.23, p = .02). This effect is driven by student performance in the geometry rule only explanation condition, where the diagram application items are a measure of transfer. Separate analysis of the geometry rule only explanation conditions shows that diagram interaction is powerful in supporting effective connections between visual representations and domain knowledge. When diagram application explanations are not required during practice, diagram interaction leads to significantly greater success linking diagrams to rules (F (1, 34) = 11.51, p = .002).
  
−
Figure 3. Mean performance on explanations for unsolvable problems by experimental condition, at pre- and posttest.<br>
+
Figure 3: Posttest Transfer Performance: Diagram-Rule Analysis
−
[[Image:Butcher_UnsolvableExplanations.jpg]]
+
[[Image:PosttestDiagramAnalysis_Contig3.jpg]]
 +
**<b>Delayed Posttest</b> At delayed posttest, students who worked with the interactive diagrams showed a significant advantage in the accuracy of their problem-solving answers compared to students who worked with the interactive tables (F (1, 53) = 4.03, p = .05; see Figure 4). In contrast, the type of explanation produced during practice had no effect on long-term retention of problem-solving skills.
  
−
**True/False items: Although there were no condition difference for performance on "true" items (F(1,89) = 2.4, p = .13), students in the Diagram (Contiguous) condition better recognized and explained false answers at posttest (F (1, 89) = 4.3, p = .04). That is, students from both conditions were equally able to recognize statements that gave valid relationships between geometry rules and diagram elements (Diagram, M = .71, SE = .04; Table, M = .72, SE = .03). However, students who interacted with diagrams during practice were better able to recognize when and explain why given geometry rules were inappropriate to relate named diagram elements (M = .23, SE = .02) than students who worked separately from diagrams during practice (M = .17, SE = .02).
+
Figure 4: Delayed Posttest Problem-Solving Performance
 +
[[Image:DelayPosttest_Contig3.jpg]]
  
−
Figure 4. Mean performance on recognizing/explaining inappropriate applications of geometry rules, by experimental condition at pre- and posttest.
+
=== Explanation ===
−
[[Image:Butcher_FalseExplanations.jpg]]
+
Why didn’t the addition of explanations that focused on diagram elements have more impact? One possibility may be that diagram application explanations typically were the last action performed on each problem-solving step and may have been largely redundant with processing that occurred earlier (in determining the numerical answer and geometry rule for the same step). In fact, on average, students’ diagram application explanations were correct 88% of the time. More meaningful connection of visual representations may occur as students struggle to find appropriate solutions to problems, making it necessary to embed visual interaction at these earlier, critical times.  
  
−
====Study 3 (In Vivo, Paper-based Difficulty Factors Analysis) ====
+
It is also possible that the diagram application explanations did not prompt students to attend to relevant diagram features in the way we anticipated. Students may have focused not on diagram configurations, but on the numerical values in the diagram (or table) that they used in their calculation of the numerical answer for the same step. In current work, we are exploring the addition of diagram interaction at key opportunities for learning, by supporting student highlighting of relevant diagram elements following student errors  (see [[Visual Feature Focus in Geometry: Instructional Support for Visual Coordination During Learning (Butcher & Aleven)]]).
−
*Summary
+
 
−
**In Vivo Study: 10th grade geometry classes in rural Pennsylvannia school
+
Our results demonstrate powerful support for robust learning from diagram interaction. These findings replicate and extend results from our previous research, in demonstrating that interacting with diagrams during problem-solving promotes deeper understanding of geometry principles. (See [[Coordinative Learning]], [[Contiguous Representations for Robust Learning (Aleven & Butcher)]]).
−
**Format: Paper-based Difficulty Factors Analysis (DFA) covering Angles content from the Geometry Cognitive Tutor
 
−
**Problems varied along two dimensions:
 
−
***Problem Format: Diagram vs. Table. In Diagram-format problems, students entered answers within the geometry diagram. In the Table format, students entered answers in a table separate from the geometry diagram.
 
−
***Explanation Type: Simple vs. Elaborated Explanations. Simple explanations required students to name only the geometry rule that justified their problem-solving steps. Elaborated explanations required students to name the geometry rule and the known diagram elements that allowed them to use the rule for each problem-solving step.
 
−
**Students using the standard version of the Geometry Cognitive Tutor took the paper-based DFA test midway through their completion of the Angles unit on the tutor.
 
  
−
*Findings
+
===Further Information===
−
**Although there is an overall trend that, with no practice, students perform best in the familiar Table format (F (1, 88) = 3.47, p = .07), the type of explanations required significantly impacted performance (F (1, 88), = 6.75, p = .01). Students performed better on problem-solving when required to provide elaborated explanations (M = 42., SE = .04) compared to when they produced only simple, rule-based explanations (M = .34, SE = .04).
 
  
−
=== Explanation ===
+
==== Connections ====
 +
<b>Implicit Support for Visual-Verbal Knowledge Integration</b>:<br>Our research is investigating multiple methods with which student learning can be supported by interactions with pictorial information during geometry learning. Our work also includes more implicit methods for supporting student integration visual and verbal knowledge components. These methods include [[ Contiguous Representations for Robust Learning (Aleven & Butcher) | Contiguous vs. Noncontiguous Representations]] and Integrated Hints [Link to be added].
  
−
Contiguous representations may support [[mapping]] between visual and verbal information in problem-solving. However, the deep transfer effects seen in Experiment 2 suggest that contiguous representations may help students integrate visual and verbal [[knowledge components]] during learning. In PSLC terms, the contiguous representations reduce the effort of deep learning paths in the [[learning event space]] (by supporting strategic inferences and reasoning directly with the diagram). Our data may also suggest that contiguous representations can have a learning path effect: students who are able to reason directly with diagram representations may attend more closely to the geometric features and relations to which geometry principles apply. This could impact meaningful learning by increasing [[feature validity]] of the visual and verbal [[knowledge components]].
+
<b>Specificity in Interactive Communication</b>:<br>The addition of visually-related explanations may force students to be more <i>specific</i> in explaining how a geometry principle applies to a problem-solving step. This potential interpretation is related to the hypothesis that [[elaborative interaction]] supports learning by increasing the specificity of another person's comments. In fact, [[Hausmann Diss | Hausmann & Chi]] showed that [[elaborative interaction]] improves deep learning.
  
−
Data analysis is currently underway for a completed in-vivo study that tested elaborated explanations in addition to the effect of [[contiguity]]. In-vivo study of integrated [[hint sequence]]s is planned for Spring 2007. We have evidence from the (paper-based) difficulty factors analysis that elaborated explanations support problem solving performance. We hypothesize that integrated hints also will support [[robust learning]]. Both of these manipulations, elaborated explanations and integrated hints, support [[coordination]] through [[self-explanation]] of visual and verbal information. Providing these multiple representations during learning likely affects path choice. For example, when students are required to explain the application of geometry principles using diagrams, there will be only small differences in estimated effort of shallow and deep [[strategies]] since shallow strategies are unlikely to achieve the correct answer. Further, we anticipate that elaborated explanations and integrated hints also produce path effects: the processes that students employ via path choice are more effective when the materials support use of visual and verbal information during [[sense making]]. Specifically, scaffolds or materials that support [[sense making]] with visual and verbal information should promote [[integration]].
+
<b>Application Information in Interactive Communication</b>:<br>The addition of visually-related explanations may help students by directing their attention to content that governs the application of information during problem solving. This potential interpretation is related to the hypothesis being tested by [[Ringenberg Examples-as-Help| Ringenberg & VanLehn]], that [[Completely justified example | completely justified examples]] support robust learning by demonstrating the reasoning and all application steps needed for problem-solving.
  
−
=== Annotated Bibliography ===
+
==== Annotated Bibliography ====
 +
*Butcher, K.R., & Aleven, V. (in press) Diagram Interaction during Intelligent Tutoring in Geometry: Support for Knowledge Retention and Deep Understanding. In V. Sloutsky, B. Love, & K. McRae (Eds.), Proceedings of the 30th Annual Cognitive Science Society. [http://www.learnlab.org/uploads/mypslc/publications/pp894-butcher.pdf Link to PDF]
 
*Presentation to the PSLC Advisory Board, Fall 2006. [http://www.learnlab.org/uploads/mypslc/talks/butchercontiguity_ab2006_final_distribute.ppt Link to Powerpoint slides]
 
*Presentation to the PSLC Advisory Board, Fall 2006. [http://www.learnlab.org/uploads/mypslc/talks/butchercontiguity_ab2006_final_distribute.ppt Link to Powerpoint slides]
−
*Butcher, K. B., & Aleven, V. A. (submitted). Integrating Visual and Verbal Knowledge During Classroom Learning with Computer Tutors. <i>Paper submitted to Cognitive Science 2007 Conference</i>.
+
*Presentation to the PSLC Interactive Communication Cluster, Fall 2006.
−
 
 
−
=== References ===
 
−
*Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995). Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition activities on problem solving. Cognition & Instruction, 13, 221-252.
 
−
*Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
 
−
*Chi, M. T. H., de Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477.
 
−
*Koedinger, K. R., & Anderson, J. R. (1990). Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science, 14, 511-550.
 
−
*Lovett, M. C., & Anderson, J. R. (1994). Effects of solving related proofs on memory and transfer in geometry problem solving. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 366-378.
 
−
*Mayer, R. E. (2001). Multimedia Learning. Cambridge, Cambridge University Press.
 
  
 +
==== References ====
 +
*Aleven, V., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. <i>Cognitive Science, 26(2),</i> 147-179.
 +
*Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995). Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition activities on problem solving. <i>Cognition & Instruction, 13,</i> 221-252.
 +
*Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. <i>Cognitive Science, 13,</i> 145-182.
 +
*Chi, M. T. H., de Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. <i>Cognitive Science, 18,</i> 439-477.
 +
*Koedinger, K. R., & Anderson, J. R. (1990). Abstract planning and perceptual chunks: Elements of expertise in geometry. <i>Cognitive Science, 14,</i> 511-550.
  
 
[[Category:Study]]
 
[[Category:Study]]

Latest revision as of 18:03, 12 May 2008

Using Elaborated Explanations to Support Geometry Learning

Vincent Aleven and Kirsten Butcher

Summary Tables

Pilot Study

PIs Vincent Aleven & Kirsten R. Butcher
Other Contributers Graduate Students: Andy Tzou (CMU HCII), Carl Angioli (CMU HCII), Michael Nugent (Pitt, Computer Science)

Research Programmers/Associates: Octav Popescu (Research Programmer, CMU HCII), Grace Lee Leonard (Research Associate, CMU HCII), Thomas Bolster (Research Associate, CMU HCII)

Study Start Date January 24, 2006
Study End Date January 31, 2006
LearnLab Site Central Westmoreland Career & Technology Center (CWCTC)
LearnLab Course Geometry
Number of Students 90
Total Participant Hours 90
DataShop n/a (Pencil & Paper Test)


Classroom Study

PIs Vincent Aleven & Kirsten R. Butcher
Other Contributers Graduate Students: Carl Angioli (CMU HCII), Michael Nugent (Pitt, Computer Science)

Research Programmers/Associates: Octav Popescu (Research Programmer, CMU HCII), Grace Lee Leonard (Research Associate, CMU HCII), Thomas Bolster (Research Associate, CMU HCII)

Study Start Date November 28, 2006
Study End Date January 15, 2007
LearnLab Site Central Westmoreland Career & Technology Center (CWCTC)
LearnLab Course Geometry
Number of Students 90
Total Participant Hours 540
DataShop Status: In Progress. Log files and CTAT online test files have been anonymized, passed to DataShop, and are awaiting conversion and input.


Abstract

Does integration of visual and verbal knowledge during learning support deep understanding? Can interactive communication using visual and verbal knowledge components support the development of integrated representations? In our research, we are exploring how student explanations in an intelligent tutoring environment that includes visual (pictorial) and verbal (textual) information can support robust learning. In domains like Geometry, where visual (diagram) and verbal (textual and conceptual) information is critical to deep understanding and successful problem solving, focusing student communication acts on conceptual relationships between critical knowledge components should support deep understanding. Thus, our research is investigating explanations that link the diagram features (visual knowledge components) and geometry rules (verbal knowledge components) related to successful problem-solving steps. We call these visual-verbal explanations "Elaborated Explanations." In our studies, students give a verbally-focused explanation (by stating a relevant geometry rule) as well as a visually-focused explanation (by stating the diagram features that used to apply the stated rule). The intelligent tutor provides feedback and hints on these elaborated, verbal-visual explanations. We hypothesize that this type of interactive communication will support integration of visual and verbal knowledge during learning, leading to better transfer and long-term retention.

Background & Significance

A rich body of prior research has demonstrated that students develop deeper understanding of instructional materials when they self-explain to themselves during learning (e.g., Bielaczyc, Pirolli, & Brown, 1995; Chi, Bassok, Lewis, Reimann, & Glaser, 1989; Chi, de Leeuw, Chiu, & LaVancher, 1994). Although much of this research has focused on students self-generated and natural language explanations, research has also shown benefits of simple, menu-based explanations in supporting learning. An existing version of the Geometry Cognitive Tutor implements student self-explanations in this simple manner. After correctly answering a geometry problem step, students must select the geometry rule or theorem that justifies their answer from a glossary menu of terms.

Despite the limitations of simple, menu-based explanations, they have been shown to promote student learning in the Geometry Cognitive Tutor (Aleven & Koedinger, 2002). However, these menu-based explanations may not be supporting the development of deep, expert-like connections between conceptual geometry principles and visual features of geometry problem diagrams. Students complete these explanations after they have successfully solved a numerical problem-solving step. There there is no requirement that students make explicit connections between the rule being applied and the visual features that are required for application. Because prior work has shown that experts' geometry problem solving is driven by recognition of key diagram features that cue relevant conceptual knowledge (Koedinger & Anderson, 1990), supporting deep student understanding in geometry may require that students explicitly connect relevant conceptual knowledge to visual diagram features.

Our approach is to support students' connections between and integration of visual and verbal knowledge components through interactive communication, where students' explanations of their problem solving steps reference both verbal, conceptual knowledge (in the form of a relevant geometry rule) and its relationship to key diagram features. Thus, our question is whether explicit forms of communication that link verbal and visual knowledge components can be more successful at promoting robust learning than communication that requires expression of verbal, declarative knowledge only.

Glossary

See Visual-Verbal Learning Project Glossary

Research questions

  1. Can elaborated explanations that connect visual (pictorial) and verbal (textual/conceptual) knowledge components support robust learning better than explanations that reference verbal information only?
  2. Are the effects of elaborated explanations strongest when considering long-term retention and transfer as opposed to results on a normal post-test?

Independent Variables

  • Type of Explanation
    • Simple, Textual Explanations (students state geometry principles only)

Figure 1. Screen shot of Geometry Cognitive Tutor interface with simple, verbal-only explanations.
Diagram SimpleExpl.jpg
vs.

  • Elaborated Explanations (students state geometry principles and the diagram features which allow the principle to be applied)

Figure 2. Screen shot of Geometry Cognitive Tutor interface with elaborated explanations
Diagram ElabExpl.jpg

Dependent variables

  • Normal post-test, and immediate transfer test measuring student performance on:
    • Problem-solving items isomorphic to the practiced problems (immediate retention)
    • Problem-solving items unlike those seen during problem practice (immediate transfer)
  • Delayed long-term retention posttest, measuring student performance on:
    • Problem-solving items isomorphic to the practiced problems (long-term retention)
    • Problem-solving items unlike those seen during problem practice (long-term transfer)
  • Log data collected during study-related tutor use, used to assess:

Hypothesis

  • Elaborated explanations promote integration of visual and verbal knowledge components during problem-solving. Thus, students who engage in this form of interactive communication (explaining verbal and visual information, and receiving feedback on all explanations) will demonstrate deeper understanding as evidenced by improved transfer and long-term retention.

Findings

Pilot Study: Paper-based Difficulty Factors Analysis

  • Study Summary
    • Participants: Three 10th grade geometry classes in a rural Pennsylvannia school
    • Design: In vivo experiment: Paper-based Difficulty Factors Analysis (DFA) covering Angles content from the Geometry Cognitive Tutor
    • Problems varied along two dimensions:
      • Problem Format: Diagram vs. Table. In Diagram-format problems, students entered answers in the appropriate location within the geometry diagram. In the Table format, students entered answers in a table separate from the geometry diagram.
      • Explanation Type: Simple vs. Elaborated Explanations. Simple explanations required students to name only the geometry rule that justified their problem-solving steps. Elaborated explanations required students to name the geometry rule and the known diagram features that allowed them to use the stated geometry rule.
    • Implementation: Students using the standard version of the Geometry Cognitive Tutor took the paper-based DFA test midway through their completion of the Angles unit on the tutor.
  • Findings
    • Although there is an overall trend that, with no practice, students perform best in the familiar Table format that they were using during their Cognitive Tutor practice (F (1, 88) = 3.47, p = .07), the type of explanations that students provided significantly influenced performance (F (1, 88), = 6.75, p = .01). Students who gave elaborated explanations (the geometry rule and relevant diagram features) performed better on problem-solving (M = .42, SE = .04) than students who gave simple explanations (the geometry rule only, M = .34, SE = .04).

Classroom: Elaborated Explanations with Tutored Practice

  • Study Summary
    • Participants: Six 10th grade geometry classes in a rural Pennsylvannia school
    • Design: In vivo experiment: 2 (Simple vs. Elaborated Explanations) X 2 (Contiguous vs. Noncontiguous tutor interface)
    • Implementation: Students were randomly assigned to one of the four experimental conditions. Time in tutor was held constant: students completed three classroom sessions with their assigned version of the tutor over a three week period. Immediate posttesting was conducted one week after the final tutor session. Delayed posttesting was completed after a four-week delay.
  • Findings
    • Practiced Skills At posttest, success with problem-solving skills practiced in the tutor (finding numerical answers and stating geometry rules) were not different for any experimental condition. These practiced results are consistent with what should be expected given mastery practice in a successful intelligent tutor.
    • Transfer Tasks
      • Solvability Judgments. At posttest students who had interacted with diagrams tended to outperform students who had worked with tables in detecting solvable and unsolvable problems (F (1, 76) = 2.70, p = .10). The type of explanation completed did not affect solvability decisions (F < 1).
      • Diagram-Rule Analysis. These items require students to indicate the diagram features (typically one or two angles) that are used in the application of the geometry rule selected in their answer. Experimental conditions differed in whether or not they had practiced this skill in the tutor. Students in the geometry rule and diagram application explanation condition did practice this skill during tutoring; they selected a geometry rule and the relevant diagram features at every problem-solving step during practice and the tutor provided feedback for these explanations. However, students in the geometry rule only explanation condition received no practice or feedback in selecting diagram elements relevant to geometry rules.

        As seen in Figure 3 below, students in the diagram interaction conditions performed better on diagram application items than students in the table interaction condition (F (1, 76) = 6.23, p = .02). This effect is driven by student performance in the geometry rule only explanation condition, where the diagram application items are a measure of transfer. Separate analysis of the geometry rule only explanation conditions shows that diagram interaction is powerful in supporting effective connections between visual representations and domain knowledge. When diagram application explanations are not required during practice, diagram interaction leads to significantly greater success linking diagrams to rules (F (1, 34) = 11.51, p = .002).

Figure 3: Posttest Transfer Performance: Diagram-Rule Analysis PosttestDiagramAnalysis Contig3.jpg

    • Delayed Posttest At delayed posttest, students who worked with the interactive diagrams showed a significant advantage in the accuracy of their problem-solving answers compared to students who worked with the interactive tables (F (1, 53) = 4.03, p = .05; see Figure 4). In contrast, the type of explanation produced during practice had no effect on long-term retention of problem-solving skills.

Figure 4: Delayed Posttest Problem-Solving Performance DelayPosttest Contig3.jpg

Explanation

Why didn’t the addition of explanations that focused on diagram elements have more impact? One possibility may be that diagram application explanations typically were the last action performed on each problem-solving step and may have been largely redundant with processing that occurred earlier (in determining the numerical answer and geometry rule for the same step). In fact, on average, students’ diagram application explanations were correct 88% of the time. More meaningful connection of visual representations may occur as students struggle to find appropriate solutions to problems, making it necessary to embed visual interaction at these earlier, critical times.

It is also possible that the diagram application explanations did not prompt students to attend to relevant diagram features in the way we anticipated. Students may have focused not on diagram configurations, but on the numerical values in the diagram (or table) that they used in their calculation of the numerical answer for the same step. In current work, we are exploring the addition of diagram interaction at key opportunities for learning, by supporting student highlighting of relevant diagram elements following student errors (see Visual Feature Focus in Geometry: Instructional Support for Visual Coordination During Learning (Butcher & Aleven)).

Our results demonstrate powerful support for robust learning from diagram interaction. These findings replicate and extend results from our previous research, in demonstrating that interacting with diagrams during problem-solving promotes deeper understanding of geometry principles. (See Coordinative Learning, Contiguous Representations for Robust Learning (Aleven & Butcher)).

Further Information

Connections

Implicit Support for Visual-Verbal Knowledge Integration:
Our research is investigating multiple methods with which student learning can be supported by interactions with pictorial information during geometry learning. Our work also includes more implicit methods for supporting student integration visual and verbal knowledge components. These methods include Contiguous vs. Noncontiguous Representations and Integrated Hints [Link to be added].

Specificity in Interactive Communication:
The addition of visually-related explanations may force students to be more specific in explaining how a geometry principle applies to a problem-solving step. This potential interpretation is related to the hypothesis that elaborative interaction supports learning by increasing the specificity of another person's comments. In fact, Hausmann & Chi showed that elaborative interaction improves deep learning.

Application Information in Interactive Communication:
The addition of visually-related explanations may help students by directing their attention to content that governs the application of information during problem solving. This potential interpretation is related to the hypothesis being tested by Ringenberg & VanLehn, that completely justified examples support robust learning by demonstrating the reasoning and all application steps needed for problem-solving.

Annotated Bibliography

  • Butcher, K.R., & Aleven, V. (in press) Diagram Interaction during Intelligent Tutoring in Geometry: Support for Knowledge Retention and Deep Understanding. In V. Sloutsky, B. Love, & K. McRae (Eds.), Proceedings of the 30th Annual Cognitive Science Society. Link to PDF
  • Presentation to the PSLC Advisory Board, Fall 2006. Link to Powerpoint slides
  • Presentation to the PSLC Interactive Communication Cluster, Fall 2006.

References

  • Aleven, V., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science, 26(2), 147-179.
  • Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995). Training in self-explanation and self-regulation strategies: Investigating the effects of knowledge acquisition activities on problem solving. Cognition & Instruction, 13, 221-252.
  • Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182.
  • Chi, M. T. H., de Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439-477.
  • Koedinger, K. R., & Anderson, J. R. (1990). Abstract planning and perceptual chunks: Elements of expertise in geometry. Cognitive Science, 14, 511-550.
Retrieved from "https://learnlab.org/wiki/index.php?title=Using_Elaborated_Explanations_to_Support_Geometry_Learning_(Aleven_%26_Butcher)&oldid=8033"