https://learnlab.org/research/wiki/api.php?action=feedcontributions&user=Scraig%40pitt.edu&feedformat=atomLearnLab - User contributions [en]2024-03-28T11:41:58ZUser contributionsMediaWiki 1.31.12https://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8338File:Craigetaltable1.JPG2008-10-07T16:47:14Z<p>Scraig@pitt.edu: uploaded a new version of "Image:Craigetaltable1.JPG": Reverted to version as of 16:40, 7 October 2008</p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8337File:Craigetaltable1.JPG2008-10-07T16:46:40Z<p>Scraig@pitt.edu: uploaded a new version of "Image:Craigetaltable1.JPG"</p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8336File:Craigetaltable1.JPG2008-10-07T16:44:15Z<p>Scraig@pitt.edu: uploaded a new version of "Image:Craigetaltable1.JPG"</p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8335File:Craigetaltable1.JPG2008-10-07T16:43:01Z<p>Scraig@pitt.edu: uploaded a new version of "Image:Craigetaltable1.JPG"</p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8334File:Craigetaltable1.JPG2008-10-07T16:40:41Z<p>Scraig@pitt.edu: uploaded a new version of "Image:Craigetaltable1.JPG"</p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8333File:Craigetaltable1.JPG2008-10-07T16:39:59Z<p>Scraig@pitt.edu: uploaded a new version of "Image:Craigetaltable1.JPG"</p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8332Craig observing2008-10-07T16:35:15Z<p>Scraig@pitt.edu: /* Hypothesis */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
There are two contrasting hypotheses being tested in this design. The hypothesis that we have spent the most time with in the current paper is the active observing hypothesis. The active observing hypothesis would predict that the learners in the collaboratively observing tutoring condition would outperform other conditions because of the highly dynamic tutoring session So, the tutoring videos would contain dialogue features (e.g. turn taking, pauses, and affect) and expert tutoring elements (e.g. corrections and scaffolding) that would promote more active engagement with the video material than the passive information display from the worked example. <br><br />
Collaboratively observing tutoring > Collaboratively observing examples = Individually observing tutoring (1)<br />
<br>However, a potential alternative hypothesis (content equivalency hypothesis) is that the content what really matters. Since we have given learners equal content, then the method in which the material is presented should not matter (Klahr & Nigam, 2004). <br><br />
Collaboratively observing tutoring = Collaboratively observing examples = Individually observing tutoring (2)<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.57 and ''M'' = .67 respectively. No differences between groups were found for immediate learning.<br><br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable2.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8331Craig observing2008-10-07T16:34:34Z<p>Scraig@pitt.edu: /* '''Far transfer data''' (robust learning). */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
There are two contrasting hypotheses being tested in this design. The hypothesis that we have spent the most time with in the current paper is the active observing hypothesis. The active observing hypothesis would predict that the learners in the collaboratively observing tutoring condition would outperform other conditions because of the highly dynamic tutoring session So, the tutoring videos would contain dialogue features (e.g. turn taking, pauses, and affect) and expert tutoring elements (e.g. corrections and scaffolding) that would promote more active engagement with the video material than the passive information display from the worked example. <br><br />
Collaboratively observing tutoring > Collaboratively observing examples = Individually observing tutoring (1)<br />
However, a potential alternative hypothesis (content equivalency hypothesis) is that the content what really matters. Since we have given learners equal content, then the method in which the material is presented should not matter (Klahr & Nigam, 2004). <br><br />
Collaboratively observing tutoring = Collaboratively observing examples = Individually observing tutoring (2)<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.57 and ''M'' = .67 respectively. No differences between groups were found for immediate learning.<br><br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable2.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8330Craig observing2008-10-07T16:30:40Z<p>Scraig@pitt.edu: /* '''Immediate Learning measures''' */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
There are two contrasting hypotheses being tested in this design. The hypothesis that we have spent the most time with in the current paper is the active observing hypothesis. The active observing hypothesis would predict that the learners in the collaboratively observing tutoring condition would outperform other conditions because of the highly dynamic tutoring session So, the tutoring videos would contain dialogue features (e.g. turn taking, pauses, and affect) and expert tutoring elements (e.g. corrections and scaffolding) that would promote more active engagement with the video material than the passive information display from the worked example. <br><br />
Collaboratively observing tutoring > Collaboratively observing examples = Individually observing tutoring (1)<br />
However, a potential alternative hypothesis (content equivalency hypothesis) is that the content what really matters. Since we have given learners equal content, then the method in which the material is presented should not matter (Klahr & Nigam, 2004). <br><br />
Collaboratively observing tutoring = Collaboratively observing examples = Individually observing tutoring (2)<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.57 and ''M'' = .67 respectively. No differences between groups were found for immediate learning.<br><br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8329Craig observing2008-10-07T16:28:56Z<p>Scraig@pitt.edu: /* '''Immediate Learning measures''' */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
There are two contrasting hypotheses being tested in this design. The hypothesis that we have spent the most time with in the current paper is the active observing hypothesis. The active observing hypothesis would predict that the learners in the collaboratively observing tutoring condition would outperform other conditions because of the highly dynamic tutoring session So, the tutoring videos would contain dialogue features (e.g. turn taking, pauses, and affect) and expert tutoring elements (e.g. corrections and scaffolding) that would promote more active engagement with the video material than the passive information display from the worked example. <br><br />
Collaboratively observing tutoring > Collaboratively observing examples = Individually observing tutoring (1)<br />
However, a potential alternative hypothesis (content equivalency hypothesis) is that the content what really matters. Since we have given learners equal content, then the method in which the material is presented should not matter (Klahr & Nigam, 2004). <br><br />
Collaboratively observing tutoring = Collaboratively observing examples = Individually observing tutoring (2)<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.57 and ''M'' = .67 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br><br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8328Craig observing2008-10-07T16:21:23Z<p>Scraig@pitt.edu: /* Hypothesis */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
There are two contrasting hypotheses being tested in this design. The hypothesis that we have spent the most time with in the current paper is the active observing hypothesis. The active observing hypothesis would predict that the learners in the collaboratively observing tutoring condition would outperform other conditions because of the highly dynamic tutoring session So, the tutoring videos would contain dialogue features (e.g. turn taking, pauses, and affect) and expert tutoring elements (e.g. corrections and scaffolding) that would promote more active engagement with the video material than the passive information display from the worked example. <br><br />
Collaboratively observing tutoring > Collaboratively observing examples = Individually observing tutoring (1)<br />
However, a potential alternative hypothesis (content equivalency hypothesis) is that the content what really matters. Since we have given learners equal content, then the method in which the material is presented should not matter (Klahr & Nigam, 2004). <br><br />
Collaboratively observing tutoring = Collaboratively observing examples = Individually observing tutoring (2)<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br><br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8327Craig observing2008-10-07T16:19:16Z<p>Scraig@pitt.edu: /* '''Immediate Learning measures''' */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br><br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8326Craig observing2008-10-07T16:18:49Z<p>Scraig@pitt.edu: </p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
[[Image:Craigetaltable1.JPG]]<br />
<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable2.JPG&diff=8325File:Craigetaltable2.JPG2008-10-07T16:17:13Z<p>Scraig@pitt.edu: </p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:Craigetaltable1.JPG&diff=8324File:Craigetaltable1.JPG2008-10-07T16:16:55Z<p>Scraig@pitt.edu: </p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8323Craig observing2008-10-07T16:16:26Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
===='''Immediate Learning measures'''====<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8322Craig observing2008-10-07T16:07:55Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, Eta sqared = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, Eta sqared = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, Eta sqared = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8321Craig observing2008-10-07T16:07:00Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, 2 = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)====<br />
An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, 2 = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning).====<br />
An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, 2 = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8320Craig observing2008-10-07T16:06:16Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, 2 = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
<br />
===='''Near transfer data.''' (robust learning)==== An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, 2 = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
<br />
===='''Far transfer data''' (robust learning)====. An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, 2 = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8319Craig observing2008-10-07T16:05:51Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, 2 = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
===='''Near transfer data.''' (robust learning)==== An ANOVA was conducted on the participants’ near transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.39, p < .05, 2 = 0.129. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05). <br />
===='''Far transfer data''' (robust learning)====. An ANOVA was run on the participants far transfer data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 4.89, p < .05, 2 = 0.142. Pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring condition significantly outperformed participants in both the collaboratively observing examples condition (p < .01) and the individually observing examples condition (p < .05).<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8318Craig observing2008-10-07T16:02:55Z<p>Scraig@pitt.edu: /* '''Long term learning measures''' (robust learning) */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
===='''Long term learning measures''' (robust learning)====<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, 2 = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8317Craig observing2008-10-07T16:02:05Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
=='''Long term learning measures''' (robust learning)==<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, 2 = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8316Craig observing2008-10-07T16:01:33Z<p>Scraig@pitt.edu: /* Results */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
'''Long term learning measures''' (robust learning)<br />
Long term retention data. An ANOVA was performed on the participants’ long term retention data to determine differences among groups. This analysis revealed a significant effect of among conditions, F(2, 59) = 3.44, p < .05, 2 = 0.104; pairwise comparisons using LSD tests for main effects revealed that participants in the collaboratively observing tutoring conditions significantly outperformed learners in the collaboratively observing examples condition (p < .05) and the individually observing examples condition (p < .05). <br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8315Craig observing2008-10-07T15:59:39Z<p>Scraig@pitt.edu: /* Abstract */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked examples]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8314Craig observing2008-10-07T15:59:13Z<p>Scraig@pitt.edu: /* Abstract */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked example]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked example]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8313Craig observing2008-10-07T15:58:55Z<p>Scraig@pitt.edu: /* Abstract */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked example]] video during problem solving in Andes ([[Collaboratively observing]] [[example]]s condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8312Craig observing2008-10-07T15:58:27Z<p>Scraig@pitt.edu: /* Abstract */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaboratively observing|collaborative observers]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked example]] video during problem solving in Andes ([[Collaboratively observing]] [[examples]] condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8311Craig observing2008-10-07T15:57:50Z<p>Scraig@pitt.edu: /* Abstract */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s in the Phsics LearnLab on the principles of rotational kinematics. The study reported here took this observational learning methodology into the classroom and tested the active observing hypothesis. In doing so, we compared [[collaborative observers|collaboratively observing]] of tutoring videos during problem solving in Andes (Collaboratively observing tutoring condition) against two control conditions that received [[worked examples]]. So the tutoring videos showed an expert human tutor helping undergraduates solve problems, while the [[worked examples]] videos showed the expert tutor solving problems while orally describing the steps and reasoning. The first control condition required pairs of students to collaboratively observe a [[worked example]] video during problem solving in Andes ([[Collaboratively observing]] [[examples]] condition). The second condition, individually observing examples condition, was comprised of individual students viewing a worked example video alone while problem solving in Andes. Since the [[Andes]] system provides video explanations for the learners on select problems, this control was analogous to the help normally provided in the course. Since both Chi et al. (2008) and Craig et al. (2004) did not find learning gains for individuals observing tutoring, the individually observing of tutoring condition was not taken into the classroom in order to avoid exposing students to an ineffectual learning condition. <br />
<br />
In the experimental conditions, students collaboratively observed videos. The videos showed either a tutoring session or worked examples. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. While these immediate learning measures did not display group differences, our long-term and transfer learning measures showed consistent differences in favor of collaboratively observing tutoring.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8310Craig observing2008-10-07T15:42:19Z<p>Scraig@pitt.edu: /* Independent variables */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s. Previous laboratory research has shown that learners who watch a video of a problem solving tutoring session while collaboratively solving the same problems with a partner learn significantly more than learners that watched the video and solved the problems alone (Chi, Hausmann, & Roy, in press). In this study, the [[robustness]] of this effect was tested in the Physics learnlab. Because Chi et al. also found that videos of competent tutees caused more learning in the observers than videos of less competent tutees, this experiment include a condition where observers viewed a video of [[worked examples]], which is the extreme case of a problem being solved by a completely competent "student." <br />
<br />
In the experimental conditions, students collaboratively observed videos on the principles of rotational kinematics. The videos showed either a tutoring session or worked examples. The tutoring videos showed an expert human tutor helping undergraduates solve problems. The [[worked examples]] video showed the expert tutor solving problems while orally describing the steps and reasoning. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. On Andes homework problems done days later (a retention and medium transfer measure), the pairs observing tutoring scored higher than the pairs observing worked examples and the solos observing worked examples.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
'''Figure 1. '''Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot1.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:CraigetalAndesscreenshot1.JPG&diff=8309File:CraigetalAndesscreenshot1.JPG2008-10-07T15:41:03Z<p>Scraig@pitt.edu: </p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=8308Craig observing2008-10-07T15:40:37Z<p>Scraig@pitt.edu: /* Independent variables */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s. Previous laboratory research has shown that learners who watch a video of a problem solving tutoring session while collaboratively solving the same problems with a partner learn significantly more than learners that watched the video and solved the problems alone (Chi, Hausmann, & Roy, in press). In this study, the [[robustness]] of this effect was tested in the Physics learnlab. Because Chi et al. also found that videos of competent tutees caused more learning in the observers than videos of less competent tutees, this experiment include a condition where observers viewed a video of [[worked examples]], which is the extreme case of a problem being solved by a completely competent "student." <br />
<br />
In the experimental conditions, students collaboratively observed videos on the principles of rotational kinematics. The videos showed either a tutoring session or worked examples. The tutoring videos showed an expert human tutor helping undergraduates solve problems. The [[worked examples]] video showed the expert tutor solving problems while orally describing the steps and reasoning. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. On Andes homework problems done days later (a retention and medium transfer measure), the pairs observing tutoring scored higher than the pairs observing worked examples and the solos observing worked examples.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
Figure 1. Schreenshot of Andes system. <br><br />
[[Image:CraigetalAndesscreenshot.JPG]]<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=File:CraigetalAndesscreenshot.JPG&diff=8307File:CraigetalAndesscreenshot.JPG2008-10-07T15:35:43Z<p>Scraig@pitt.edu: </p>
<hr />
<div></div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_questions&diff=7295Craig questions2008-03-18T22:03:59Z<p>Scraig@pitt.edu: /* Annotated bibliography */</p>
<hr />
<div>== Investigating the robustness of vicarious learning: Sense making with deep-level reasoning questions ==<br />
Scotty Craig, Kurt VanLehn, and Micki Chi''<br />
<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || 12-1-05<br />
|-<br />
| '''Study End Date''' || 8-1-06<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 17<br />
|-<br />
| '''Total Participant Hours''' || 24 hrs.<br />
|-<br />
| '''DataShop''' || Target date: June 15, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
Earlier work (Craig, at al. 2006; Gholson & Craig, in press) found that inserting relevant [[deep-level question]]s into observed video material both increased deep level question asking and improved learning. These lab studies had student learn topics in computer literacy by viewing videos of both monologues and dialogues; some material included [[deep-level question]]s, some included shallow questions and some included no questions. The conditions that included [[deep-level question]]s learned more than the others. However, it is not known how this method works compared to other methods for enhancing learning from observed materials (e.g. prompting for [[self-explanation]]). It is also not known if this effect can be useful for learning outside the lab setting. <br />
<br />
Our [[in vivo experiment]] presented identical core content on magnetism using example problems from the Andes tutoring system in three different ways. The material was presented in three formats. All three of these formats were presented as a video of a [[worked examples|worked example]] with each step corresponding to a [[knowledge component]]. The [[knowledge components]] were preceded by a [[deep-level question]] (e.g. What are the implications of having the magnetic field close to an electrified wire?), a prompt for learners to reflection on the material (i.e. a pause in the video) or a [[self-explanation]] prompt (e.g. Please begin your self explanation). Measures of [[Andes]] transfer, and long term [[robust learning]] were measured. The learners’ interaction with Andes were coded for differences on completion time, within task behavior, and the completion rates of the Andes homework.<br />
<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a difference found in the amount of time needed to complete homework. This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition. However, both of these findings are difficult to interprete given that there was an average of 39 days between initial training and homework completion by the learners.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig questions|Craig deep-level questions Glossary]]<br />
<br />
=== Research question ===<br />
Is robust learning better achieved by observing multimedia displays integrated with [[deep-level question]]s, prompts for [[reflection questions|reflection]], or [[self-explanation]]?<br />
<br />
=== Independent variables ===<br />
The current study varied the level of guidance provided. The level of guidance was varied by presenting students with a deep-level reasoning condition, a self-explanation condition and a reflection condition. The deep-level reasoning questions provided a step-by-step guide that scaffolded the learner during the learning process. The self-explanation condition asked that students build the links of these scaffolds by self-explaining the steps. As a control for time on task, the reflection condition presented materials to the participants with a pause before each step.<br />
<br />
'''Examples for each condition'''<br />
{| border="1" cellspacing="0" cellpadding="0" style="text-align: left;"<br />
| <br />
|-<br />
| ''Deep-level question'' || ''Self Explanation'' || ''Reflection''<br />
|-<br />
| What effect does a straight current-carrying wire have on magnetic field lines? || Please begin your self-explanation || Pause for 10 seconds<br />
|-<br />
| ''Corresponding Example text''<br />
|-<br />
| Magnetic field lines near a straight current-carrying wire take the form of <br />
concentric circles with the wire at their center <br />
|}<br />
<br><br />
<br />
=== Hypothesis ===<br />
A guided learning hypothesis would predict that since the deep-level questions provided a constant cognitive guide the deep-level question condition would improve learning over the reflection condition and possibly the self-explanation condition if the students could not produce the guidance while producing the self-explanations. Alternatively, a content equivalency hypothesis would be that since all three conditions provide the same content they should all produce learning of the material (Klahr & Nigam, 2004).<br />
<br />
=== Dependent variables ===<br />
<br />
* ''[[Long-term retention]], homework on Andes'': After training, students did their regular homework problems using Andes. Students could do them whenever they wanted, but most students normally completed them just before the exam (''M'' = 39 days after training). The more similar homework problems (near transfer) were analyzed.<br />
<br />
=== Results ===<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a marginally significant trend found on the completion time data in favor of participants in the deep-level question condition over those in the reflection condition (t (9) = 2.14, p = .07). This difference for completion time became significant when participants in the two unguided conditions were collapsed and compared against participants in the guided condition (t (15) = 2.41, p < .05). This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition.<br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met: <br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the deep-level question condition and the self-explanation condition could comprise the learning-by-doing paths in that learners are guided to produce clearer mental models of the material. Alternatively the participants in the reflection condition only received pauses during the presentation, thus these participants were not guided to produce better mental models. These participants relied more on the video to provide relevant links for them instead of actively constructing these links.<br />
* The student should take the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with deep-level questions that guide them in building better mental models. However, the self-explanation and reflection conditions require the students to produce the learning by doing path. In these conditions, if the production becomes too difficult for the students then they will not learn. This study is testing whether students will learn more by being encouraged to take a learning-by-doing path, via deep-level questions, than an alternative path. Since none of the students attempted more than a few self-explanations, it appears that the students in the self-explanation conditions did not take the learning-by-doing path.<br />
<br />
<br />
=== Annotated bibliography ===<br />
* Presented at LRDC Supergroup meeting July, 2006<br />
* Presented at PSLC Roadshow - Memphis November, 2006<br />
* Presented at LRDC Graduate student recruitment - Pittsburgh Feburary, 2007<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007, April). PSLC AERA Symposium: In vivo experimentation for understanding robust learning: Pros and cons.<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007). PSLC EARLI Symposium.<br />
* Craig, S. D., VanLehn, K., & Chi. M.T.H. (2008). Promoting learning by observing deep-level reasoning questions on quantitative physics problem solving with Andes. In K. McFerrin, R. Weber, R. Weber, R. Carlsen, & D.A. Willis (Eds.). The proceedings of the 19th International conference for the Society for Information Technology & Teacher Education. (pp. 1065-1068). Chesapeake, VA: AACE.<br />
<br />
=== References ===<br />
<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (under revision). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. ''Cognitive Science, 13'', 145-182.<br />
* Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. ''Cognitive Science, 18'', 439-477.<br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Use of Questions during learning<br />
* [[Reflective_Dialogues_%28Katz%29 | Reflective Dialogues (Katz)]]<br />
* [[Post-practice reflection (Katz) | Post-practice reflection (Katz)]]<br />
* [[FrenchCulture | FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]<br />
<br />
Self explanations during learning<br />
* [[Hausmann Study2 | The Effects of Interaction on Robust Learning (Hausmann & Chi)]]<br />
* [[Hausmann Study | A comparison of self-explanation to instructional explanation (Hausmann & Vanlehn)]]<br />
<br />
Learning by Observing<br />
* [[Craig_observing | Learning from Problem Solving while Observing Worked Examples (Craig, Gadgil, & Chi)]]<br />
--[[User:Scraig@pitt.edu|Scotty]] 12:12, 19 September 2006 (EDT)</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_questions&diff=6686Craig questions2008-01-09T17:32:15Z<p>Scraig@pitt.edu: /* Annotated bibliography */</p>
<hr />
<div>== Investigating the robustness of vicarious learning: Sense making with deep-level reasoning questions ==<br />
Scotty Craig, Kurt VanLehn, and Micki Chi''<br />
<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || 12-1-05<br />
|-<br />
| '''Study End Date''' || 8-1-06<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 17<br />
|-<br />
| '''Total Participant Hours''' || 24 hrs.<br />
|-<br />
| '''DataShop''' || Target date: June 15, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
Earlier work (Craig, at al. 2006; Gholson & Craig, in press) found that inserting relevant [[deep-level question]]s into observed video material both increased deep level question asking and improved learning. These lab studies had student learn topics in computer literacy by viewing videos of both monologues and dialogues; some material included [[deep-level question]]s, some included shallow questions and some included no questions. The conditions that included [[deep-level question]]s learned more than the others. However, it is not known how this method works compared to other methods for enhancing learning from observed materials (e.g. prompting for [[self-explanation]]). It is also not known if this effect can be useful for learning outside the lab setting. <br />
<br />
Our [[in vivo experiment]] presented identical core content on magnetism using example problems from the Andes tutoring system in three different ways. The material was presented in three formats. All three of these formats were presented as a video of a [[worked examples|worked example]] with each step corresponding to a [[knowledge component]]. The [[knowledge components]] were preceded by a [[deep-level question]] (e.g. What are the implications of having the magnetic field close to an electrified wire?), a prompt for learners to reflection on the material (i.e. a pause in the video) or a [[self-explanation]] prompt (e.g. Please begin your self explanation). Measures of [[Andes]] transfer, and long term [[robust learning]] were measured. The learners’ interaction with Andes were coded for differences on completion time, within task behavior, and the completion rates of the Andes homework.<br />
<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a difference found in the amount of time needed to complete homework. This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition. However, both of these findings are difficult to interprete given that there was an average of 39 days between initial training and homework completion by the learners.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig questions|Craig deep-level questions Glossary]]<br />
<br />
=== Research question ===<br />
Is robust learning better achieved by observing multimedia displays integrated with [[deep-level question]]s, prompts for [[reflection questions|reflection]], or [[self-explanation]]?<br />
<br />
=== Independent variables ===<br />
The current study varied the level of guidance provided. The level of guidance was varied by presenting students with a deep-level reasoning condition, a self-explanation condition and a reflection condition. The deep-level reasoning questions provided a step-by-step guide that scaffolded the learner during the learning process. The self-explanation condition asked that students build the links of these scaffolds by self-explaining the steps. As a control for time on task, the reflection condition presented materials to the participants with a pause before each step.<br />
<br />
'''Examples for each condition'''<br />
{| border="1" cellspacing="0" cellpadding="0" style="text-align: left;"<br />
| <br />
|-<br />
| ''Deep-level question'' || ''Self Explanation'' || ''Reflection''<br />
|-<br />
| What effect does a straight current-carrying wire have on magnetic field lines? || Please begin your self-explanation || Pause for 10 seconds<br />
|-<br />
| ''Corresponding Example text''<br />
|-<br />
| Magnetic field lines near a straight current-carrying wire take the form of <br />
concentric circles with the wire at their center <br />
|}<br />
<br><br />
<br />
=== Hypothesis ===<br />
A guided learning hypothesis would predict that since the deep-level questions provided a constant cognitive guide the deep-level question condition would improve learning over the reflection condition and possibly the self-explanation condition if the students could not produce the guidance while producing the self-explanations. Alternatively, a content equivalency hypothesis would be that since all three conditions provide the same content they should all produce learning of the material (Klahr & Nigam, 2004).<br />
<br />
=== Dependent variables ===<br />
<br />
* ''[[Long-term retention]], homework on Andes'': After training, students did their regular homework problems using Andes. Students could do them whenever they wanted, but most students normally completed them just before the exam (''M'' = 39 days after training). The more similar homework problems (near transfer) were analyzed.<br />
<br />
=== Results ===<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a marginally significant trend found on the completion time data in favor of participants in the deep-level question condition over those in the reflection condition (t (9) = 2.14, p = .07). This difference for completion time became significant when participants in the two unguided conditions were collapsed and compared against participants in the guided condition (t (15) = 2.41, p < .05). This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition.<br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met: <br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the deep-level question condition and the self-explanation condition could comprise the learning-by-doing paths in that learners are guided to produce clearer mental models of the material. Alternatively the participants in the reflection condition only received pauses during the presentation, thus these participants were not guided to produce better mental models. These participants relied more on the video to provide relevant links for them instead of actively constructing these links.<br />
* The student should take the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with deep-level questions that guide them in building better mental models. However, the self-explanation and reflection conditions require the students to produce the learning by doing path. In these conditions, if the production becomes too difficult for the students then they will not learn. This study is testing whether students will learn more by being encouraged to take a learning-by-doing path, via deep-level questions, than an alternative path. Since none of the students attempted more than a few self-explanations, it appears that the students in the self-explanation conditions did not take the learning-by-doing path.<br />
<br />
<br />
=== Annotated bibliography ===<br />
* Presented at LRDC Supergroup meeting July, 2006<br />
* Presented at PSLC Roadshow - Memphis November, 2006<br />
* Presented at LRDC Graduate student recruitment - Pittsburgh Feburary, 2007<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007, April). PSLC AERA Symposium: In vivo experimentation for understanding robust learning: Pros and cons.<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007). PSLC EARLI Symposium.<br />
* Craig, S. D., VanLehn, K., & Chi. M.T.H. (2008). Promoting learning by observing deep-level reasoning questions on quantitative physics problem solving with Andes. The 19th International conferences for the society for Information technology & teacher education.<br />
<br />
=== References ===<br />
<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (under revision). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. ''Cognitive Science, 13'', 145-182.<br />
* Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. ''Cognitive Science, 18'', 439-477.<br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Use of Questions during learning<br />
* [[Reflective_Dialogues_%28Katz%29 | Reflective Dialogues (Katz)]]<br />
* [[Post-practice reflection (Katz) | Post-practice reflection (Katz)]]<br />
* [[FrenchCulture | FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]<br />
<br />
Self explanations during learning<br />
* [[Hausmann Study2 | The Effects of Interaction on Robust Learning (Hausmann & Chi)]]<br />
* [[Hausmann Study | A comparison of self-explanation to instructional explanation (Hausmann & Vanlehn)]]<br />
<br />
Learning by Observing<br />
* [[Craig_observing | Learning from Problem Solving while Observing Worked Examples (Craig, Gadgil, & Chi)]]<br />
--[[User:Scraig@pitt.edu|Scotty]] 12:12, 19 September 2006 (EDT)</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=6128Craig observing2007-10-15T16:34:49Z<p>Scraig@pitt.edu: /* Dependent variables */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s. Previous laboratory research has shown that learners who watch a video of a problem solving tutoring session while collaboratively solving the same problems with a partner learn significantly more than learners that watched the video and solved the problems alone (Chi, Hausmann, & Roy, in press). In this study, the [[robustness]] of this effect was tested in the Physics learnlab. Because Chi et al. also found that videos of competent tutees caused more learning in the observers than videos of less competent tutees, this experiment include a condition where observers viewed a video of [[worked examples]], which is the extreme case of a problem being solved by a completely competent "student." <br />
<br />
In the experimental conditions, students collaboratively observed videos on the principles of rotational kinematics. The videos showed either a tutoring session or worked examples. The tutoring videos showed an expert human tutor helping undergraduates solve problems. The [[worked examples]] video showed the expert tutor solving problems while orally describing the steps and reasoning. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. On Andes homework problems done days later (a retention and medium transfer measure), the pairs observing tutoring scored higher than the pairs observing worked examples and the solos observing worked examples.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_observing&diff=6127Craig observing2007-10-15T16:33:54Z<p>Scraig@pitt.edu: /* Dependent variables */</p>
<hr />
<div>== Learning from Problem Solving while Observing Worked Examples ==<br />
''Scotty Craig, Soniya Gadgil, Kurt VanLehn, and Micki Chi''<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || Sept. 1, 2006<br />
|-<br />
| '''Study End Date''' || Aug. 31, 2007<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 64<br />
|-<br />
| '''Total Participant Hours''' || 128 hrs.<br />
|-<br />
| '''DataShop''' || Target date: April 30, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
This research project investigated why students learn from [[collaboratively observing]] [[example]]s. Previous laboratory research has shown that learners who watch a video of a problem solving tutoring session while collaboratively solving the same problems with a partner learn significantly more than learners that watched the video and solved the problems alone (Chi, Hausmann, & Roy, in press). In this study, the [[robustness]] of this effect was tested in the Physics learnlab. Because Chi et al. also found that videos of competent tutees caused more learning in the observers than videos of less competent tutees, this experiment include a condition where observers viewed a video of [[worked examples]], which is the extreme case of a problem being solved by a completely competent "student." <br />
<br />
In the experimental conditions, students collaboratively observed videos on the principles of rotational kinematics. The videos showed either a tutoring session or worked examples. The tutoring videos showed an expert human tutor helping undergraduates solve problems. The [[worked examples]] video showed the expert tutor solving problems while orally describing the steps and reasoning. In the control condition, students viewed the [[worked examples]] video alone, without a collaborating peer. The same problems were shown in all videos. The [[Andes]] system was used throughtout the experiment both as the backdrop for the two sets of videos and by the students who solved Andes problems both during training and as transfer assesments. In summary, three conditions for the current study were: collaboratively observing tutoring, [[collaboratively observing]] [[worked examples]], and [[individually observing]] [[worked examples]].<br />
<br />
Analyses have been conducted on immediate learning (Normal pretest/posttest, near transfer) and retention measures. No differences between groups were found for immediate learning. On Andes homework problems done days later (a retention and medium transfer measure), the pairs observing tutoring scored higher than the pairs observing worked examples and the solos observing worked examples.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig observing|Craig Observing tutoring Glossary]]<br />
<br />
=== Research question ===<br />
How is robust learning affected by collaboratively versus individually observing different types of worked examples?<br />
<br />
=== Independent variables ===<br />
The current study varied both number of observers and type of video observed. The multiple-observer variable consisted of two participants observing a video while problem solving or an individual participant watching a video while problem solving -- see [[collaboration]]. Information presentation format was used to manipulate the example type variable. Participants watched one of two videos. They either watched an expert worked example of Andes problem solving that provided the solution steps for Andes problems along with information on why the steps where needed. Alternatively, they watched a tutoring session where a human tutor worked with a tutee to help solve the Andes problems -- see [[vicarious learning]]. Since this study was conducted in the learnlab, the condition where an individual observed the tutoring session was eliminated because previous lab studies have not shown this contrast to be effective.<br />
<br />
=== Hypothesis ===<br />
A dialogue hypothesis for collaboratively observing while problem solving from worked examples is that viewing the expert tutoring session should produce more learning (normal or robust) than viewing a content equivalent condition of expert problem solving. However, an alternative (Content equivalence hypothesis) is that since the expert tutoring session and the expert worked example both provide good learning conditions with the same content they should both produce mastery of the material (Klahr & Nigam, 2004). Process data collected in this study will help to tease out these hypotheses.<br />
<br />
=== Dependent variables ===<br />
* ''[[Transfer]], immediate'': After exposure to the treatment, students completed three transfer problems in Andes. These problems will test the same concepts from training in new situations that require implementation of the problems in new ways. <br />
<br />
* ''[[Normal post-test]]'': Students were given a 12 item multiple choice pretest and posttest that taps into their ability to apply the principles of rotational kinematics to new situations. This served as a measure of immediate learning for the study.<br />
<br />
* ''Homework as [[long-term retention]] and [[transfer]] items'': After training, students completed their regular homework problems using Andes. Students could do them whenever they want, but most normally complete them just before the exam. The homework problems were divided based on similarity to the training problems. Homework for both similar (near transfer) and dissimilar (far transfer) problems will be analyzed. <br />
<br />
* ''[[Accelerated future learning]]'': The training was on Rotational kinematics, and it was followed in the course by a unit on Rotational Dynamics. Andes log files from this homework will be analyzed as a measure of acceleration of future learning.<br />
<br />
1. University level: Talk to Cathy and Sussie at Research support<br />
** Set up application with Cathy (give her a copy of the call)<br />
** Set up preliminary budget with sussie<br />
** Set up preliminary IRB for Grant with Sussie<br />
<br />
2. MSC level: Hernandez contact person<br />
** fill out research application form<br />
** get letter (transfer to PDF format) of support from schools for Appendix A<br />
<br />
3. ALEX corp<br />
** get letter of support (transfer to PDF format) for appendix A<br />
<br />
=== Results ===<br />
Preliminary analyses have been conducted on immediate learning (MC pretest/posttest, Andes problem solving) and long-term retention measures. An analysis of the data has yielded significant learning gains between pretest to posttest, ''F'' (1, 65) = 14.99, ''p'' <.001 with a proportional ''M'' =.56 and ''M'' = .66 respectively. No differences between groups were found for immediate learning. However, data from Andes problem solving while completing homework (long-term retention measure, medium transfer) have showed significant differences among groups, ''F'' (1, 60) = 3.47, ''p'' <.05 in which the collaboratively observing tutoring (''M'' = .88) pairs performed significantly better than the collaboratively observing worked examples condition (''M'' = .75) and the individually observing worked examples condition(''M'' = .73).<br />
<br />
<br />
'''Table'''. “Means and standard deviations for long term retention measure of Andes problem solving”.<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''Condition''' || ''' M''' || '''SD'''<br />
|-<br />
| '''Collaboratively Observing Tutoring''' || .88 || .136<br />
|-<br />
| '''Collaboratively Observing worked example''' || .75 || .258<br />
|-<br />
| ''' Individually Observing worked example ''' || .73 || .175<br />
|}<br />
<br><br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met. Specific explanations for the current study follow.<br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the collaboration conditions could comprise the learning-by-doing paths where learners can work together to complete the Andes problems or the paired learners could rely on the video as their information providing agent and simply copy the steps. Alternatively the participants in the solo condition would have to rely exclusively on the video for information and thus rely on more direct copying of steps thus allowing another agent (the video) to do most of the work. In this case, both learning conditions offer the alternate copying path. However, copying could differ in frequency and be more likely to be discouraged in the collaborative condition due to the more social nature of the task.<br />
* The student takes the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with the tutorial dialogue that could encourage communication or an expert worked example that gives a walk through of the problem without the dialogue interaction. So, in the conditions where students are more likely to take the learning-by-doing path (the tutoring dialogue conditions), they are more likely to learn more, as compared to the conditions where they are more likely to take an alternative path (in the expert worked example conditions).<br />
<br />
=== Annotated bibliography ===<br />
* Craig, S., Vanlehn, K., Gadgil, S., & Chi, M. (2007). Learning from Collaboratively Observing during problem solving with videos. AIED07: 13th International Conference on Artificial Intelligence in Education, Los Angeles, CA. [http://andes3.lrdc.pitt.edu/~scraig/publications/AIED_Observer_learning_LearnLab.pdf]<br />
<br />
=== References ===<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Collaboration during learning<br />
* [[Hausmann_Study2 | The Effects of Interaction on Robust Learning ]]<br />
* [[Walker_A_Peer_Tutoring_Addition| Collaborative Extensions to the Cognitive Tutor Algebra: A Peer Tutoring Addition ]]<br />
* [[Rummel_Scripted_Collaborative_Problem_Solving | Collaborative Extensions to the Cognitive Tutor Algebra: Scripted Collaborative Problem Solving ]]<br />
<br />
Worked examples and learning<br />
* [[Does_learning_from_worked-out_examples_improve_tutored_problem_solving%3F | Does learning from worked-out examples improve tutored problem solving? ]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Vicarious_learning&diff=5223Vicarious learning2007-05-25T17:10:50Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Vicarious Learning, although originally coined by Bandura (1962) to refer to learning of behavior (e.g., aggression) form watching videos of that behavior, it is used here to refer to a [[instructional method]] that occurs when learners see and/or hear a learning situation (i.e., a observed learner in an instructional situation) for which they are not the addressees and do not interact with the observed learner nor the observed learner's instruction(Gholson & Craig, 2006; Rosenthal & Zimmerman, 1978). Although the learning situation is often presented as video recordings of human interactions or as cartoon-like recreations of learning situations (Bandura, 1986), the definition encompasses live vicarious learning, e.g., students watching another student at the front of the class interacting with the teacher. <br />
<br />
When manipulated, this variable often involves a contrast with<br />
* different kinds of learning situation being observed, e.g., a problem being solved by an instruction (e.g., Chi, Roy & Hausmann, in press; Craig et al. 2000; Craig, et al. 2006; Driscoll et al. 2003; [[Craig questions|PSLC project example]]) , or<br />
* different kinds of dyadic instruction, e.g., being a tutee. (Chi, Roy & Hausmann, in press; Craig et al., 2004; [[Craig observing|PSLC project example]] )<br />
<br />
"Learning by observing" is a somewhat broader term. <br />
<br />
References<br />
* Bandura, A. (1962). Social learning through imitation. In M. R. Jones (Ed.), Nebraska symposium of motivation (pp. 211-269). Lincoln: University of Nebraska Press. <br />
* Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.<br />
* Chi, M. T. H., Roy, M., & Hausmann, R. G. M. (in press). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Craig, S., D., Gholson B., Ventura, M., Graesser, A. C., & the Tutoring Research Group. (2000). Overhearing dialogues and monologues in virtual tutoring sessions: effects on questioning and vicarious learning. International Journal of Artificial Intelligence in Education (Special Issue: Analyzing Educational Dialogue Interaction), 11, 242-253. <br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Rosenthal, R. L., & Zimmerman, B. J. (1978). Social learning and cognition. New York: Academic Press. <br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5208Deep-level question2007-05-23T19:27:48Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes (Craig, et al. 2006; Gholson & Craig, 2006). This would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy (Bloom, 1956). Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers ([[Post-practice reflection (Katz)|PSLC project example]]), some deep-level questions are rhetorical in that students are not able to enter answers([[Craig questions|PSLC project example]]). <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
* Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives. Handbook 1: Cognitive domain. New York: McKay. <br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137. <br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* McNamara, D. S., & Kintsch, W. (1996). Learning from text: Effects of prior knowledge and text coherence. Discourse Processes, 22, 247-288.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5207Deep-level question2007-05-23T19:22:07Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes (Craig, et al. 2006; Gholson & Craig, 2006). This would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy (Bloom, 1956). Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers ([[Post-practice reflection (Katz)|PSLC project example]]), some deep-level questions are rhetorical in that students are not able to enter answers([[Craig questions|PSLC project example]]). <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
* Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives. Handbook 1: Cognitive domain. New York: McKay. <br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137. <br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5206Deep-level question2007-05-23T19:21:31Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes (Craig, et al. 2006; Gholson & Craig, 2006). This would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy (Bloom, 1956). Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers ([[Post-practice reflection (Katz)|PSLC project example]]), some deep-level questions are rhetorical ([[Craig questions|PSLC project example]]) in that students are not able to enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
* Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives. Handbook 1: Cognitive domain. New York: McKay. <br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137. <br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5205Deep-level question2007-05-23T19:19:56Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes (Craig, et al. 2006; Gholson & Craig, 2006). This would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy (Bloom, 1956). Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Post-practice reflection (Katz)|PSLC project example]], some deep-level questions are rhetorical [[Craig questions|PSLC project example]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
* Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives. Handbook 1: Cognitive domain. New York: McKay. <br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Graesser, A. C., & Person, N. K. (1994). Question asking during tutoring. American Educational Research Journal, 31, 104-137. <br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5203Deep-level question2007-05-23T19:04:07Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Post-practice reflection (Katz)|PSLC project example]], some deep-level questions are rhetorical [[Craig questions|PSLC project example]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5201Deep-level question2007-05-23T19:02:05Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Post-practice_ reflection_ Katz|PSLC project example]], some deep-level questions are rhetorical [[Craig questions|PSLC project example]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5200Deep-level question2007-05-23T19:01:48Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Post-practice_reflection_Katz|PSLC project example]], some deep-level questions are rhetorical [[Craig questions|PSLC project example]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5199Deep-level question2007-05-23T19:01:03Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Post-practice reflection Katz|PSLC project example]], some deep-level questions are rhetorical [[Craig questions|PSLC project example]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5197Deep-level question2007-05-23T19:00:20Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Post-practice_reflection_%28Katz%2|PSLC project example]], some deep-level questions are rhetorical [[Craig questions|PSLC project example]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5196Deep-level question2007-05-23T18:51:26Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy (e.g. causal antecedent, causal consequence, comparison, & interpretation) and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Katz]], some deep-level questions are rhetorical [[Craig]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Deep-level_question&diff=5195Deep-level question2007-05-23T18:47:24Z<p>Scraig@pitt.edu: </p>
<hr />
<div>Deep-level question: This is a method of instruction wherein questions are added to instruction that is otherwise able to function without them. The questions invite students to draw links between mechanisms, components or processes. These would encompass many of the long answer question categories of the Graesser & Person (1994) Taxonomy such as ... and the higher level categories of Bloom's taxonomy. Comprehension monitoring questions, text-base questions (McNamara & Kintch, 1996), content-free self-explanation prompts (Chi et al., 2001), are not included.<br />
<br />
Although deep-level questions often require students to type or otherwise express their answers [[Katz]], some deep-level questions are rhetorical [[Craig]] in that students are not able enter answers. <br />
<br />
When deep-level questions precede the original instruction, they are a kind of advance organizers (Ausubel, 1960). When they follow the original instruction, they are a kind of [[reflection questions]]. However, they can also be inserted in the midst of the original instruction. <br />
<br />
When this independent variable is manipulated, the contrast is often with the original instruction, which is often the [[ecological control]], that did not include the deep-level questions.<br />
<br />
<br />
References <br />
* Ausubel, D.P. (1960). The use of advance organizers in the learning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.<br />
<br />
[[Category:Glossary]]<br />
[[Category:Independent Variables]]<br />
[[Category:Interactive Communication]]<br />
[[Category:Craig questions]]</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_questions&diff=5194Craig questions2007-05-23T18:34:35Z<p>Scraig@pitt.edu: /* Independent variables */</p>
<hr />
<div>== Investigating the robustness of vicarious learning: Sense making with deep-level reasoning questions ==<br />
Scotty Craig, Kurt VanLehn, and Micki Chi''<br />
<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || 12-1-05<br />
|-<br />
| '''Study End Date''' || 8-1-06<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 17<br />
|-<br />
| '''Total Participant Hours''' || 24 hrs.<br />
|-<br />
| '''DataShop''' || Target date: June 15, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
Earlier work (Craig, at al. 2006; Gholson & Craig, in press) found that inserting relevant [[deep-level question]]s into observed video material both increased deep level question asking and improved learning. These lab studies had student learn topics in computer literacy by viewing videos of both monologues and dialogues; some material included [[deep-level question]]s, some included shallow questions and some included no questions. The conditions that included [[deep-level question]]s learned more than the others. However, it is not known how this method works compared to other methods for enhancing learning from observed materials (e.g. prompting for [[self-explanation]]). It is also not known if this effect can be useful for learning outside the lab setting. <br />
<br />
Our [[in vivo experiment]] presented identical core content on magnetism using example problems from the Andes tutoring system in three different ways. The material was presented in three formats. All three of these formats were presented as a video of a [[worked examples|worked example]] with each step corresponding to a [[knowledge component]]. The [[knowledge components]] were preceded by a [[deep-level question]] (e.g. What are the implications of having the magnetic field close to an electrified wire?), a prompt for learners to reflection on the material (i.e. a pause in the video) or a [[self-explanation]] prompt (e.g. Please begin your self explanation). Measures of [[Andes]] transfer, and long term [[robust learning]] were measured. The learners’ interaction with Andes were coded for differences on completion time, within task behavior, and the completion rates of the Andes homework.<br />
<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a difference found in the amount of time needed to complete homework. This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition. However, both of these findings are difficult to interprete given that there was an average of 39 days between initial training and homework completion by the learners.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig questions|Craig deep-level questions Glossary]]<br />
<br />
=== Research question ===<br />
Is robust learning better achieved by observing multimedia displays integrated with [[deep-level question]]s, prompts for [[reflection questions|reflection]], or [[self-explanation]]?<br />
<br />
=== Independent variables ===<br />
The current study varied the level of guidance provided. The level of guidance was varied by presenting students with a deep-level reasoning condition, a self-explanation condition and a reflection condition. The deep-level reasoning questions provided a step-by-step guide that scaffolded the learner during the learning process. The self-explanation condition asked that students build the links of these scaffolds by self-explaining the steps. As a control for time on task, the reflection condition presented materials to the participants with a pause before each step.<br />
<br />
'''Examples for each condition'''<br />
{| border="1" cellspacing="0" cellpadding="0" style="text-align: left;"<br />
| <br />
|-<br />
| ''Deep-level question'' || ''Self Explanation'' || ''Reflection''<br />
|-<br />
| What effect does a straight current-carrying wire have on magnetic field lines? || Please begin your self-explanation || Pause for 10 seconds<br />
|-<br />
| ''Corresponding Example text''<br />
|-<br />
| Magnetic field lines near a straight current-carrying wire take the form of <br />
concentric circles with the wire at their center <br />
|}<br />
<br><br />
<br />
=== Hypothesis ===<br />
A guided learning hypothesis would predict that since the deep-level questions provided a constant cognitive guide the deep-level question condition would improve learning over the reflection condition and possibly the self-explanation condition if the students could not produce the guidance while producing the self-explanations. Alternatively, a content equivalency hypothesis would be that since all three conditions provide the same content they should all produce learning of the material (Klahr & Nigam, 2004).<br />
<br />
=== Dependent variables ===<br />
<br />
* ''[[Long-term retention]], homework on Andes'': After training, students did their regular homework problems using Andes. Students could do them whenever they wanted, but most students normally completed them just before the exam (''M'' = 39 days after training). The more similar homework problems (near transfer) were analyzed.<br />
<br />
=== Results ===<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a marginally significant trend found on the completion time data in favor of participants in the deep-level question condition over those in the reflection condition (t (9) = 2.14, p = .07). This difference for completion time became significant when participants in the two unguided conditions were collapsed and compared against participants in the guided condition (t (15) = 2.41, p < .05). This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition.<br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met: <br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the deep-level question condition and the self-explanation condition could comprise the learning-by-doing paths in that learners are guided to produce clearer mental models of the material. Alternatively the participants in the reflection condition only received pauses during the presentation, thus these participants were not guided to produce better mental models. These participants relied more on the video to provide relevant links for them instead of actively constructing these links.<br />
* The student should take the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with deep-level questions that guide them in building better mental models. However, the self-explanation and reflection conditions require the students to produce the learning by doing path. In these conditions, if the production becomes too difficult for the students then they will not learn. This study is testing whether students will learn more by being encouraged to take a learning-by-doing path, via deep-level questions, than an alternative path. Since none of the students attempted more than a few self-explanations, it appears that the students in the self-explanation conditions did not take the learning-by-doing path.<br />
<br />
<br />
=== Annotated bibliography ===<br />
* Presented at LRDC Supergroup meeting July, 2006<br />
* Presented at PSLC Roadshow - Memphis November, 2006<br />
* Presented at LRDC Graduate student recruitment - Pittsburgh Feburary, 2007<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007, April). PSLC AERA Symposium: In vivo experimentation for understanding robust learning: Pros and cons.<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007). PSLC EARLI Symposium.<br />
<br />
=== References ===<br />
<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (under revision). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. ''Cognitive Science, 13'', 145-182.<br />
* Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. ''Cognitive Science, 18'', 439-477.<br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Use of Questions during learning<br />
* [[Reflective_Dialogues_%28Katz%29 | Reflective Dialogues (Katz)]]<br />
* [[Post-practice reflection (Katz) | Post-practice reflection (Katz)]]<br />
* [[FrenchCulture | FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]<br />
<br />
Self explanations during learning<br />
* [[Hausmann Study2 | The Effects of Interaction on Robust Learning (Hausmann & Chi)]]<br />
* [[Hausmann Study | A comparison of self-explanation to instructional explanation (Hausmann & Vanlehn)]]<br />
<br />
Learning by Observing<br />
* [[Craig_observing | Learning from Problem Solving while Observing Worked Examples (Craig, Gadgil, & Chi)]]<br />
--[[User:Scraig@pitt.edu|Scotty]] 12:12, 19 September 2006 (EDT)</div>Scraig@pitt.eduhttps://learnlab.org/wiki/index.php?title=Craig_questions&diff=5193Craig questions2007-05-23T18:34:16Z<p>Scraig@pitt.edu: /* Research question */</p>
<hr />
<div>== Investigating the robustness of vicarious learning: Sense making with deep-level reasoning questions ==<br />
Scotty Craig, Kurt VanLehn, and Micki Chi''<br />
<br />
=== Summary Table ===<br />
{| border="1" cellspacing="0" cellpadding="5" style="text-align: left;"<br />
| '''PI''' || Scotty Craig<br />
|-<br />
| '''Other Contributers''' || Robert N. Shelby (USNA), Brett van de Sande (Pitt)<br />
|-<br />
| '''Study Start Date''' || 12-1-05<br />
|-<br />
| '''Study End Date''' || 8-1-06<br />
|-<br />
| '''LearnLab Site''' || USNA<br />
|-<br />
| '''LearnLab Course''' || Physics<br />
|-<br />
| '''Number of Students''' || ''N'' = 17<br />
|-<br />
| '''Total Participant Hours''' || 24 hrs.<br />
|-<br />
| '''DataShop''' || Target date: June 15, 2007<br />
|}<br />
<br><br />
<br />
=== Abstract ===<br />
Earlier work (Craig, at al. 2006; Gholson & Craig, in press) found that inserting relevant [[deep-level question]]s into observed video material both increased deep level question asking and improved learning. These lab studies had student learn topics in computer literacy by viewing videos of both monologues and dialogues; some material included [[deep-level question]]s, some included shallow questions and some included no questions. The conditions that included [[deep-level question]]s learned more than the others. However, it is not known how this method works compared to other methods for enhancing learning from observed materials (e.g. prompting for [[self-explanation]]). It is also not known if this effect can be useful for learning outside the lab setting. <br />
<br />
Our [[in vivo experiment]] presented identical core content on magnetism using example problems from the Andes tutoring system in three different ways. The material was presented in three formats. All three of these formats were presented as a video of a [[worked examples|worked example]] with each step corresponding to a [[knowledge component]]. The [[knowledge components]] were preceded by a [[deep-level question]] (e.g. What are the implications of having the magnetic field close to an electrified wire?), a prompt for learners to reflection on the material (i.e. a pause in the video) or a [[self-explanation]] prompt (e.g. Please begin your self explanation). Measures of [[Andes]] transfer, and long term [[robust learning]] were measured. The learners’ interaction with Andes were coded for differences on completion time, within task behavior, and the completion rates of the Andes homework.<br />
<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a difference found in the amount of time needed to complete homework. This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition. However, both of these findings are difficult to interprete given that there was an average of 39 days between initial training and homework completion by the learners.<br />
<br />
=== Glossary ===<br />
See [[:Category:Craig questions|Craig deep-level questions Glossary]]<br />
<br />
=== Research question ===<br />
Is robust learning better achieved by observing multimedia displays integrated with [[deep-level question]]s, prompts for [[reflection questions|reflection]], or [[self-explanation]]?<br />
<br />
=== Independent variables ===<br />
The current study varied the level of guidance provided. The level of guidance was varied by presenting students with a deep-level reasoning condition, a self-explanation condition and a reflection condition. The deep-level reasoning questions provided a step-by-step guide that scaffolded the learner during the learning process. The self-explanation condition asked that students build the links of these scaffolds by self-explaining the steps. As a control for time on task, the reflection condition presented materials to the participants with a pause before each step.<br />
<br />
'''Examples for each condition'''<br />
{| border="1" cellspacing="0" cellpadding="0" style="text-align: left;"<br />
| <br />
|-<br />
| ''Deep question'' || ''Self Explanation'' || ''Reflection''<br />
|-<br />
| What effect does a straight current-carrying wire have on magnetic field lines? || Please begin your self-explanation || Pause for 10 seconds<br />
|-<br />
| ''Corresponding Example text''<br />
|-<br />
| Magnetic field lines near a straight current-carrying wire take the form of <br />
concentric circles with the wire at their center <br />
|}<br />
<br><br />
<br />
=== Hypothesis ===<br />
A guided learning hypothesis would predict that since the deep-level questions provided a constant cognitive guide the deep-level question condition would improve learning over the reflection condition and possibly the self-explanation condition if the students could not produce the guidance while producing the self-explanations. Alternatively, a content equivalency hypothesis would be that since all three conditions provide the same content they should all produce learning of the material (Klahr & Nigam, 2004).<br />
<br />
=== Dependent variables ===<br />
<br />
* ''[[Long-term retention]], homework on Andes'': After training, students did their regular homework problems using Andes. Students could do them whenever they wanted, but most students normally completed them just before the exam (''M'' = 39 days after training). The more similar homework problems (near transfer) were analyzed.<br />
<br />
=== Results ===<br />
Participants’ homework performance was investigated by looking at Andes homework scores and completion time data. There were no differences found on Andes homework scores among the three groups. However, there was a marginally significant trend found on the completion time data in favor of participants in the deep-level question condition over those in the reflection condition (t (9) = 2.14, p = .07). This difference for completion time became significant when participants in the two unguided conditions were collapsed and compared against participants in the guided condition (t (15) = 2.41, p < .05). This significant differences represented a 55% savings in time to complete the problem for the participants in the deep-level questions condition.<br />
<br />
=== Explanation ===<br />
This study is part of the Interactive Communication cluster, and its hypothesis is a specialization of the IC cluster’s central hypothesis. The IC cluster’s hypothesis is that robust learning occurs when two conditions are met: <br />
* The learning event space should have paths that are mostly learning-by-doing along with alternative paths where a second agent does most of the work. In this study, the deep-level question condition and the self-explanation condition could comprise the learning-by-doing paths in that learners are guided to produce clearer mental models of the material. Alternatively the participants in the reflection condition only received pauses during the presentation, thus these participants were not guided to produce better mental models. These participants relied more on the video to provide relevant links for them instead of actively constructing these links.<br />
* The student should take the learning-by-doing path unless it becomes too difficult. This study attempts to control the student’s path choice by presenting them with deep-level questions that guide them in building better mental models. However, the self-explanation and reflection conditions require the students to produce the learning by doing path. In these conditions, if the production becomes too difficult for the students then they will not learn. This study is testing whether students will learn more by being encouraged to take a learning-by-doing path, via deep-level questions, than an alternative path. Since none of the students attempted more than a few self-explanations, it appears that the students in the self-explanation conditions did not take the learning-by-doing path.<br />
<br />
<br />
=== Annotated bibliography ===<br />
* Presented at LRDC Supergroup meeting July, 2006<br />
* Presented at PSLC Roadshow - Memphis November, 2006<br />
* Presented at LRDC Graduate student recruitment - Pittsburgh Feburary, 2007<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007, April). PSLC AERA Symposium: In vivo experimentation for understanding robust learning: Pros and cons.<br />
* Presented at VanLehn, K., Hausmann, R., & Craig, S. (2007). PSLC EARLI Symposium.<br />
<br />
=== References ===<br />
<br />
* Chi, M. T. H., Hausmann, R. G. M., & Roy, M. (under revision). Learning from observing tutoring collaboratively: Insights about tutoring effectiveness from vicarious learning. ''Cognitive Science.'' <br />
* Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. ''Cognitive Science, 13'', 145-182.<br />
* Chi, M. T. H., de Leew, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. ''Cognitive Science, 18'', 439-477.<br />
* Craig, S. D., Driscoll, D., & Gholson, B. (2004). Constructing knowledge from dialog in an intelligent tutoring system: Interactive learning, vicarious learning, and pedagogical agents. ''Journal of Educational Multimedia and Hypermedia, 13'', 163-183. [http://andes3.lrdc.pitt.edu/~scraig/publications/Craigetal2004VL.pdf]<br />
* Craig, S. D., Sullins, J., Witherspoon, A. & Gholson, B. (2006). Deep-Level Reasoning Questions effect: The Role of Dialog and Deep-Level Reasoning Questions during Vicarious Learning. ''Cognition and Instruction, 24(4)'', 565-591.<br />
* Gholson, B. & Craig, S. D. (2006). Promoting constructive activities that support vicarious learning during computer-based instruction. ''Educational Psychology Review, 18'', 119-139. [http://andes3.lrdc.pitt.edu/~scraig/publications/Gholson&Craig2006.pdf]<br />
* Klahr, D. & Nigam, M. (2004). The equivalence of learning paths in early science instruction: Effects of direct instruction and discovery learning. ''Psychological Science, 15'', 661-667.<br />
<br />
=== Connections ===<br />
This project shares features with the following research projects:<br />
<br />
Use of Questions during learning<br />
* [[Reflective_Dialogues_%28Katz%29 | Reflective Dialogues (Katz)]]<br />
* [[Post-practice reflection (Katz) | Post-practice reflection (Katz)]]<br />
* [[FrenchCulture | FrenchCulture (Amy Ogan, Christopher Jones, Vincent Aleven)]]<br />
<br />
Self explanations during learning<br />
* [[Hausmann Study2 | The Effects of Interaction on Robust Learning (Hausmann & Chi)]]<br />
* [[Hausmann Study | A comparison of self-explanation to instructional explanation (Hausmann & Vanlehn)]]<br />
<br />
Learning by Observing<br />
* [[Craig_observing | Learning from Problem Solving while Observing Worked Examples (Craig, Gadgil, & Chi)]]<br />
--[[User:Scraig@pitt.edu|Scotty]] 12:12, 19 September 2006 (EDT)</div>Scraig@pitt.edu